Setup: HoloLens 2, front camera
Input data: Gaze origin, gaze direction of user, camera intrinsics
Desired output: 2D gaze coordinates
Objective: I would like to determine 2D gaze coordinates on the camera projection plane of the front camera of the HoloLens 2. For that, I have placed a virtual camera with representative camera intrinsics of the physical front camera in the scene in Unity3D. Given the gaze origin and direction I could cast a ray into the scene and look for intersection points with the camera projection plane. However, at what depth/distance should I do that? I am not looking for intersections for certain objects but gaze coordinates at all points in time.
Current approach:
- Use Extended Eye Tracking (EET) SDK for Unity3D
- Retrieve vergence point
- Convert 3D point into screen space of virtual camera
Problem: Can't get a valid vergence distance from EET SDK using following approach. TryGetVergenceDistance() always returns false. GazeDirection & GazeOrigin are valid.
Getting gaze data as follows:
var combinedGazeReadingInWorldSpace = extendedEyeGazeDataProvider.GetWorldSpaceGazeReading(ExtendedEyeGazeDataProvider.GazeType.Combined);
Modified ExtendedEyeGazeDataProvider script, contained method GetWorldSpaceGazeReading and GazeReading class to return vergence distance as follows:
public class GazeReading
{
public Vector3 EyePosition;
public Vector3 GazeDirection;
// Added: Vergence distance
public float VergenceDistance;
public GazeReading() { }
public GazeReading(Vector3 position, Vector3 direction)
{
EyePosition = position;
GazeDirection = direction;
}
}
public GazeReading GetWorldSpaceGazeReading(GazeType gazeType, DateTime timestamp)
{
...
...
// Added: Vergence distance added to gaze reading
bool vd_valid = _eyeGazeTrackerReading.TryGetVergenceDistance(out float vd);
_gazeReading.VergenceDistance = vd_valid ? vd : 1;
return _gazeReading;
}
EDIT:
It seems like vergence distance is not supported on HL2. Can anyone confirm? _eyeGazeTracker.IsVergenceDistanceSupported returns false on HL2.