Determine vergence point on HL2

67 Views Asked by At

Setup: HoloLens 2, front camera

Input data: Gaze origin, gaze direction of user, camera intrinsics

Desired output: 2D gaze coordinates

Objective: I would like to determine 2D gaze coordinates on the camera projection plane of the front camera of the HoloLens 2. For that, I have placed a virtual camera with representative camera intrinsics of the physical front camera in the scene in Unity3D. Given the gaze origin and direction I could cast a ray into the scene and look for intersection points with the camera projection plane. However, at what depth/distance should I do that? I am not looking for intersections for certain objects but gaze coordinates at all points in time.

Current approach:

  1. Use Extended Eye Tracking (EET) SDK for Unity3D
  2. Retrieve vergence point
  3. Convert 3D point into screen space of virtual camera

Problem: Can't get a valid vergence distance from EET SDK using following approach. TryGetVergenceDistance() always returns false. GazeDirection & GazeOrigin are valid.

Getting gaze data as follows:

var combinedGazeReadingInWorldSpace = extendedEyeGazeDataProvider.GetWorldSpaceGazeReading(ExtendedEyeGazeDataProvider.GazeType.Combined);

Modified ExtendedEyeGazeDataProvider script, contained method GetWorldSpaceGazeReading and GazeReading class to return vergence distance as follows:

    public class GazeReading
    {
        public Vector3 EyePosition;
        public Vector3 GazeDirection;
        // Added: Vergence distance
        public float VergenceDistance;

        public GazeReading() { }
        public GazeReading(Vector3 position, Vector3 direction)
        {
            EyePosition = position;
            GazeDirection = direction;
        }
    }
public GazeReading GetWorldSpaceGazeReading(GazeType gazeType, DateTime timestamp)
{
    ...
    ...

    // Added: Vergence distance added to gaze reading
    bool vd_valid = _eyeGazeTrackerReading.TryGetVergenceDistance(out float vd);
    _gazeReading.VergenceDistance = vd_valid ? vd : 1;

    return _gazeReading;
}

EDIT:

It seems like vergence distance is not supported on HL2. Can anyone confirm? _eyeGazeTracker.IsVergenceDistanceSupported returns false on HL2.

0

There are 0 best solutions below