I did not figure it out. Would you please post details @Av ?
I know where the issue lies, though, it's a difference between the view provided by "Left Eye" vs the full image rendered to the headset.
I have made a simple example, where the camera coordinates of an object are printed directly on screen. The project is attached (EyeTrackingIssue.zip), but the full code is the following:
using UnityEngine;
using UnityEngine.UI;
using UnityEngine.XR;
public class DisplaySSCoordinates : MonoBehaviour
{
public Transform debugSphere;
public Text displayText;
public Camera vrCamera;
private void Update()
{
var coords = vrCamera.WorldToScreenPoint(debugSphere.position, Camera.MonoOrStereoscopicEye.Left);
var eyeTextureSize = new Vector2(XRSettings.eyeTextureWidth, XRSettings.eyeTextureHeight);
var units = new Vector2(coords.x / eyeTextureSize.x, coords.y / eyeTextureSize.y);
displayText.text = $"({units.x:F2},{units.y:F2})";
}
}
You can see that in the bottom-left and top-right corners, the coordinates are not what I would expect. For the center it is correct:
However when I switch from the "LeftEye" to "OcclusionMesh", the coordinates are correct:
This is obviously not SRanipal issue only, probably will also happen for other EyeTracking systems, however I think this is the prescribed way of getting the screen-space position (it's a while I'm not sure, I'm happy for a different workflow).
I noticed that the error changes with the aspect ratio, which makes sense as the screen-crop of the "LeftEye" changes too.
I'd need to obtain the information how the "EyeTexture" is cropped into the game view to calculate the position correctly. Any recommendations?
EyeTrackingIssue.zip