[How to get which pixel of frame the user is looking at]

I also would like to know the answer to this question. I am saving to a .mp3 file the recording of the environment presented to the participant using Unity Recorder and I am also saving out every single variable using the SRanipal_Eye_Framework at 120 hertz to a .txt file. I also need to know exactly which pixel of the current frame the user fixates upon.

 

@Corvus @Daniel_Y @zzy

 

[How do you record video and eyetracking data simultaneously in Unity?]

By video, I mean the virtual scene that is presented to the participant on the HTC VIVE PRO head mounted display or what the participant sees when they place the headset on their head.

My use scenario is as follows: 

- Using the Unity program, participants view a virtual scene with the HTC VIVE PRO for roughly 10 minutes.

- I need to know the coordinates of each location which the participant focuses their eyes upon in this virtual environments at the precise time they focus their gaze on that location. 

- To do this, I need some way to record both the virtual scene presented on the HTC VIVE PRO and the Eye Data (the gaze origin, gaze direction, pupil diameter, etc.) such that I know where the participant is focusing their eyes in the virtual environment.

By "Focus Sample", are you referring to the SRanipal_EyeFocusSample.cs script? That script does not meet my needs since it does not save out the EyeData to a file nor does it save out the virtual scene the participant is viewing. 

[How do you record video and eyetracking data simultaneously in Unity?]

Specifically, I need to know where my participants will be looking at a precise time, so the video and the samples from the EyeData object must be perfectly aligned in time. I am also wondering what the equation is to calculate where the participant is looking from the left and right gaze_origins and the left and right gaze_directions or if there is a better way than trying to compute where the participant is looking in VR from those 4 sets of values. @Corvus @Daniel_Y