How do you record video and eyetracking data simultaneously in Unity?

[ryankirkpatrick]

Specifically, I need to know where my participants will be looking at a precise time, so the video and the samples from the EyeData object must be perfectly aligned in time. I am also wondering what the equation is to calculate where the participant is looking from the left and right gaze_origins and the left and right gaze_directions or if there is a better way than trying to compute where the participant is looking in VR from those 4 sets of values. @Corvus @Daniel_Y

[Daniel_Y]

Not sure what video means. Would you describe your use scenario?

Have you tried Focus sample enclosed in SRanipal Unity plugin?

 

@ryankirkpatrick

[ryankirkpatrick]

By video, I mean the virtual scene that is presented to the participant on the HTC VIVE PRO head mounted display or what the participant sees when they place the headset on their head.

My use scenario is as follows: 

- Using the Unity program, participants view a virtual scene with the HTC VIVE PRO for roughly 10 minutes.

- I need to know the coordinates of each location which the participant focuses their eyes upon in this virtual environments at the precise time they focus their gaze on that location. 

- To do this, I need some way to record both the virtual scene presented on the HTC VIVE PRO and the Eye Data (the gaze origin, gaze direction, pupil diameter, etc.) such that I know where the participant is focusing their eyes in the virtual environment.

By "Focus Sample", are you referring to the SRanipal_EyeFocusSample.cs script? That script does not meet my needs since it does not save out the EyeData to a file nor does it save out the virtual scene the participant is viewing. 

[Corvus]

@ryankirkpatrick For recording the eye data please check this forum post with sample code:
https://forum.vive.com/topic/5897-getting-verbosedata-at-the-fastest-rate-possible/?tab=comments#comment-26428

 

For recording video there are a number of solutions available on the Unity Asset store or GitHub. Here is a popular choice:

https://github.com/keijiro/FFmpegOut

 

[kkelly2024]

Hi @ryankirkpatrick, did you figure out how to do this? I need to do something similiar as well with the HTC VIVE Eye Pro and Unity and could use some tips. Any help would be greatly appreciated!

Edited February 7 by kkelly2024