Getting started with VR eye tracking development for research

[Djangi]

Hi, 

I'm a game developer at the Utrecht University in the Netherlands where I build VR applications/prototypes for research. The Vive Pro Eye and the Pupil Labs Binocular Add-on for the Vive are currently on my radar.
I'm a especially interested in the Vive Pro Eye since the software/sdk seems to be more mature but that is merely based on assumption and bias, I need to info to back this up.

While I've read the documentation on the Vive Developer and tobii xr sdk pages, I still have many unanswered questions. I tried to contact HTC(support and sales) and Tobii (xr development and sales) but have yet to receive a reply.

For context, I'm building a environment in Unity that is experienced from a seated position. I want to tracks pupil size and gaze(both timestamped) , I also want to measure saccadic eye movement.I'm currently researching VR eye tracking solutions for an upcoming project.

My questions:

1. What is the difference between Vive Eye Tracking SDK (SRanipal) and the tobii xr sdk. While tobii published a clear api reference, I can't find something similar about SRanipal
2. Are there events that indicate the start and end of saccadic eye movement or are is the prefabricated code available to measure this?
3. How does does the data flow work from the integrated eye tracker into unity, is all raw data available in Unity or do I get data that is filtered for in game performance. Also, is the data directly fed into the unity plugin or does it run through external software like with other professional tobii products.
4. If I want to record data from offline analysis, or do I have build some sort of logger or is there a function in the sdk. Also, is the eye tracking data only available in the plugin or can I also access it externally in windows.
5. I've recently read an article on slippage: https://link.springer.com/article/10.3758/s13428-019-01307-0 
   The Tobii glasses seem to perform better than the competition in this matter, I'm wondering how far this issue is effecting the HTC Vive Pro Eye.
6. This question might seem a bit naive. Is there an active HTC Vive Pro Eye and/or VR Eye tracking community. So far I've only encountered the Pupil labs discord(which is not very active), the tobii xr support forum and this forum. I hope there are more sources out there.

I hope someone here is able to answer my questions. 

Thanks in advance, I'm looking forward to our input.

@Corvus

 

 

 

[Djangi]

Hi @Corvus,

Since you answer a lot of the questions here, any change that you can help me out?

Thanks.

[Corvus]

@Djangi I'm happy to help you out and answer these questions. Let me know if you need any more information or details.

     What is the difference between Vive Eye Tracking SDK (SRanipal) and the tobii xr sdk. While tobii published a clear api reference, I can't find something similar about SRanipal
- Feel free to download the SRanipal SDK and the api documentation is included with it. The main differences have to do with pricing, access to raw camera images, and licensing terms where the Tobii SDK requires a special license for analytics usages.

https://developer.vive.com/resources/knowledgebase/vive-sranipal-sdk/

    Are there events that indicate the start and end of saccadic eye movement or are is the prefabricated code available to measure this?
- No, the SRanipal SDK does not provide functions for eye saccades.

    How does does the data flow work from the integrated eye tracker into unity, is all raw data available in Unity or do I get data that is filtered for in game performance. Also, is the data directly fed into the unity plugin or does it run through external software like with other professional tobii products.
- You will get filtered data in Unity via the SRanipal Runtime software.

    If I want to record data from offline analysis, or do I have build some sort of logger or is there a function in the sdk. Also, is the eye tracking data only available in the plugin or can I also access it externally in windows.
- There is no function in the SDK but there are samples available on the forums. Please check this link to see an example of recording the data at 120hz.
https://forum.vive.com/topic/5897-getting-verbosedata-at-the-fastest-rate-possible/?tab=comments#comment-26428

    I've recently read an article on slippage: https://link.springer.com/article/10.3758/s13428-019-01307-0
    The Tobii glasses seem to perform better than the competition in this matter, I'm wondering how far this issue is effecting the HTC Vive Pro Eye.
- I will review this article, thanks for sharing it. FYI, we have partnered with Tobii for the eye tracking and SDK so should expect similar performance.

    This question might seem a bit naive. Is there an active HTC Vive Pro Eye and/or VR Eye tracking community. So far I've only encountered the Pupil labs discord(which is not very active), the tobii xr support forum and this forum. I hope there are more sources out there.
- Eye tracking and VR eye tracking are still relatively new technology and niche communities. If you find any more useful communities or resources please feel free to share them here. 😃

 

[Djangi]

@Corvus Thank you for your answers

I do have a few additional questions, some are specifically aimed at the Tobii XR SDK:

1. It seems that TobiiXR_EyeTrackingData contains a combined gaze ray for both eyes, I would also like to know the gaze/eye position for each individual eye, is this possible?
2. According to the Vive Pro Eye product page the pupil position and size should be available in the data output, yet they are not described in the Tobii XR SDK api reference page. How do I access pupil output and is this part of the Tobii XR SDK or SRanipal?
3. What is the latency of the eyetracker and how is this effected when used with the wireless add-on. I need to know the specifics on this one.
4. What are the costs for a Tobii Analytical License.
5. Does a Tobii Analytical License. come with any benefits (analysis software, addition data options, tools)?

Thanks in advance, I know I'm asking a lot. I'ts very hard to get past the available marketing info and get some deeper information. I really appreciate it.
Also note that I also reached out to Tobii but have yet to receive a reply.

[Corvus]

@Djangi For the Tobii XR SDK questions you need to reach out to Tobii directly or maybe @Brand can assist you.

We do not have official latency specs to share. There was some independent results shared on the forum you can find here but the latency and filters have improved with the v2 version available with the "SR_runtime v1.1.2.0 " release.

https://forum.vive.com/topic/5970-pro-eye-unsuitable-for-research-data-super-heavily-filtered-during-fixation/?do=findComment&comment=27437

 

[avi]

@Corvus

I am unable to get the eye tracking data (pupil dia) from the Vive Pro Eye. I am using SRanipal SDK 1.1.0.1, TobiiXR SDK 1.7.0.160 and Unity 2018.3.14. I using various code segment from the community. It always show pupil dia as 0. Can you please help? Also tagging @imarin18, as I have been using his code segment. Imarin18 can you please help as well?

Thanks

Avi

[Djangi]

On 1/23/2020 at 6:25 PM, Corvus said:

@Djangi For the Tobii XR SDK questions you need to reach out to Tobii directly or maybe @Brand can assist you.

We do not have official latency specs to share. There was some independent results shared on the forum you can find here but the latency and filters have improved with the v2 version available with the "SR_runtime v1.1.2.0 " release.

https://forum.vive.com/topic/5970-pro-eye-unsuitable-for-research-data-super-heavily-filtered-during-fixation/?do=findComment&comment=27437

 

Thanks for your feedback. Might have additional questions in the future since there should be Vive Pro Eye on it's way.

Thanks again.

[Djangi]

It's been a while. I decided to go with the Vive Pro Eye. I'm currently trying to setup a prototype  and I'm figuring out how to get pupil diameter.
I might occasionally check in with remarks/question during the following weeks.

[Corvus]

@Djangi You can get the pupil diameter with the "pupil_diameter_mm" api. More information is available in the SDK documentation.

float ViveSR.anipal.Eye.SingleEyeData.pupil_diameter_mm

 

[Djangi]

On 4/6/2020 at 6:20 PM, Corvus said:

@Djangi You can get the pupil diameter with the "pupil_diameter_mm" api. More information is available in the SDK documentation.

float ViveSR.anipal.Eye.SingleEyeData.pupil_diameter_mm

 

Thanks @corvus for your input. Maybe this is because of lack of experience but I actually had a pretty hard time to access the wanted parameters since you can't access the wanted directly. It would be nice to have some code examples in the documentation.
I did the following:

private void Update()
{
    SRanipal_Eye_API.GetEyeData(ref data);
    Debug.Log("left eye pupil: " + data.verbose_data.left.pupil_diameter_mm);
    Debug.Log("right eye pupil: " + data.verbose_data.right.pupil_diameter_mm);
}

I'll also try to get this running at 120hz with the example code from the thread that you shared with me previously.

Btw, is there also an online reference to the documentation?

Edited April 8, 2020 by Djangi
words