Eye Tracking Data Export to CSV in Unreal

[cte]

Application:

• Academic Research

Goals:

• Install SDK -----------------------------  [ X ]
• Get Eye Gaze -------------------------   [     ]
• Get Fixation ---------------------------   [     ]
• Get Pupil Dilation -------------------   [     ]
• Run Subjects & Get Tenure -----  [     ]

Question:

• How do I reference the SDK's framework / API to extract close to real-time eye tracking data that prints either in a data.frame or CSV file.

-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Hi there, I've been able to get the VR and eye tracking set up and working per a previous thread with UE 4.25.3 and have a demo with VR functional.

The only thing is I'm not sure where to go from here to get the SDK to write/print data to a csv or similar file.

Presently I don't need any eye tracking interactions within the VR environment so the dartboards and mannequin head are useful in that they show this data exists but I need to get that data out of whatever loop it is in and write it to a data file for processing in statistical programs and the like.

@MariosBikos_HTC , you've been a great help so far. Let me know if you or another HTC fellow are the right ones to ask about this function.

Once I get something functional I'll definitely be sharing it for future folks in my position.

 

[cte]

@Corvus would you perhaps be willing to help out with this low-level explanation of how to get at the SDK's data? I'm new to C++ and compiler languages in general so don't know how to call the API (if that is even how this works).

[cte]

:: UPDATE :: 10-07-20 ::

Application:

• Academic Research

Goals:

• Install SDK -----------------------------  [ X ]
• Get Eye Gaze -------------------------   [ X ]
• Get Fixation ---------------------------   [     ]
• Get Pupil Dilation -------------------   [     ]
• Run Subjects & Get Tenure -----  [     ]

Question:

• How do I reference the SDK's framework / API to extract close to real-time eye tracking data that prints either in a data.frame or CSV file.

---------------------------------------------------------------------------------------------------

I've given up (more or less) on getting this to work in UE4 so tried out the sample code for _C in the SDK and had a bit more success getting data.

I can now get gaze data from the sample project in some fashion but have to figure out now how to get dilation data which might mean I do a similar kind of thing to what @MariosBikos_HTC did here:

I'm not sure how to wrap my head around this function library code but am assuming this is a type of blueprint class reference in unreal that I can recycle for calculating the same information and providing it to me in a string format in C.

Not sure about this though since I'm not much a C programming language user but I will try to create a new header and call the function in the .cpp file such that it will  show up as a keyed option "7" to start the string and "8" to stop it. Doing so, I think I can just mirror the existing gaze data keys but again, I'm not really sure I know exactly what I'm doing so appreciate any guidance in this process.

 

[cte]

I was able to get this to work now in UE4 and what it took was learning a bit about actors and blueprints.

To get gaze you can create a new actor and place it in the scene, then create a blueprint on that actor.

For getting gaze data I produced the following blueprint:

A key feature of this blueprint was the File IO Save String Arrray which I got from a function library plugin called "Victory Plugin" which you can get here: https://www.mediafire.com/file/pgrpsf9my7tv1ae/VictoryPlugin25.zip/file

Though this doesn't solve my pupil dilation problem, I can probably get that data via a functionb posted in a different forum:

I'll be trying that code next and creating a new actor and blueprint for that particular function.

The data I'm getting now looks like this:

From the CSV file. I'd like to get this in a different format where things are columns rather than rows but I can probably work wiht this data and manipulate it to that effect in r or something I know better than C++ functions.

Anyway, that is the update on the extracting data from SRanipal for eye direction, fixation, and confidence value with a unix timestamp.

Would love to get a few thoughts on how to improve this blueprinting setup.

-CTE

 

[cte]

47 minutes ago, cte said:

I was able to get this to work now in UE4 and what it took was learning a bit about actors and blueprints.

To get gaze you can create a new actor and place it in the scene, then create a blueprint on that actor.

For getting gaze data I produced the following blueprint:

A key feature of this blueprint was the File IO Save String Arrray which I got from a function library plugin called "Victory Plugin" which you can get here: https://www.mediafire.com/file/pgrpsf9my7tv1ae/VictoryPlugin25.zip/file

Though this doesn't solve my pupil dilation problem, I can probably get that data via a functionb posted in a different forum:

I'll be trying that code next and creating a new actor and blueprint for that particular function.

The data I'm getting now looks like this:

From the CSV file. I'd like to get this in a different format where things are columns rather than rows but I can probably work wiht this data and manipulate it to that effect in r or something I know better than C++ functions.

Anyway, that is the update on the extracting data from SRanipal for eye direction, fixation, and confidence value with a unix timestamp.

Would love to get a few thoughts on how to improve this blueprinting setup.

-CTE

 

UPDATED Blueprint :: Found the Join String Array which Makes a significantly easier to work with CSV file. 

New Blueprint Structure:

Leads to new output in csv file:

[Jori1110]

Hi 

How are you going with achieving the other goals in your list?

I am just getting started using SRanipal and UNE4 also for academic research.  Id be very keen to hear your learnings if you are willing to chat?

 

[woody loks]

@cte kudos for your effort. I tried to implement the same and i am getting the x,y,z values for left eye and right eye directions in the 2nd and 1st column respectively as shown in the image attached below.

On 10/9/2020 at 7:54 PM, cte said:

Anyway, that is the update on the extracting data from SRanipal for eye direction, fixation, and confidence value with a unix timestamp.

But do you know why i am getting the fixation and confidence values (3rd and 4th column values) as zero every time? 

On 10/9/2020 at 8:43 PM, cte said:

UPDATED Blueprint :: Found the Join String Array which Makes a significantly easier to work with CSV file. 

I implemented the above blueprint in the sample project ("EyeSample") that came with vive SRanipal sdk zip folder. (the one which has DART board and a human head to track our eye)

Were you able to get some data other than zero for fixation and confidence values?  If yes, pls share your insights on it. Thanks in advance!!

 

Have a great day!

[NikP]

On 10/10/2020 at 1:13 AM, cte said:

UPDATED Blueprint :: Found the Join String Array which Makes a significantly easier to work with CSV file. 

New Blueprint Structure:

Leads to new output in csv file:

When using Join String array, i get all the data in a single cell, separated by a comma.

How did you get it to go into different columns?

Edited July 5, 2023 by NikP
Issue with Excel. - Fixed. Went to Data Tab, and Selected from Text/CSV