Jump to content

Pupil dilation in mm vs. eye openess parameter


Recommended Posts

Hi,

I am trying to write a blink/missing data interpolation algorithm for the recorded pupil dilation in mm. To this extent, I was thinking of using the EyeOpeness parameter to identify when the pupil was not visible by the HMD and interpolate those specific values.

However, when I plot the pupil size in mm (BLUE line) and the eye openess parameter (ORANGE line) on top of each other, there's a big chunck of missing data in the middle for which the pupil diameter is -1 (which means missing, I guess) but the eye openess parameter shows 1 (which means the eye is completely opened). This seems weird to me and makes the Eye Openess parameter useless, or am I missing something here?

Thank you!

 

Figure_1.png

Link to post
Share on other sites

Dear @kbombeke

 

I observed the similar situation previously. I recommend using the information of eye validity first.

 

image.png.72efad528c0d1ca7315996383c5f1480.png

 

image.png.a408ceaf16047e736ba89c2354cf653e.png

 

If I understand correctly, the following shows how it works, though I am not sure if this is correct.

Enumerator

Binary digit if valid

Decimal digit if valid

SINGLE_EYE_DATA_GAZE_ORIGIN_VALIDITY

00001

1

SINGLE_EYE_DATA_GAZE_DIRECTION_VALIDITY

00010

2

SINGLE_EYE_DATA_PUPIL_DIAMETER_VALIDITY

00100

4

SINGLE_EYE_DATA_EYE_OPENNESS_VALIDITY

01000

8

SINGLE_EYE_DATA_PUPIL_POSITION_IN_SENSOR_AREA_VALIDITY

10000

16

 

 

I hope this helps.

 

Best regards,

imarin18

 

Link to post
Share on other sites

Dear @imarin18,

Thank you for your fast reply. I will definitely look into this "eye validity" variable. We know for sure that the participant closed his eyes during this interval of missing data, so it's still weird that a parameter called "eye openess" returns the maximum value of 1 (meaning "eyes are open") for that entire interval.

I'll let you know when we find a solution.

Best regards,
Klaas

 

Link to post
Share on other sites
       public struct SingleEyeData
            {
                /** The bits containing all validity for this frame.*/
                public System.UInt64 eye_data_validata_bit_mask;
                /** The point in the eye from which the gaze ray originates in meter miles.(right-handed coordinate system)*/
                public Vector3 gaze_origin_mm;
                /** The normalized gaze direction of the eye in [0,1].(right-handed coordinate system)*/
                public Vector3 gaze_direction_normalized;
                /** The diameter of the pupil in meter miles*/
                public float pupil_diameter_mm;
                /** A value representing how open the eye is.*/
                public float eye_openness;
                /** The normalized position of a pupil in [0,1]*/
                public Vector2 pupil_position_in_sensor_area;

                public bool GetValidity(SingleEyeDataValidity validity)
                {
                    return (eye_data_validata_bit_mask & (ulong)(1 << (int)validity)) > 0;
                }
            }

Here is the source code of the SingleEyeData, but I don't know how to run GetValidity(SingleEyeDataVadity validity). What is the input variable validity?  I can easily access to the property of SingleEyeData through: eye.data.gaze_origin_mm, but when it comes to GetValidity, I don't know which is the input. Could you please do me a favor?

@imarin18 @Daniel_Y@kbombeke

Thank you in advance. 

Edited by alwyuyang
Link to post
Share on other sites
21 hours ago, imarin18 said:

Dear @alwyuyang

I suppose you can get the validity data from "eye_data_validata_bit_mask" in your code. The type of the output is UInt64, not bool.

image.png.70f4bcf16c4d5a63058b06264a1efa12.png

 

Best regards,

imarin18 

 

Dear @imarin18,

I am checking the timestamp and frame_sequence, but I found the following the results in the log file. 

Column name: timestamp, frame_sequence, eye_data_validata_bit_mask

row 1: 62613 4848 31

row 2: 4205382 4964 31

row 3: 4205415 4968 31 

row 3: 4205457 4973 31

@Daniel_Y

I don't understand why there is a huge skip from row 1  to row 2 regarding the timestamp. The row 1 is first output I obtained from the system. 

In addition, what does "31" means for the eye_data_validata_bit_mask? I only know it is uint64 data type, but how to interpret it?

 

Thank you.

 

 

 

 

 

 

 

Link to post
Share on other sites

Dear @alwyuyang

As explained in the following post, the timestamp cannot be correctly measured with the current SDK version according to the technical support team.

 

If I understand correctly, I reckon that the value of validity follows the table below. If you sum all the decimal digits (1+2+4+8+16), you get 31. Though again I am not sure if my understanding is correct.

Enumerator

Binary digit if valid

Decimal digit if valid

SINGLE_EYE_DATA_GAZE_ORIGIN_VALIDITY

00001

1

SINGLE_EYE_DATA_GAZE_DIRECTION_VALIDITY

00010

2

SINGLE_EYE_DATA_PUPIL_DIAMETER_VALIDITY

00100

4

SINGLE_EYE_DATA_EYE_OPENNESS_VALIDITY

01000

8

SINGLE_EYE_DATA_PUPIL_POSITION_IN_SENSOR_AREA_VALIDITY

10000

16

 

Best regards,

imarin18

Link to post
Share on other sites
3 minutes ago, imarin18 said:

Dear @alwyuyang

As explained in the following post, the timestamp cannot be correctly measured with the current SDK version according to the technical support team.

 

If I understand correctly, I reckon that the value of validity follows the table below. If you sum all the decimal digits (1+2+4+8+16), you get 31. Though again I am not sure if my understanding is correct.

Enumerator

Binary digit if valid

Decimal digit if valid

SINGLE_EYE_DATA_GAZE_ORIGIN_VALIDITY

00001

1

SINGLE_EYE_DATA_GAZE_DIRECTION_VALIDITY

00010

2

SINGLE_EYE_DATA_PUPIL_DIAMETER_VALIDITY

00100

4

SINGLE_EYE_DATA_EYE_OPENNESS_VALIDITY

01000

8

SINGLE_EYE_DATA_PUPIL_POSITION_IN_SENSOR_AREA_VALIDITY

10000

16

 

Best regards,

imarin18

Dear @imarin18,

Thank you for sharing your opinions. However, 

1, although the timestamp is not measured precisely, how could it jump from 62613 to 4205382  in two subsequent collection?

2, i agree with your explanation of "31". However,  I also got uint64  "8"  for the eye_data_validata_bit_mask when it failed to measure, then it seems hard to explain by the summing theory. 

 

                writeLeftEyeData.WriteLine("{0} {1} {2} {3} {4} {5} {6} {7} {8} {9} {10} {11} {12} {13} {14} {15} {16} {17} {18}",
                                          Sample_MainThread.UnityTime,
                                          unixTime,
                                          data.no_user,
                                          data.timestamp,
                                          data.frame_sequence,

                                          data.verbose_data.left.eye_data_validata_bit_mask,
                                          
                                          data.verbose_data.left.gaze_origin_mm.x,
                                          data.verbose_data.left.gaze_origin_mm.y,
                                          data.verbose_data.left.gaze_origin_mm.z,

                                          data.verbose_data.left.gaze_direction_normalized.x,
                                          data.verbose_data.left.gaze_direction_normalized.y,
                                          data.verbose_data.left.gaze_direction_normalized.z,

                                          data.verbose_data.left.pupil_diameter_mm,
                                          data.verbose_data.left.eye_openness,

                                          data.verbose_data.left.pupil_position_in_sensor_area.x,
                                          data.verbose_data.left.pupil_position_in_sensor_area.y,

                                          data.expression_data.left.eye_frown,
                                          data.expression_data.left.eye_squeeze,
                                          data.expression_data.left.eye_wide                                          
                                          );

 

I attach my code and the corresponding output file so you can see that I got 31 when it worked and 8 when it failed. Anything to comment? Thank you:)

@Daniel_Obrien @Daniel_Y

Best regards,

 

 

 

 

leftEyeDataSub.txt

Link to post
Share on other sites

Dear @alwyuyang

I suppose that the technical support team meant that the huge gap from 62613 to 4205382 in timestamp data would be also a part of improvement.

I'm not sure exactly about what you meant by "when it failed to measure", but when you get 8 in your validity data, I reckon that you have the valid data only for eye openness (SINGLE_EYE_DATA_EYE_OPENNESS_VALIDITY) having the value of 8 and the other four parameters have the value of 0, meaning that the four are not valid.

Best regards,

imarin18

 

Link to post
Share on other sites

Please sign in to comment

You need to be a member in order to leave a comment

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...