How to detect object that is being focused ?

[Ihshan Gumilar]

Hi,

I am trying to use SRanipal SDK. I would like to detect which object that is being looked at. By just returning the name of objects that are being looked at should be fine with me. Could anyone give me a hand please ? I am trying to look at the sample script but could not really get it.

Here is the script that I call from Update function in UNITY

using Photon.Pun;
using System;
using System.IO;
using UnityEngine;
using ViveSR.anipal.Eye;


void Update()
{
        CheckFocus();
}

public void CheckFocus()
{
        Ray GazeRay;

        bool eyeFocus;
        if (SRanipal_Eye.Focus(GazeIndex.COMBINE, out GazeRay, out focusInfo)) { }
        else if (SRanipal_Eye.Focus(GazeIndex.RIGHT, out GazeRay, out focusInfo)) { }
        else if (SRanipal_Eye.Focus(GazeIndex.LEFT, out GazeRay, out focusInfo)) { }
        else return;

        var currentFocusObject = focusInfo.collider.gameObject.name;

}

 

Kindly see the error that I have got.

NullReferenceException: Object reference not set to an instance of an object
ViveSR.anipal.Eye.SRanipal_Eye.Focus (ViveSR.anipal.Eye.GazeIndex index, UnityEngine.Ray& ray, ViveSR.anipal.Eye.FocusInfo& focusInfo, System.Single radius, System.Single maxDistance, System.Int32 focusableLayer, ViveSR.anipal.Eye.EyeData eye_data) (at Assets/ViveSR/Scripts/Eye/SRanipal_Eye.cs:357)
ViveSR.anipal.Eye.SRanipal_Eye.Focus (ViveSR.anipal.Eye.GazeIndex index, UnityEngine.Ray& ray, ViveSR.anipal.Eye.FocusInfo& focusInfo, ViveSR.anipal.Eye.EyeData eye_data) (at Assets/ViveSR/Scripts/Eye/SRanipal_Eye.cs:462)
ViveSR.anipal.Eye.SRanipal_Eye.Focus (ViveSR.anipal.Eye.GazeIndex index, UnityEngine.Ray& ray, ViveSR.anipal.Eye.FocusInfo& focusInfo) (at Assets/ViveSR/Scripts/Eye/SRanipal_Eye.cs:475)
EyeInfo.CheckFocus () (at Assets/Scripts/EyeInfo.cs:180)
EyeInfo.Update () (at Assets/Scripts/EyeInfo.cs:63)

 

Thanks

 

[C3D]

Hi,

I just took a quick look at the script and the Focus method in the SRanipal_Eye script. I think I'm on a different version of the SDK (line 357 is a comment for me), but I suspect the issue is caused by Camera.main.

In Unity, does your camera have the MainCamera tag?

[Ihshan Gumilar]

Hi,

Nope. It does not have any MainCamera tag.

I tried to put it. But it does not work

Thanks

[Corvus]

@Ihshan Gumilar  Here is a sample for using the Focus api in Unity to get the current gameobject and it's name with FocusName(). Also, make sure you have the SRanipal_Eye_Framework script attached to a gameobject in the scene with "Enable Eye" checked. This method will only report gameobjects with colliders attached. And as @C3D mentioned you will need the MainCamera tag to prevent the NullReferenceException errors.

 

    private static Ray testRay;
    private static FocusInfo focusInfo;

    /// <summary>
    /// Checks for current object in focus.
    /// </summary>
    public static GameObject Focus() 
    {
        if (EyeTool.GetVersion() == 1)
        {
            if (SRanipal_Eye.Focus(GazeIndex.COMBINE, out testRay, out focusInfo)) { }
            else if (SRanipal_Eye.Focus(GazeIndex.LEFT, out testRay, out focusInfo)) { }
            else if (SRanipal_Eye.Focus(GazeIndex.RIGHT, out testRay, out focusInfo)) { }
            else return null;
        }
        else if (EyeTool.GetVersion() == 2)
        {
            if (SRanipal_Eye_v2.Focus(GazeIndex.COMBINE, out testRay, out focusInfo)) { }
            else if (SRanipal_Eye_v2.Focus(GazeIndex.LEFT, out testRay, out focusInfo)) { }
            else if (SRanipal_Eye_v2.Focus(GazeIndex.RIGHT, out testRay, out focusInfo)) { }
            else return null;
        }

        return focusInfo.collider.gameObject;
    }

    /// <summary>
    /// Checks name for current object in focus.
    /// </summary>
    public static string FocusName() 
    {
        if (Focus() is null)
            return "";
        else
            return Focus().name;
    }

 

[Ihshan Gumilar]

Hi @Corvus,

 

Thanks for the code. Yet, I could not find for EyeTool. Where does it refer to ?

Cheers,

 

[Corvus]

@Ihshan Gumilar Sorry, EyeTool is a helper toolkit/library/demo I'm working on for using the eye tracking SDK. For now you can just use the block of code that corresponds to the SRanipal Eye version you are using (SRanipal_Eye vs SRanipal_Eye_v2).

[camusic]

Hi @Corvus

I have been using this example to detect focused objects fine at 60 fps, however I run into issues when trying to use the same logic in your EyeCallback example at 120 fps. Is it possible to detect focused objects at 120 fps? When I put the focus detection code above into the EyeCallback, the callback fails to even execute and I am wondering what the best solution to this is.

[ABUSHATTAL]

Hi @Corvus
I have tried the script and it return object name. but when I recall it in the update function to keep tracking where the player is looking it is freeze unity after hit play button. I also note when disable a script that recall the eye data from SRanipal it works fine. can you help me how to make it work to capture focus using update function?.  I have used the following script to get eye data using SRanipal eye frame and i want to incluse the detected object to the update section:

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using System;
using System.Threading;
using System.IO;
using ViveSR.anipal.Eye;
namespace Test120FPS
{
    public class Sample_GetDataThread : MonoBehaviour
    {
        public GameObject Status;
        private string state;
        public GameObject Car;
        private string Carx;
        private string Cary;
        private string Bicyclex;
        private string Bicycley;
        private string distance;
        public GameObject Bicycle;
        public EyeData data = new EyeData();
        private Thread thread; private const int FrequencyControl = 1;
        private const int MaxFrameCount = 3600;
        void Start()
        {
            thread = new Thread(QueryEyeData);
            thread.Start();
            
            string text =  "distance"+ "?" + "Carx" + "?" + "Cary" + "?" +"Bicyclex" + "?" +"Bicycley"+"?"+"Left eye dia: "+"?" + "Right eye dia: " + "?" + "Left eye openess " + "?" + "Right eye openess " + "?" + "Left eye gaze " + "?" + "Right eye gaze " + "?" + "Status" + "?" + "CurrentFrameSequence" + "?" +  "CurrentSystemTime(ms)"  + Environment.NewLine;
            File.AppendAllText("DataRecord.txt", text);
        }
        private void OnApplicationQuit()
        {
            thread.Abort();                
        }
        private void OnDisable()
        {
            thread.Abort();
        }

      

        /// <summary>
        /// Checks name for current object in focus.
        /// </summary>
        

        void FixedUpdate()
        {
            Carx = Car.transform.position.x.ToString();
            Cary = Car.transform.position.z.ToString();
            Bicyclex = Bicycle.transform.position.x.ToString();
            Bicycley = Bicycle.transform.position.z.ToString();
            state = Status.name.ToString();
            float dist = Vector3.Distance(Car.transform.position, Bicycle.transform.position);
            float feet = dist * 3.28f;
            distance = ((int)dist*3.28f).ToString();
            
            

        }

        // You can only use C# native function in Unity's thread.
        // Use EyeData's frame_sequence to calculate frame numbers and record data in file.
        void QueryEyeData()
        {
            int FrameCount = 0;
            int PrevFrameSequence = 0, CurrFrameSequence = 0;
            bool StartRecord = false;
            while (FrameCount < MaxFrameCount)
            {
                ViveSR.Error error = SRanipal_Eye_API.GetEyeData(ref data);
                if (error == ViveSR.Error.WORK)
                {
                    CurrFrameSequence = data.frame_sequence;
                    if (CurrFrameSequence != PrevFrameSequence)
                    { 
                        FrameCount ++;
                        PrevFrameSequence = CurrFrameSequence;
                        StartRecord = true;
                    } 
                }
                // Record time stamp every 120 frame.
                if (FrameCount % 60 == 0 && StartRecord)
                {

                    string text = distance + "?" + Carx+ "?"+ Cary+ "?" +Bicyclex + "?" + Bicycley +"?" + data.verbose_data.left.pupil_diameter_mm + "?" + data.verbose_data.right.pupil_diameter_mm + "?" + data.verbose_data.left.eye_openness + "?" + data.verbose_data.right.eye_openness + "?" + data.verbose_data.left.gaze_direction_normalized +"?"+ data.verbose_data.right.gaze_direction_normalized + "?" + state + "?" + CurrFrameSequence + "?" + DateTime.Now.ToString() + Environment.NewLine;
                    File.AppendAllText("DataRecord.txt", text);
                    FrameCount = 0;
                } 
                Thread.Sleep(FrequencyControl);
            }
        }
    }}