Corvus

@AdamStreck Thanks for sharing the update!

@apellisscot @prajaktakhandve

VIVE Eye Tracking at 120hz Note: First setup the VIVE Eye Tracking SDK and verify basic functionality (calibration & demo scene) Note: Callback runs on a separate thread to report data at ~120hz Note: Unity is not thread-safe and cannot call any UnityEngine api from within callback thread. Unity Guide Setup Project for Eye Tracking Import VIVE Eye Tracking SDK Unity Plugin (SRanipal) Add the SRanipal Eye Framework to the Unity scene Drag "SRanipal Eye Framework" prefab into scene hierarchy or Attach "SRanipal_Eye_Framework.cs" script to gameobject in scene Setup Framework Settings Check "Enable Eye" (enabled by default) Check "Enable Eye Data Callback" Setup EyeData Callback Register EyeData Callback Note: Only register callback when the eye tracking framework status is working and do not register more than one callback at once. SRanipal_Eye.WrapperRegisterEyeDataCallback(Marshal.GetFunctionPointerForDelegate((SRanipal_Eye.CallbackBasic)EyeCallback)); Use EyeData Callback private static EyeData eyeData = new EyeData(); /// Required class for IL2CPP scripting backend support internal class MonoPInvokeCallbackAttribute : System.Attribute { public MonoPInvokeCallbackAttribute() { } } [MonoPInvokeCallback] private static void EyeCallback(ref EyeData eye_data) { eyeData = eye_data; // do stuff with eyeData... } Unregister EyeData Callback Note: Must unregister callback when complete or application is disabled/quit. SRanipal_Eye.WrapperUnRegisterEyeDataCallback(Marshal.GetFunctionPointerForDelegate((SRanipal_Eye.CallbackBasic)EyeCallback)); Unity Sample Code using UnityEngine; using ViveSR.anipal.Eye; using System.Runtime.InteropServices; /// <summary> /// Example usage for eye tracking callback /// Note: Callback runs on a separate thread to report at ~120hz. /// Unity is not threadsafe and cannot call any UnityEngine api from within callback thread. /// </summary> public class CallbackExample : MonoBehaviour { private static EyeData eyeData = new EyeData(); private static bool eye_callback_registered = false; private void Update() { if (SRanipal_Eye_Framework.Status != SRanipal_Eye_Framework.FrameworkStatus.WORKING) return; if (SRanipal_Eye_Framework.Instance.EnableEyeDataCallback == true && eye_callback_registered == false) { SRanipal_Eye.WrapperRegisterEyeDataCallback(Marshal.GetFunctionPointerForDelegate((SRanipal_Eye.CallbackBasic)EyeCallback)); eye_callback_registered = true; } else if (SRanipal_Eye_Framework.Instance.EnableEyeDataCallback == false && eye_callback_registered == true) { SRanipal_Eye.WrapperUnRegisterEyeDataCallback(Marshal.GetFunctionPointerForDelegate((SRanipal_Eye.CallbackBasic)EyeCallback)); eye_callback_registered = false; } } private void OnDisable() { Release(); } void OnApplicationQuit() { Release(); } /// <summary> /// Release callback thread when disabled or quit /// </summary> private static void Release() { if (eye_callback_registered == true) { SRanipal_Eye.WrapperUnRegisterEyeDataCallback(Marshal.GetFunctionPointerForDelegate((SRanipal_Eye.CallbackBasic)EyeCallback)); eye_callback_registered = false; } } /// <summary> /// Required class for IL2CPP scripting backend support /// </summary> internal class MonoPInvokeCallbackAttribute : System.Attribute { public MonoPInvokeCallbackAttribute() { } } /// <summary> /// Eye tracking data callback thread. /// Reports data at ~120hz /// MonoPInvokeCallback attribute required for IL2CPP scripting backend /// </summary> /// <param name="eye_data">Reference to latest eye_data</param> [MonoPInvokeCallback] private static void EyeCallback(ref EyeData eye_data) { eyeData = eye_data; // do stuff with eyeData... } }

@apellisscot Can you please share a code snippet?

@jaalto1 are you removing the HMD during calibration? Generally the notification icon is just for monitoring the status of the runtime and generating the logs. If the calibration is progressing or the sample app is functioning then it is not necessary to remove the HMD and check the icon.

@jaalto1 Thanks for sharing feedback about the Getting Started Guide. The guide you linked is actually maintained by Tobii (our partner) and is aimed at developers who will use the SDK (software development kit). The robot icon eyes will only turn green when the eye tracking is active (such as during calibration or while running a game/app that uses eye tracking). Also, make sure you are not moving your head during the calibration step, and only following the dot with your eyes. Vive Pro Eye Setup: https://www.vive.com/us/support/vive-pro-eye/category_howto/setting-up-for-the-first-time.html

@jc10487 I'll look into this further and follow up.

@LochoChoco Yes, that should be the latest. Did you run the uninstaller and/or use the uninstall reg tool? And make sure to restart the PC after it is uninstalled.

@jc10487 Sorry for any confusion. Yes, Wave SDK is for android.

@jc10487 It is possible to disable the render mask with the Wave XR Plugin. The Unity OpenXR Plugin is still in preview and maintained by Unity so any bugs or feature requests should be made directly. https://hub.vive.com/storage/docs/en-us/UnityXR/UnityXRSettings.html#rendermask

@LochoChoco Can you confirm that you are using the latest runtime installer?

@adrianmeyerart Viveport does not support .OBB expansion files yet but there is no .APK file size limit. The DRM wrapper can only support up to 50GB files but games can use the SDK DRM option if that is an issue.

@anagashi_zz Can you share a crash log file to see what causes the issue?

@Federica Here is the documentation about SRWorks Pass Through to get you started: https://hub.vive.com/storage/srworks/srworks_unity/plugin_guide/unity_passthrough.html

@Cakebatter Can you take a screenshot? (power button and volume down button on the HMD) And can you share the PC specs and network information? Like is your PC ethernet wired? This sounds like a possible bug (technically Viveport streaming is still in Beta).

@LochoChoco Were you able to fix the issue?

@jboss It is possible to have the runtime start with windows startup and only require the UAC prompt once. Copy the app shortcut (the one created on the desktop during install) to the windows startup folder (%APPDATA%\Microsoft\Windows\Start Menu\Programs\Startup)

@Asish Here is the EyeData image included with the SDK documentation which has information about the available output data and gaze angles.

@Annabell @killerbee @AstroVR If someone is not focusing on an object (i.e. "staring off into the air") then each eye's gaze vector will generally not intersect. Even with perfect tracking quality, the eyes may be looking parallel at a far enough away point and parallel lines do not intersect. https://en.wikipedia.org/wiki/Vergence

@Asish Were you able to measure the gaze angles? It should be possible with the gaze direction and Focus api.

@qxxxb @jboss Thanks for reporting this bug. The team will look into it.

EyeData Image

@monetl I'll try and respond to each of these questions but let me know if you need further clarification on anything. 1. I don't fully understand this question. Would you like clarification on the "Focus" api functionality? Or are you looking for suggestions of an alternative solution? Please explain a little more or include a diagram. 2. IPD of virtual cameras is generally handled automatically by the game engine and plugins used. Are you using the legacy SteamVR plugin, the Unity XR Steam plugin, or the built in Unity SteamVR support on older Unity versions? It is not normal to modify the distance of the virtual camera IPD but was previously possible depending on the plugin/engine/rendering method. 3. The Unity Update loop is called for each rendered frame which is usually ~90Hz when running with Vive Pro Eye (unless performance issues are causing frames to drop). The callback method allows for multi-threaded data access at the fastest rate possible for the eye tracking device which is on average ~120Hz. 4. convergence_distance_mm is not implemented and will not report valid results. I believe this feature is available in the Tobii XR SDK which does work with the VIVE Eye Tracking SDK (SRanipal). 5. There was a bug fixed related to timestamps in the latest release. Please test and confirm if it is fixed for you. 6. Are you able to reproduce this issue in the sample scene with the mirror & gaze ray? Can you take a screenshot or video? 7. There is no access to the raw camera eye images. Some developers have explored this option with Tobii directly. 8... Filtering is added to prevent gaze tremble (ex. when looking at distant objects). Use SetEyeParameter to set the filtering level with "sensitive_factor". You can adjust the variable to effect the cutoff slope in the filter. 1 is raw gaze vector without adding a filter post-processing for gaze tremble. 0 is post processed with strongest filter to remove gaze tremble.` The default value is 0.007 and you can multiply or divide the variable by 10 until you notice the effect on the gaze data. Note: SetEyeParameter filtering only works on gaze rays. void SetFiltering(double filteringLevel) { EyeParameter parameter = new EyeParameter { gaze_ray_parameter = new GazeRayParameter(), }; parameter.gaze_ray_parameter.sensitive_factor = filteringLevel; Error error = SRanipal_Eye_API.SetEyeParameter(parameter); } 9. This looks related to a bug reported previously. https://forum.vive.com/topic/9172-gaze-origin-bug-in-getgazeray-of-unity-package/?ct=1610732330 10. You can use the "Combined Gaze Origin" to get the center point between the eyes. See the EyeData.png diagram in the documentation included with the SDK download.