zzy

Hi @User_25506 I understand your concern. One direction I can give is that you can try with Vive Elite, which also supports lighthouse tracking and avoids SteamVR camera API.

Hi @Liyea Have you tried to run the samples from Vive Hand Tracking & Vive Input Utility SDK? Please let me know if editor crash happens when you only use tracker or only use hand in the editor.

Hi @User_25506 Hand Tracking SDK need to use OpenVR API to get camera info and frames, so I think it's difficult to bypass SteamVR. I think the only way to refresh the camera is to restart SteamVR, but that doesn't solve the problem if camera freezes every minute or so. I'm now running out of advice. Maybe you can consider using Vive Cosmos series instead, it doesn't use SteamVR to acess camera.

Hi @User_25506 From my previous experience, when SteamVR experience camera failure (e.g. package lost) it will mark camera failed and never retry. This makes the camera freeze. So I would recommend to try to make sure camera communication is working by reducing usage of USB bus by other devices. Here are my advices: Always make sure Vie Pro is connected to USB 3.0 port. Try to remove other usb devices if possible. Try to plugin other devices (e.g. keyboard and mouse) to USB 2.0 ports if any. If you are using other usb devices that is also transmitting a lot of data, you may consider adding USB extension hardware to make sure all devices can runs at full speed.

Hi @long_then Which HMD are you using? The depth of hand may not be accurate if you are using Vive, since it only has 1 camera. On HMD with 2 cameras (e.g. Vive Pro & Cosmos series), the depth will be more accurate.

Hi @Vega The joint rotation is in the same coordinate as the hand joint position. The coordinate system is slightly different, based on if you have called UseExternalTransform before StartGestureDetection or not. If you called UseExternalTransform(true), we would assume transform passed in by SetCameraTransform as HMD transform. If you haven't called the function, we will get HMD transform from OpenVR, this is same as calling GetDeviceToAbsoluteTrackingPose with vr::TrackingUniverseStanding. All hand positions & rotations are then multiplied by HMD transform, this makes the results in the global coordinate system. All rotation of 21 joints are in the global coordinate system, so if you want to get relative joint rotation, you need to calculate it your self.

Hi @User_25506 Thanks for the info. I think the camera from HMD will freeze if you try to access the camera device outside of SteamVR. I can also confirm this, if you try to use Camera app from Windows to access the HMD camera, it will freeze. I think this maybe caused by the driver or there are some softwared problem inside SteamVR. I would suggest to not access the camera using other other APIs. Or if other process must access camera on start, you can try to plugin a usb camera first, and plugin your HMD after it. Then the default camera device will the usb camera and the process will not access the HMD camera.

Hi @kotauchisunsun OK, from the log it seems the root cause is that vive hand tracking sdk doesn't detect arcore, thus it choose the normal android camera API to get camera stream. Since camera is already in use, the detection failed to start. So the problem is inside code to detect arcore, which is defined in ViveHandTrackingEngine.cs (L195). 195> #if VIVEHANDTRACKING_UNITYXR_ARCORE && UNITY_ANDROID && !UNITY_EDITOR 196> IEnumerator SetupARCore() { 197> var settings = UnityEngine.XR.Management.XRGeneralSettings.Instance; 198> if (settings == null) yield break; 199> var manager = settings.Manager; 200> if (manager == null || manager.activeLoader == null) yield break; 201> var loader = manager.ActiveLoaderAs<UnityEngine.XR.ARCore.ARCoreLoader>(); 202> if (loader == null) yield break; So we need to check 1) if this function is called 2) if this function returned early due to missing arcore loader. So here is the check list: The function is called at the beginning of StartDetection function (L100). Please check if VIVEHANDTRACKING_UNITYXR_ARCORE is defined. This is to make sure "com.unity.xr.arcore" package is installed and listed in package.json. Before you add GestureProvider script in ARSessionWatcher.cs, you can check if settings, manager and loader are all not null. You can copy the code from L197-L202 and add logs to see if these variables are null. If any of these are null, you may try to wait for a few frames and retry. Please only add Gestureprovider script after all the check passed to make sure hand tracking SDK knows arcore is running.

Hi @kotauchisunsun Thanks for providing the info. I have checked the arfoundation_with_vive_handtracking_20211030.txt. Here is related parts from the file: 10-30 16:42:53.092: I/Unity(6528): enabled 10-30 16:42:53.092: I/Unity(6528): #0 0xbe6e5819 (libunity.so) ? 0x0 10-30 16:42:53.092: I/Unity(6528): #1 0xbedaaa9f (libunity.so) ? 0x0 10-30 16:42:53.092: I/Unity(6528): #2 0xbe8d55e5 (libunity.so) ? 0x0 10-30 16:42:53.092: I/Unity(6528): #3 0xbe8d5551 (libunity.so) ? 0x0 10-30 16:42:53.092: I/Unity(6528): #4 0xcd8c8da6 (Unknown) ? 0x0 10-30 16:42:53.134: I/Aristo(6528): Starting Vive Hand Tracking SDK 1.0.0 as 32-bit binary 10-30 16:42:53.148: I/Aristo(6528): Selected camera 0 I can see that after you enabled GestureProvider script, GestureProvider directly starts the detection with android camera. This means that is does not detect ARCore. Since camera is already used by ARCore, Vive Hand Tracking will fail to start. However, there is something strange here. From GestureProvider.Start function (line 90-111), there should be logs that output engine selection process (no matter success or fail). But I cannot find any of these logs. So I think GestureProvider.Start may be already called before you enabled it from the script. I would recommend you to check it in Unity Editor: 1) make sure GestureProvider is disabled in the scene 2) make sure GestureProvider script remains disabled after entered play mode in Unity Editor 3) you can manually enable GestureProvider from inspector and see if there are logs from GestureProvider.Start function in the console. Another solution is that, you can completely remove GestureProvider in the scene and AddComponent in ARSessionWatcher.run function after ARCore is enabled.

Hi @DDD_KK Yes, for initial state of your project, you need to copy the script and remove the #if clause. If you don't want to change the script, another solution is to install Unity XR packages (while keeping SteamVR plugin). SteamVR will setup UnityXR for you.

Hi @DDD_KK You are using OpenXR, not OpenVR. If you are using SteamVR Unity Plugin v2.7.3 on Unity 2020, it should install OpenVR support for Unity XR automatically. See it's release note here: https://github.com/ValveSoftware/steamvr_unity_plugin/releases/tag/2.7.3

Hi @DDD_KK I'm sure if UnityXR supports tracker. You can try to copy the WindowsPlayModeMonitor.cs editor script to your project and remove the #if clause in the file. That should fix your problem.

Hi @Lusther Good to know the install succeeded. As for the prefab location, the document was written for the old unitypackage plugin. The package installed from scoped registry are indeed having different paths. You can just replace “Assets/ViveHandTracking" by "Packages/com.htc.upm.vivehandtracking/Runtime".

Hi @Lusther The scoped registry was down temporarily, it's up and running now. Please have a try. It's recommended to install from scoped registry for Unity 2019 or newer.

Hi @RRob Sorry for the late reply, I just got some time to test this. Indeed the ARCore failed to start on my side. It seems that ARCore is taking more than 1 frame to start. Hand Tracking starts before ARCore and thus occupying the camera. A quick way to fix this is that, you can disable GestureProvider script in your scene. You can add a script to wait until ARCore is started, and then enable GestureProvider script.