Jump to content
VIVE Forum
Need to reach the Vive support team? - Click here

zzy

Verified Members
 View Profile  See their activity

  • Posts

    219

  • Joined

  • Last visited

 Content Type 

Posts posted by zzy

    Vive Pro cameras freezing in SteamVR when using Vive hand tracking

    in VIVE Hand Tracking SDK

    Posted

    Hi @User_25506

    Hand Tracking SDK need to use OpenVR API to get camera info and frames, so I think it's difficult to bypass SteamVR. I think the only way to refresh the camera is to restart SteamVR, but that doesn't solve the problem if camera freezes every minute or so.

    I'm now running out of advice. Maybe you can consider using Vive Cosmos series instead, it doesn't use SteamVR to acess camera.

    Vive Pro cameras freezing in SteamVR when using Vive hand tracking

    in VIVE Hand Tracking SDK

    Posted

    Hi @User_25506

    From my previous experience, when SteamVR experience camera failure (e.g. package lost) it will mark camera failed and never retry. This makes the camera freeze.

    So I would recommend to try to make sure camera communication is working by reducing usage of USB bus by other devices. Here are my advices:

    1. Always make sure Vie Pro is connected to USB 3.0 port.
    2. Try to remove other usb devices if possible.
    3. Try to plugin other devices (e.g. keyboard and mouse) to USB 2.0 ports if any.
    4. If you are using other usb devices that is also transmitting a lot of data, you may consider adding USB extension hardware to make sure all devices can runs at full speed.

    [Vive Pro SDK 1.0.0] Rotation frame information from hand tracking SDK

    in VIVE Hand Tracking SDK

    Posted

    Hi @Vega

    The joint rotation is in the same coordinate as the hand joint position. The coordinate system is slightly different, based on if you have called UseExternalTransform before StartGestureDetection or not.

    If you called UseExternalTransform(true), we would assume transform passed in by SetCameraTransform as HMD transform.

    If you haven't called the function, we will get HMD transform from OpenVR, this is same as calling GetDeviceToAbsoluteTrackingPose with vr::TrackingUniverseStanding.

    All hand positions & rotations are then multiplied by HMD transform, this makes the results in the global coordinate system. All rotation of 21 joints are in the global coordinate system, so if you want to get relative joint rotation, you need to calculate it your self.

    • Thanks 1

    Vive Pro cameras freezing in SteamVR when using Vive hand tracking

    in VIVE Hand Tracking SDK

    Posted

    Hi @User_25506

    Thanks for the info. I think the camera from HMD will freeze if you try to access the camera device outside of SteamVR. I can also confirm this, if you try to use Camera app from Windows to access the HMD camera, it will freeze. I think this maybe caused by the driver or there are some softwared problem inside SteamVR.

    I would suggest to not access the camera using other other APIs. Or if other process must access camera on start, you can try to plugin a usb camera first, and plugin your HMD after it. Then the default camera device will the usb camera and the process will not access the HMD camera.

    ARFoundation sample issue

    in VIVE Hand Tracking SDK

    Posted

    Hi @kotauchisunsun

    OK, from the log it seems the root cause is that vive hand tracking sdk doesn't detect arcore, thus it choose the normal android camera API to get camera stream. Since camera is already in use, the detection failed to start.

    So the problem is inside code to detect arcore, which is defined in ViveHandTrackingEngine.cs (L195). 

    195> #if VIVEHANDTRACKING_UNITYXR_ARCORE && UNITY_ANDROID && !UNITY_EDITOR
196>  IEnumerator SetupARCore() {
197>    var settings = UnityEngine.XR.Management.XRGeneralSettings.Instance;
198>    if (settings == null) yield break;
199>    var manager = settings.Manager;
200>    if (manager == null || manager.activeLoader == null) yield break;
201>    var loader = manager.ActiveLoaderAs<UnityEngine.XR.ARCore.ARCoreLoader>();
202>    if (loader == null) yield break;

    So we need to check 1) if this function is called 2) if this function returned early due to missing arcore loader.

    So here is the check list:

    1. The function is called at the beginning of StartDetection function (L100). Please check if VIVEHANDTRACKING_UNITYXR_ARCORE is defined. This is to make sure "com.unity.xr.arcore" package is installed and listed in package.json.
    2. Before you add GestureProvider script in ARSessionWatcher.cs, you can check if settings, manager and loader are all not null. You can copy the code from L197-L202 and add logs to see if these variables are null. If any of these are null, you may try to wait for a few frames and retry.

    Please only add Gestureprovider script after all the check passed to make sure hand tracking SDK knows arcore is running.

    ARFoundation sample issue

    in VIVE Hand Tracking SDK

    Posted

    Hi @kotauchisunsun

    Thanks for providing the info. I have checked the arfoundation_with_vive_handtracking_20211030.txt. Here is related parts from the file: 

    10-30 16:42:53.092: I/Unity(6528): enabled
10-30 16:42:53.092: I/Unity(6528):  #0 0xbe6e5819 (libunity.so) ? 0x0
10-30 16:42:53.092: I/Unity(6528):  #1 0xbedaaa9f (libunity.so) ? 0x0
10-30 16:42:53.092: I/Unity(6528):  #2 0xbe8d55e5 (libunity.so) ? 0x0
10-30 16:42:53.092: I/Unity(6528):  #3 0xbe8d5551 (libunity.so) ? 0x0
10-30 16:42:53.092: I/Unity(6528):  #4 0xcd8c8da6 (Unknown) ? 0x0
10-30 16:42:53.134: I/Aristo(6528): Starting Vive Hand Tracking SDK 1.0.0 as 32-bit binary
10-30 16:42:53.148: I/Aristo(6528): Selected camera 0

    I can see that after you enabled GestureProvider script, GestureProvider directly starts the detection with android camera. This means that is does not detect ARCore. Since camera is already used by ARCore, Vive Hand Tracking will fail to start.

    However, there is something strange here. From GestureProvider.Start function (line 90-111), there should be logs that output engine selection process (no matter success or fail). But I cannot find any of these logs. So I think GestureProvider.Start may be already called before you enabled it from the script.

    I would recommend you to check it in Unity Editor: 1) make sure GestureProvider is disabled in the scene 2) make sure GestureProvider script remains disabled after entered play mode in Unity Editor 3) you can manually enable GestureProvider from inspector and see if there are logs from GestureProvider.Start function in the console.

    Another solution is that, you can completely remove GestureProvider in the scene and AddComponent in ARSessionWatcher.run function after ARCore is enabled.

    ARFoundation sample issue

    in VIVE Hand Tracking SDK

    Posted

    Hi @RRob

    Sorry for the late reply, I just got some time to test this.

    Indeed the ARCore failed to start on my side. It seems that ARCore is taking more than 1 frame to start. Hand Tracking starts before ARCore and thus occupying the camera.

    A quick way to fix this is that, you can disable GestureProvider script in your scene. You can add a script to wait until ARCore is started, and then enable GestureProvider script.

    [Vive Hand Tracking SDK v 1.0.0] [SteamVR Plugin v 2.7.3] Unity Editor crashes when exiting from Play mode

    in VIVE Hand Tracking SDK

    Posted

    Hi @DDD_KK

    Thanks for the report.  Have you tried to use UnityXR instead of SteamVR plugin? We recommend to use Unity 2020 with UnityXR.

    We have an editor script WindowsPlayModeMonitor.cs, which is used to solve this problem when using UnityXR. However the script is not enabled when you don't have the UnityXR package.

    You can try to copy the script in your project and remove the #if clause at beginning of the file.

    ARFoundation sample issue

    in VIVE Hand Tracking SDK

    Posted

    Hi @RRob

    Can you please share the full adb log? I would suspect that hand tracking plugin doesn't detected arcore and thus getting camera stream using ndk camera, not from arcore apis.

    Although you may have done this, please make sure you follow the instructions in the ARFoundation sample to replace "AR Session" and "AR Session Origin" game object with latest version.

    Can you please also try Unity LTS versions? I have previously tested on 2019.4 and 2020.3 which should work fine. I'll check if I can reproduce this on my side.

    Hand Tracking Vive Pro 2 Unreal

    in VIVE Hand Tracking SDK

    Posted

    Hi @Zeploc

    Some requirements for your hand:

    1. Roll up your sleeves to make sure wrist is fully visible in the image
    2. Make sure that you don't have something very bright in your background, which can cause the hand to be quite dark and lead to bad result.
    3. If you find the hand quite jittery, try to move your hand around or rotate the body and face another direction to see if this helps. Some random environment objects can cause bad results too. You can also try a pure colored wall if possible.

    Another way to check the image quality is to use camera tab in SteamVR settings and use preview to make sure your hand is clear (including textures of joints) and fully visible (including your wrist).

    If you still encounter these problems, can you please share some screenshots about your environment (SteamVR settings) and the virsual hands (app)? You can send through private chat if you don't want to make it public.

    Vive Hand Tracking SDK Release 1.0.0

    in VIVE Hand Tracking SDK

    Posted

    Hi everyone,

    Vive Hand Tracking SDK has a new release v1.0.0 today. You can download it from here. For Unity developer using Unity 2019 or newer, please install it from scoped registry by following steps in our document.

    Highlight of changes:

    1. Update deep learning models for all platforms with improved accuracy and improved skeleton detection speed
    2. Add support for Vive Pro 2
    3. Add support ARCore in Unity/Unreal/C++ plugin
    4. Reworked smoothing to make 3d result more stable and responsive
    5. Detection is now paused when HMD is took off for PC HMDs. This uses proximity sensor on device
    6. Add support for Wave 4.0 (Vive Focus 3 only) in Unity/Unreal plugin

    For detailed changes, please refer to release notes.

×
×
  • Create New...