Jump to content
 

zzy

Employee
  • Content Count

    170
  • Joined

  • Last visited

Posts posted by zzy


  1. Hi @Tima

    Let me try to explain this. Please remember is that hand tracking is running at 40-60 fps, so the hand raw data is updated at lower frequency than VR rendering.

    When working in camera space, you apply current HMD transform to GestureResult every frame. When HMD is moving fastly, GestureResult does not change quite often, this results in virtual hand moves with HMD while hand is staying still in reality.

    When working in HMD space, the HMD transform is not applied to GestureResult every frame, instead the HMD transform is applied only raw hand position is calculated. Therefore, no matter how your HMD moves, hand stays still in the virtual environment.

    So in this case, it's better to use HMD space. There is also other cases, where it's better to use camera space. For example, when you moves HMD and hand together, making hand relative position to HMD not changed.


  2. Hi @Tima

    To solve this problem, please add the following lines to interface_gesture.hpp:

    struct GestureVector3 {
      float x, y, z;
    };
    
    struct GestureQuaternion {
      float x, y, z, w;
    };
    
    INTERFACE_GESTURE_EXPORTS void SetCameraTransform(GestureVector3 position,
                                                      GestureQuaternion rotation);

    Please call SetCameraTransform function every VR frame (i.e. 90 times per second) to pass the HMD transform to native side. Like GestureResult, position and rotation are in Unity axis. After calling SetCameraTransform function, you no longer need to apply HMD transform to GestureResult.

    Note: This change will also be available starting from next release.


  3. Hi @Tima

    Do you call UseExternalTransform(true) before StartGestureDetection? This could be the cause for the problem, as you are applying hmd transform on GestureResult, making your hands move with your HMD.

    By the way, since the raw camera fps is 60, setting maxFPS above 60 has no use.


  4. Hi @Tima

    The current plan for next release is about early February.

    About grab strength, do you have any special use case that cannot use pinch strength?

    Also, do you have any suggestion about how to calculate grab strength from skeleton? I would think of using average closing angle of each finger, but not sure how to handle the case when some fingers are closed while others are stright (e.g. point/two gesture).


  5. Hi @kilik

    For first question, another thing I can think of is to install MSVC 2015 update 3 runtime on your PC. The dll not found problem might be caused by failure to load dependency dll.

    For second question, if your image is not correct, then it's possible hand detection fails. If you are hand in editor, is that using EditorEngine (which generates fake hand data)? For camera problem, I would suggest to follow steps here: https://hub.vive.com/storage/tracking/misc/steamvr.html


  6. Hi @ericvids

    Yes, it's not normal why the program picked up Intel graphics but not NVIDIA one. I would suggest to check if your NVIDIA driver and OpenCL function is installed correctly. I recommend to download the latest driver from https://www.nvidia.com/Download/index.aspx?lang=en-us

    Also, thanks for your suggestion for the log. I'll add it in the next version. For now, you can modify `AssetsViveHandTracking\Scripts\Engine\ViveHandTrackingEngine.cs` (about line 127) and add the log:

    if (index <= lastIndex)
      return;
    if (index > 0 && lastIndex == 0) Debug.Log("Detection initialization completed");
    lastIndex = index;

     


  7. Hi @ericvids

    Indeed, thre is no error message from unity log. So I assume the hand tracking is running (or still initializing).

    To run on Windows, our SDK will neeed to compile OpenCL kernels at runtime, which can be quite time consuming for the first run. This might take a few minutes before the hands can be detected. Please make sure you let the program run for a few minutes and check if hands are detected or not.

    There also can be other reasons which caused detection failure. I would suggest to upgrade GPU driver to latest version from NVIDIA/AMD official site.


  8. Hi everyone,

    Vive Hand Tracking SDK has a new release v0.9.4 today. You can download it from here.

    Highlight of changes:

    1. Update deep learning models for all platforms.
    2. Add the option to limit max raw detection fps to save computing resources.
    3. (Unity) Add assembly definition files as separate package for Unity 2019 or newer.
    4. (Unity) Add support for Wave XR Plugin.
    5. (Unreal) Move native binary to a module in the project. All the contents of plugin are inside project folder.
    6. (Unreal) Experimental support for WaveVR Hand API (requires WaveVR 3.2.0).
    7. (C/C++) Add x86 binary.

    For detailed changes, please refer to release notes.

    • Like 1

  9. Hi @ericvids

    After a quick test, I cannot reproduce this problem locally. My steps are listed below. Please let me know if I missed something.

    1. Unity version is 2018.4.19f, Vive Hand Tracking SDK 0.9.3, SteamVR plugin 2.6.1 (latest from Asset store). SteamVR version 1.14.16
    2. I used SteamVR_TestTrackedCamera example scene as a start point, since this already has a camera texture. Please first make sure this scene alone can work fine.
    3. Attach GestureProvider script to [CameraRig]/Camera game object.
    4. For display purpose, add LeftHandRenderer and RightHandRenderer prefabs to the scene. Prefabs are inside Assets/ViveHandTracking/Prefab folder.
    5. Play the scene now, you can see both hand skeleton and camera texture. Tested on both Vive and Vive Pro.

  10. Hi @ericvids

    From my previous experiences, I think cameras from SteamVR can be accessed by multiple callers. A quick example is that when hand tracking is running, you can still see the camera preview in camera tab from SteamVR settings.

    As for how to access the camera frame/textures, you can refer to the OpenVR sample code here: https://github.com/ValveSoftware/openvr/tree/master/samples/tracked_camera_openvr_sample

    I'll have a check next week to see if I can reproduce this on my machine.


  11. Hi @毛.sy

    Can you please try if the following changes solve your problem?

    Please change UpdateResult function in Assets\ViveHandTracking\Scripts\Engine\ViveHandTrackingEngine.cs to:

      public override void UpdateResult() {
        var transform = GestureProvider.Current.transform;
    
        IntPtr ptr;
        int index;
        var size = GestureInterface.GetGestureResult(out ptr, out index);
    
        if (index < 0) {
          Debug.LogError("Gesture detection stopped");
          State.Status = GestureStatus.Error;
          State.Error = GestureFailure.Internal;
          return;
        }
        if (index <= lastIndex)
          return;
        lastIndex = index;
        if (State.Status == GestureStatus.Starting)
          State.Status = GestureStatus.Running;
    
        if (size <= 0) {
          State.LeftHand = State.RightHand = null;
          return;
        }
    
        bool isLeft = false;
        var structSize = Marshal.SizeOf(typeof(GestureResultRaw));
        for (var i = 0; i < size; i++) {
          var gesture = (GestureResultRaw)Marshal.PtrToStructure(ptr, typeof(GestureResultRaw));
          ptr = new IntPtr(ptr.ToInt64() + structSize);
          for (int j = 0; j < 21; j++)
            gesture.points[j] = transform.TransformPoint(gesture.points[j]);
          State.SetRaw(gesture);
          isLeft = gesture.isLeft;
        }
        if (size == 1) {
          if (isLeft)
            State.RightHand = null;
          else
            State.LeftHand = null;
        }
      }

     

×
×
  • Create New...