Android hand tracking - starting tips ?

[VirtualPierogi]

Hello, 
im trying to start hand tracking module on my samsung s8, and I had not achieved anything yet,
even after manually enabling the camera permissions, there were no feedback .

What should one do to start android hand tracking hello world style?

[HackPerception]

,

 

I'm pretty sure that this SDK strictly provides support for Vive branded VR devices at this point in time - it's not designed with the intent to support other devices.  will be able to confirm or correct me on this. 

[zzy]

Hi  

 

We have basic support for Android devices. Please read the Android support document for Unity plugin, Android plugin or C++ plugin. One thing to note, your screen orientation must be Landscape Left.

 

Best Regards,

zzy

[VirtualPierogi]

Okay,
so ive compiled the sample provided from developer site along with the hands plugin, on daydream, removed aristo wavevr and referenced script define symbols, added google vr symbols, made sure that everything is landscape left
but

after compiling no hand tracking is seen.
Anybody can point me to the right direction?

[zzy]

Hi  

 

One thing first, if you are only using Vive Hand Tracking SDK with GoogleVR plugin and not use with WaveVR plugin, the default editor script should work for you. No need to manually remove libraries or modify script define symbols.

 

Now back to your question.

 

1. Please make sure camera permission is granted to your app.

2. Are you enabling Daydream as XR? If so, please make sure your cellphone's rear camera is not blocked by the Daydream viewer. From my knowledge, the daydream viewer blocks the rear camera and thus we cannot detect any hand. I would recommend test without daydream viewer, or use a cardboard viewer that does not block the camera.

 

If you still encounter the problem after you unblock the camera, please attach the adb log for us to investigate.

You can use the command `adb logcat -s Unity Aristo` to include all the logs from Unity and our native plugin.

 

Best Regards,

zzy

[VirtualPierogi]

1. Camera is unobstructed and all permissions are given 

2. Nothing (no hand interaction or bones are seen)

3. adb log goes as follows :

 

05-26 15:41:35.288 17353 17387 I Unity   : SystemInfo CPU = ARMv7 VFPv3 NEON, Cores = 8, Memory = 3679mb05-26 15:41:35.288 17353 17387 I Unity   : SystemInfo ARM big.LITTLE configuration: 4 big (mask: 0xf0), 4 little (mask: 0xf)05-26 15:41:35.289 17353 17387 I Unity   : ApplicationInfo com.XRPierogi.XRtest version 0.1 build 3ef1d2df-e88b-4ffd-915c-760db1216eb805-26 15:41:35.289 17353 17387 I Unity   : Built from '2018.3/release' branch, Version '2018.3.6f1 (a220877bc173)', Build type 'Release', Scripting Backend 'mono', CPU 'armeabi-v7a'05-26 15:41:35.631 17353 17387 E Unity   : Unable to find libaudioplugingvrunity05-26 15:41:37.777 17353 17387 I Unity   : Created eye textures with a "separate" layout.  The "multi-pass" stereo mode will be used.05-26 15:41:37.777 17353 17387 I Unity   :05-26 15:41:37.777 17353 17387 I Unity   :05-26 15:41:37.777 17353 17387 I Unity   : (Filename:  Line: 2264)05-26 15:41:37.777 17353 17387 I Unity   :05-26 15:42:14.731 17353 17387 E Unity   : NullReferenceException: Object reference not set to an instance of an object05-26 15:42:14.731 17353 17387 E Unity   :   at Draw.OnEnable () [0x00006] in <24d534f4e3d94938bcab8bc32901764a>:005-26 15:42:14.731 17353 17387 E Unity   :05-26 15:42:14.731 17353 17387 E Unity   : (Filename: <24d534f4e3d94938bcab8bc32901764a> Line: 0)05-26 15:42:14.731 17353 17387 E Unity   :05-26 15:42:15.027 17353 17387 E Unity   : AndroidJavaException: java.lang.NullPointerException: Attempt to invoke virtual method 'int android.app.Activity.checkSelfPermission(java.lang.String)' on a null object reference05-26 15:42:15.027 17353 17387 E Unity   : java.lang.NullPointerException: Attempt to invoke virtual method 'int android.app.Activity.checkSelfPermission(java.lang.String)' on a null object reference05-26 15:42:15.027 17353 17387 E Unity   :      at com.google.gvr.permissionsupport.PermissionsFragment.hasPermission(PermissionsFragment.java:88)05-26 15:42:15.027 17353 17387 E Unity   :      at com.unity3d.player.UnityPlayer.nativeRender(Native Method)05-26 15:42:15.027 17353 17387 E Unity   :      at com.unity3d.player.UnityPlayer.c(Unknown Source:0)05-26 15:42:15.027 17353 17387 E Unity   :      at com.unity3d.player.UnityPlayer$e$2.queueIdle(Unknown Source:72)05-26 15:42:15.027 17353 17387 E Unity   :      at android.os.MessageQueue.next(MessageQueue.java:395)05-26 15:42:15.027 17353 17387 E Unity   :      at android.os.Looper.loop(Looper.java:181)05-26 15:42:15.027 17353 17387 E Unity   :      at com.unity3d.player.UnityPlayer$e.run(Unknown Source:32)05-26 15:42:15.027 17353 17387 E Unity   :   at UnityEngine.AndroidJNISafe.CheckException () [0x00091] in <303b1ed396ed420184be31eb8dc21cdb>:005-26 15:42:15.027 17353 17387 E Unity   :   at UnityEngine.AndroidJNISafe.CallBooleanMethod (System.IntPtr obj, System.IntPtr methodID, UnityEngine.jvalue[] args) [0x00011] in <303b1

[zzy]

From the log, the problem is caused by the AndroidJavaException part, and the detection is never started.

 

If you have already granted camera permission, you can try to remove the ARISTO_WITH_GOOGLEVR script define symbol and try agin.

 

In the meanwhile, I'll try to reproduce it on my side and see if it can be fixed in next version.

Can you please share some of your project info:

1. Version of Unity

2. Version of Google VR plugin

3. Current active XRDevice (I assume it's daydream).

[zzy]

Hi  

 

I haved tested on my side and it seems to work with latest Unity and GoogleVR. Here are my steps:

1. Create a new 3D project using Unity 2019.1.3f1

2. Import Vive Hand Tracking SDK v0.8.0 and Google VR SDK v1.200.0. Please note after you imported both SDK, the ARISTO_WITH_GOOGLEVR define is added automatically.

3. Switch platform to Android and change some player settings to make sure build can succeed:

    1. Change company name, product name and Android package name

    2. Enable Daydream in XRSettings, and remove Vulkan from Android Graphics API (since Unity does not support Vulkan + XR yet)

    3. Change Android Min API level to 24

4. Build and run on Pixel. When the app is run for the first time, it calls daydream API to grant camera permission, you need to take the phone off the viewer, hold it as portrait, accept camera permission and put it back to viewer.

 

Note: You don't need to change the binaries or defines manually.

 

Please let me if this steps works on your side or not. If you are using different versions, please let me know.

[VirtualPierogi]

Hey,
did everything as prescribed and added the sample scene from examples unitypackage for hand tracking, but still nothing happens when i use gestures shown on the infographics.
Should I change the the gesture provider type from skeleton to point 2d or 3d?
Or make empty scene with all the prefabs only just to test if it works on Samsung 8?

 

[zzy]

Hi  

 

Are you still experiencing the same error from adb log?

 

I think you might not have Daydream setup on your Samsung S8. You need to install the Daydream app from Google play and setup the viewer to make a daydream app running. If you don't have a daydream viewer, I suggest you use Cardboard as your XRDevice instead.