Hand tracking test 0.91 Vive Cosmos

[davide445]

Testing hand tracking using the provided sample. Was not able to make it work on UE4.

The result it's in this video

https://youtu.be/5ARvNxZJvi0

Seems really unreliable so far. Setup using GTX 1070, latest 442.50 GPU driver, latest firmware provided by you (not sure if will have any role). Windows light hidden from roll-ups so no too high light message.

In this status we can't consider using it in a project, there is anything wrong in my setup? Nothing I can do to improve the situation?

I was really expecting to use hand tracking in upcoming project since using controllers will be a problem with untrained users, and need to decide what to do in a few days.

[zzy]

Hi @davide445

Please let me know what problem you encountered in UE4 so we help to solve the problem.

Regarding to the video, I think this is what to except at the moment, basically all your hands can be tracked correctly with small jitterness. There are some occasional wrong track result (especially for left hand) and we are trying to solve that issue.

To improve the situation, I think here are the things you can try:

1. Try to increase the overall brightness of your environment. We have got reports that brighter room has better performance. Please avoid sunlight directly on your hand.
2. Roll up your sleeves if any, this can help to reduce the chance of wrong track restuls.
3. Try a cleaner environment.

Please let me know if this helps.

[davide445]

Hi @zzy

This time with office lights on and sleeves rolled up. Seems a bit better especially the left hand, but as you see still shacky, and the grip it's also unreliable during rotation activities.

Sure it's a beta so room for improvement, thinking about the project needs (reliable system for unexperienced and uneducated users) I can re-evaluate this solution - I will really prefer not needing other hardware - when we will need to start developing. Any idea about a v1 release date? Also what about expected licensing terms?

Since I need anyway to push out a proposal based on hand tracking, as fail safe option there is any other commercial solution you can suggest is known working with Cosmos? As first idea I was thinking about Leap Motion but seems they are not selling it anymore.

[davide445]

PS was wrong, Leap Motion it's available for purchase on third party distributors. So the question is if there is any way to make it work with Vive Cosmos.

[zzy]

Hi @davide445

Thanks for your video test. I think the problem is when your hand is small (e.g. fist gesture) or your hand is a little further from the camera, the result is not good enough. We will try to see how can we improve this situation.

As for the v1 release, we currently don't have a solid date for v1, but we definitely hope to have it this year. That's all I can say about the release date right now. We will keep using the same license as the current release.

Also, please note that due to the nature of the cameras, some self-occluded gestures may not have good performance, even in the v1 release. I would suggest you to avoid using such gestures in your application.

[davide445]

Hi @zzy thanks for your fast answer.

Having it this year seems really too far for our project needs, this leave me no other options than third party solutions. From what you say - understandable for an optical tracking system - a hardware based solution will be probably better anyway for our target users, that I can't think educated enough to care about all these finesse.

So I know Leap Motion was perfectly working with Vive Pro, but there are connectors on the Cosmos where we can attach it? Also what can be the best position so to don't occlude cameras?

Edited March 11, 2020 by davide445

[zzy]

Hi @davide445

I haven't used Leap Motion for about 3 years, so I'm not sure if there is any place feasible on Cosmos for it. I think you have to test it yourself. As for the USB port, you might need the to connect to all the way back to your PC (ugly, but should be working).

Also, there is a new product that is announced recently: the Cosmos XR. We would expect hand tracking to perform better on it since camera is better.

[davide445]

Hi @zzy Cosmos XR didn't have any availability date you can share I suppose, so not sure if can be applicable to our project. Nor there is any information on the pricing.

Hard to say, but I'm starting considering switching to Vive Pro for this project, just for this reason. Or can I expect the hand tracking SDK working better on the Vive Pro?

[zzy]

Hi @davide445

Considering we collected more data from Vive Pro than Vive Cosmos, I think Vive Pro might perform better than Vive Cosmos, but that depends on your environment. If you already have a Vive Pro, I would suggest to test it yourself.

[davide445]

Hi @zzy

Didn't have a Pro. As another maybe more standard option, what about using the new Cosmos faceplate for external tracking and Vive knuckles with finger tracking? Not informed myself on the topic, is this working with the Cosmos?