Jump to content
 
duelgame

Hand Tracking's need to support Unreal Engine

Recommended Posts

I've tested Vive Hand Tracking in Unreal Engine 4.,24

CPU : I7-7700k
GPU : GTX 1660
RAM : 32Gb
Vive Cosmos (https://www.vive.com/us/product/vive-cosmos/overview/)
ViveHandTracking 0.9.2

image.thumb.png.d0fb600e184555df7a6ad2cd0d79a55d.png

a79625a7-da96-e156-0639-37c222ac275c

6db8ddbc-bb60-1232-236f-9b0cc0f17e7c

c9fa2a7e-c15f-76ee-502c-9fa61bdc0d9f

 

I saw this post (https://forum.vive.com/topic/8459-vive-hand-tracking-camera-resolution/#) and was wondering if only Unreal had issues with performance. So I tested the Unity samples using RenderDoc. Here are the results below:

05e9a689-21a3-51e2-d129-db0d8ae29394

 

Unity sample used two textures of 1584x1872.

It rendered the Left Eye and Right Eye. separately.

Basically Unreal Engine uses Instanced Stereo Rendering.

Even if I turn off the Instanced Stereo Rendering, the Resolution is not changed.

eed2d1c9-ecaf-04e9-ec26-ef4c293f690e

I think VIVE hand tracking uses image processing so it's performance is greatly affected by the resolution it processes.

The performance is fine when I test Unreal Sample with vr.PixelDensity 0.5

image.thumb.png.3db70bcd61e4ed02776e1a08c0c9d04c.png

 

Unity has a similar option called single pass stereo rendering.

https://docs.unity3d.com/Manual/SinglePassStereoRendering.html

So I think VIVE needs to support Instanced Stereo Rendering and Single Pass Stereo Rendering to improve the overall FPS.

I need to know your thoughts on this and whether you have plans to update the SDK to improve performance. 

 

 

 

 

 

 

Share this post


Link to post
Share on other sites

Hi @duelgame

I cannot see most of the pictures you attached, but here is my answer based on text in your question.

To make it clear, here is hand tracking process: Camera frame -> Hand Tracking Process (GPU) -> output result in engine plugin. This has nothing to do with the render texture size, or vr.PixelDensity.

However, VR rendering also uses GPU. Hand Tracking and render shares the GPU resource, so if you reduce workload for either side, you can observe performance improvement. This is case when you reduce rendering workload (i.e. by reducing vr.PixelDensity or using Instanced Stereo Rendering). You can observe the same thing by using 3d point mode instead of skeleton mode in Vive Hand Tracking.

I'm not expert for Unreal, but I think you need to check SteamVR plugin for Instanced Stereo Rendering support in Unreal.

Share this post


Link to post
Share on other sites

@zzy

I attached a PDF file for the images above in case you don't see them.

There seems to be no performance increase when I use 3d point mode .

Only rendering a yellow plane, two blue hands that are using basic shaders and few triangles shouldn't be consuming much GPU resources... 

How can I make a simple vr game with this if the performance is so low?

If this is an Unreal issue, there should be someone that knows Unreal in the VIVE team that is able to handle these type issues. 

 

01.pdf

Share this post


Link to post
Share on other sites

Hi @duelgame

I can see all the images now, I think both Unreal and Hand Tracking need to optimize GPU usage:

  1. Next release version of Vive Hand Tracking will improve GPU usage (target end of this month).
  2. From a quick google search, I found Instanced Stereo Rendering support is already in VR: https://docs.unrealengine.com/en-US/Platforms/VR/DevelopVR/VRPerformance/index.html. Is there any reason that you cannot enable this feature?

We will keep improving GPU usage in future versions, but please understand currently improving accuracy has higher priority.

 

Share this post


Link to post
Share on other sites

Hi @duelgame

v0.9.3 is now released, you can download it from here. It improves GPU usage in hand tracking part. Please have a try.

Share this post


Link to post
Share on other sites

Please sign in to comment

You need to be a member in order to leave a comment

Sign in

Already have an account? Sign in here.

Sign In Now

×
×
  • Create New...