You should make AR support universal for all games.

[pbbbbb]

I mean like some games don't know that they would work good as an AR game so it would be cool if user could enable an option to force any game into AR mode.

That way games wouldn't need to be specifically developed for AR.

[HackPerception]

@pbbbbb This generally wouldn't be kosher with developers presently. When they develop a VR game - they're generally targeting a known level of performance and reproduction quality - MR quality on current HMDs throws that out the window. That's a huge reason why VR content is progressing faster than MR/AR - you can control for more factors which impact your reproduction quality.

The latency of pass-through is currently high enough that it would bother most people if used long-term; as hardware improves, you'll likely see this type of MR become more commonplace.

That said, SteamVR can roughly achieve this with the following settings (This is going to work better on Vive Pro than Vive due to the stereo cameras):

Set you activation distance to be really high (I'd recommend experimenting with this - my current playspace is really small so I can't test):

Enable room view with translucent pass-through and ensure "show camera at room edge" is turned on

[jakelogg]

I am trying to replicate room view in game but with the addition of a chroma key feature. The closest I have come is using the beta version of this overlay application: https://store.steampowered.com/app/844060/Reality_Blender/.

However, I don't understand how room view actually works. There is obviously some sort of image scaling, stretching, and cropping going on but there is very little documentation on it, and then there must be some easy method to overlay the right and left images on top of each other. None of these appear to be accessible to developers, particularly ones that do not have access to your headsets. And yet, room view works perfectly in multiple types of headsets.

If you aren't going to allow users to to add MR through the settings, could you make it easier for developers of overlay applications so that they can?

Edited March 28, 2020 by jakelogg

[dario]

Hi @jakelogg,

For MR to be properly managed in your application you need the camera characteristics and ability as you mentioned to manage undistorted and distorted images. There's limited support in OpenVR APIs for all things XR that can be done with the front facing cameras. Starting with the Vive Pro we introduced our AR/MR "XR" SDK called SRWorks. You can read more here with example videos: https://developer.vive.com/resources/knowledgebase/intro-vive-srworks-sdk/  This SDK supports, Vive Pro, Vive Pro Eye and Vive Cosmos as well as Vive Cosmos + XR face plate (higher res cameras).

You can even manage the calibration between the camera positions and store them so that every app can use that calibration.
 

[dario]

Just want to emphasize that stereo seperation / ipd can be handled with SRWorks and that we include an overlay example. Yes, this has to be included in your app  and is not a 3rd party overlay solution for all existing apps, but you can definitely experiment with more AR/MR features until a universal overlay solution is available.

[pbbbbb]

Thanks I tried the passthrough mode on my Vive Pro and It's actually not bad! 

If the camera feed was perfectly clear it could be a killer feature for the XR faceplate.

[HackPerception]

@jakelogg If you haven't looked at this blog, it's pretty helpful. There's just not alot of OpenVR API's to interface with the cameras so your level of access will definitely have some constraints and the end-user needs to have all of the camera settings/permissions in place for the API call to work.

https://blogs.unity3d.com/2017/06/16/codesnippets-toggle-vives-front-facing-camera-and-tron-mode-at-run-time/

 

[dario]

Please check out last year's GDC Vive Developer day presentations and we'll have a new GDC 2020 sessions shortly as webinars. You're not limted to that old Unity blog anymore (plus OpenVR has changed since).

Again the SRWorks SDK is your best bet for Vive Pro and Vive Cosmos and the upcoming XR faceplate for Cosmos.

Upcoming webinar sessions:
https://blg.vive.com/us/2020/03/24/htc-vives-gdc-sessions-go-live/

Last year's sessions:
https://www.youtube.com/watch?list=PL1JC6N9u4ERirCMPljaSzud3UNNrM2PNB

 

Again this is AR for developers to include in their apps - for overlay solutions that work arcoss any app I would recommend the aardvarkxr project: https://github.com/aardvarkxr/aardvark

You can always play games currently with the pass through games on if you set the mode to transparent (no filter effects) for applications with black backgrounds it works well - e.g. TiltBrush.

 

 

 

 

Edited March 31, 2020 by Dario
clarification

[Zzzilly]

On 3/23/2020 at 3:01 PM, VibrantNebula said:

@pbbbbb This generally wouldn't be kosher with developers presently. When they develop a VR game - they're generally targeting a known level of performance and reproduction quality - MR quality on current HMDs throws that out the window. That's a huge reason why VR content is progressing faster than MR/AR - you can control for more factors which impact your reproduction quality.

The latency of pass-through is currently high enough that it would bother most people if used long-term; as hardware improves, you'll likely see this type of MR become more commonplace.

 

 

Enable room view with translucent pass-through and ensure "show camera at room edge" is turned on

The latency can be an issue when a scene is busy but this translucent pass-through is a great trick to supplement with the SRWorks pass-through. It's good to realize that this can be adjusted with the Vive SR_HMD Camera Shifter script. Thanks for pointing this out. The translucent pass-through is misaligned vertically - a stereo no-no.  SRWorks aside, is there a way to shift vertically?

 

On 3/28/2020 at 4:45 PM, Dario said:

Just want to emphasize that stereo seperation / ipd can be handled with SRWorks and that we include an overlay example. Yes, this has to be included in your app  and is not a 3rd party overlay solution for all existing apps, but you can definitely experiment with more AR/MR features until a universal overlay solution is available.