Improve Performance & Battery Life with Adaptive Quality in Wave SDK

[MariosBikos]

We realise how important it is for the development community to have easy-to-use tools that improve content performance. We recently released the new update of Wave SDK 3.1.94 [Early Access] with several Experimental Features for developers who create content for standalone VR headsets of the Wave Ecosystem such as the Vive Focus Plus and Vive Focus. In this update, we introduced changes to the Adaptive Quality feature, which can help to automatically adjust the rendering quality of your VR application according to the system workload in order to achieve better performance and improve battery life by up to 15%. This blog post will explain more about Adaptive Quality, why it is important and how to apply it to your own projects. We will provide an overview of the solution and its design/implementation and share a few tips on how developers can get started using it. We also describe how it works in synergy with other features of the Wave SDK such as Dynamic Fixed Foveated Rendering and Dynamic Resolution for better results.

* Please note that Wave SDK 3.1.94 is an Early Access version that includes new features for developers to experiment with and provide feedback or suggest changes. These features are available only with a specific developer ROM update (and Adaptive Quality requires ROM v3.07.623.336 for Focus Plus) but content developed with this Developer ROM can't be published to Viveport until we release a public ROM update (coming soon!). Please refer to this article for more information on how to get access to the Developer ROM and test Wave SDK 3.1.94.

Introduction
Standalone VR headsets may have all the necessary components required to provide VR experiences, but unlike PC-based VR headsets, utilising the full power of their hardware requires an intricate balance for VR apps to run smoothly and with consistent performance. Heat generated from headsets working extra hard trying to render VR content can result in Throttling. The hardware will detect the high temperatures and when a predefined limit is crossed it will attempt to lower the clock speed of the CPU/GPU to prevent the system from overheating. When the temperature levels get back to normal, the system will increase the CPU/GPU clock speed and performance will bounce back again. Unfortunately, this process may be repeated and leads to poor battery life and inconsistent performance as the system is not able to quickly get rid of the generated heat. Although developers can mitigate this issue by trying to make sure games perform at their best at all times, this is not always possible.

Adaptive Quality
Adaptive Quality works by providing a way for developers to balance the performance of their VR applications and power consumption in real time, offering automatic adjustment of the CPU/GPU performance according to the workload of the system. Furthermore, it allows defining a set of strategies on how the application should respond to system changes to improve FPS when the rendering performance is insufficient.

Adaptive Quality can be combined with the Dynamic Resolution feature to adjust the image quality of the application according to system changes and with Fixed Foveated Rendering to dynamically change the quality of the peripheral region when improving the performance is essential. This results in better battery consumption management and smooth frame rates, as developers have more control and can create and customise their own policies to dynamically handle hardware changes. Especially for GPU fragment-bound apps, this leads to less throttling and a better experience for the end users. 

Adaptive Quality v1.0 was first introduced in Wave SDK 3.1.1 supporting automatic CPU/GPU adjustment and system events according to workload. Starting from Wave SDK 3.1.94 we introduced new features and changes (v2.0) adding Dynamic Resolution and Fixed Foveated Rendering to the mix.

 

 

Auto CPU/GPU Adjustment

Standalone VR headsets are powered by a battery so making sure that power is not drained too quickly is important.

With Adaptive Quality, we’ve made the management of CPU and GPU clock rates much simpler by making it almost entirely automatic. If Adaptive Quality is enabled, the system can dynamically change the CPU and GPU performance level to maintain performance based on the system load. So when the performance of your application is insufficient, CPU/GPU clock speeds will increase to improve the FPS. Likewise, if the application already runs at high FPS and the complexity of the scene is low, Adaptive quality can scale down the clock rates to save battery power for the headset and prolong its usage.

Although we don’t provide direct access to the maximum/minimum-allowed clock speeds, Adaptive Quality can configure those properties based on its knowledge about the current system load and define if the levels should be lowered or raised. Of course, when Adaptive Quality is disabled, developers can still manually increase/decrease the CPU/GPU performance based on practical demands according to this API.

 

System Events & Custom Policy

Although we can change the clock rates to reduce power consumption or improve performance, that may not always be enough to achieve better results. Sometimes, more things need to change within the VR application software itself to get a stable performance. Adaptive Quality can be configured to broadcast events whenever there are changes to the system workload. This mechanism is really useful, since developers can create and customise their own policy to reduce GPU load and ensure constant frame rates over a longer period of time.

Depending on the situation, the VR application can react differently according to those events and developers may create their own policies and choose whether and how to handle those changes and change scene complexity themselves. Developers can subscribe to receive performance events that recommend lowering or raising the rendering quality of the application and actively modify the rendering settings of the application in an equivalent way by enabling or disabling MSAA or other rendering settings that can help boost performance.
 

Dynamic Resolution

Dynamic Resolution is a feature that works together with Adaptive Quality and helps adjust the image quality of the application by changing the eye buffer size according to the events broadcasted (that we mentioned in the previous section). More specifically, the Resolution Scale will be increased automatically when the event received denotes that the quality can be higher and similarly, when the event type received  recommends lowering the quality of the running application, the resolution scale will be decreased.

In order to ensure that the Resolution Scale will not decrease to a point where the application is unusable, Dynamic Resolution comes with built-in functionality that helps determine the lower bound of Resolution Scale for different VR devices to maintain text readability. What’s great about this feature is that there is no extra latency introduced with the change in resolution scale.

Dynamic Fixed Foveated Rendering

Foveated Rendering is a technique that exploits the anatomy of the human eye and suggests that applications can drop the quality of graphics in the peripheral vision by lowering the resolution in that region while focusing all of the available processing power on a smaller area of the image(foveated region).

The term “Foveated” derives from the word “Fovea”, the part of the eye retina that allows us to have a sharp central vision (Foveal Vision) which lets us focus on the visual details of what is important in a narrow region right in front of us. Anything outside the fovea region belongs to the peripheral vision and despite the fact that it is also useful, it can only detect fast-moving content changes and color, hence why it feels comparatively less detailed and blurry.

It’s worth noting that there are 2 types of Foveated Rendering and the terms are sometimes confusing:

• Fixed Foveated Rendering assumes that the foveated region should always be at the center of the field of the view of the user and that lower resolution pixels should be rendered at the distortion region around the lens as things are already not clearly visible there.
• Eye-Tracked or Dynamic Foveated Rendering can be used with headsets that support eye tracking modules (Check Vive Pro Eye) to accurately define the foveated region based on gaze direction. It’s called dynamic because as the human eye moves, the foveated region keeps changing and the peripheral region keep changing dynamically.

Adaptive Quality can be combined with Fixed Foveated Rendering to help increase the performance of VR applications according to system workload (dynamically). Fixed Foveated Rendering can be automatically enabled by Adaptive Quality to further reduce the GPU load and improve performance whenever it’s required.
 

Results

The benefits of Adaptive Quality can become more clearly articulated through some examples. The graph below illustrates how Adaptive Quality helps deliver smooth and high frame rates with a native fragment-bound application. In green, you can see the frame rate fluctuating when Adaptive Quality is disabled, with FPS going down as GPU workload increases. In pink, you can see the stable results after enabling Adaptive Quality running at 75 FPS. The x-axis indicates the time passed since the start of the application and the y-axis shows the FPS on the left and the GPU Level on the right. The yellow letters show the fragment loading values that increase every 5 seconds. Notice how the GPU clock rates go down when the FPS are high to save power(after 11 seconds and 61 seconds). 

 

Another graph below shows the results of a test using a Unity application and a GPU fragment-bound application. When Adaptive Quality and Dynamic Resolution are both enabled there is an increase of 13 FPS on average. In green you can see the case where Adaptive Quality is disabled, while in yellow you can notice how Dynamic Resolution tries to decrease the resolution scale to improve FPS and when FPS are high enough we increase the resolution scale back up again. 
 

 

We also tested Adaptive Quality with other applications such as Viveport’s Sureshot Game and SPGM. As you can see from the results, the energy consumption was improved by 10% and 15% respectively, indicating how useful it can be to use this feature in order to extend the battery life of the VR headset. 

 

How to use Adaptive Quality

WaveVR SDK 3.1.0 to 3.1.6:

• Adaptive Quality is not enabled by default. The Wave SDK provides an API function called  WVR_EnableAdaptiveQuailty that needs to be called manually so that the CPU/GPU performance levels can be adjusted automatically.
• The system events WVR_EventType_RecommendedQuality_Lower or WVR_EventType_RecommendedQuality_Higher will be broadcasted based on system workload and can be monitored to initiate actions according to them using the Wave System Event. Check System event to know how to listen these system events.

WaveVR SDK 3.1.94 or later:

• Adaptive Quality is enabled by default and system events are broadcasted to change rendering quality when needed.
• One or multiple strategies can be used to adjust display quality and improve the FPS when performance is insufficient (WVR_QualityStrategy_Default, WVR_QualityStrategy_SendQualityEvent, WVR_QualityStrategy_AutoFoveation)

Unity

The WaveSDK package for Unity supports all the features of Adaptive Quality including the new features introduced with Adaptive Quality 2.0 (Dynamic Resolution and Dynamic Fixed Foveated Rendering) and it’s really easy to use. The WaveVR_AdaptiveQuality script can be used to enable the AdaptiveQuality feature. However,  starting from WaveVR 3.1.94, this component will be pre-attached to the WaveVRAdaptiveQuality GameObject of the WaveVR Prefab. In this case, WaveVR_AdaptiveQuality is enabled by default.

As you can see in the screenshot below, the WaveVRAdaptiveQuality game object is part of the WaveVR game object that is required to build VR applications that support the Wave ecosystem. There are 2 scripts attached to it:

• WaveVR_AdaptiveQuality: When this script is enabled, automatic CPU/GPU clock adjustment will take place and developers can tick the boxes under the Rendering Performance Improve Strategy section to define what strategies should be used (e.g I want system events to be broadcasted and Fixed Foveated Rendering to be enabled. In this case I need to tick both boxes).
• WaveVR_Dynamic_Resolution: This script is responsible for the Dynamic Resolution feature that adjusts the resolution scale of the VR application according to workload. An list of Resolution Scale values can be defined that will be used to adjust the resolution scale whenever there are events triggered by Adaptive Quality. Also the Text Size slider can be used to define the smallest size of text that will be used in the application to avoid having Dynamic Resolution making text unreadable.
   

 

Unreal Engine

Adaptive Quality can also be used with the Wave Plugin in Unreal Engine. Two functions are provided to enable the Adaptive Quality Feature and query whether Adaptive Quality is enabled or not. Although Auto CPU/GPU Adjustment is automatically supported when Adaptive Quality is enabled, Dynamic Resolution and Dynamic Fixed Foveated Rendering can’t be utilised at the moment with this plugin. There will be more updates on this soon. Wave 3.2 is expected to be released soon, adding support for system events in Unreal Engine.

 

 

Summary Table

 

Best Practices & Tips

Now that you know more about Adaptive Quality here are some tips and advice:

• Rendering improvements by WaveVR AdaptiveQuality has limits. It is still highly recommended to optimise your app as much as possible first. Check our Mobile VR performance optimisation tips.
• Enabling WaveVR AdaptiveQuality can help lightweight VR apps, such as photo or video playing apps, be used longer.
• Enabling WaveVR AdaptiveQuality with WVR_QualityStrategy_SendQualityEvent and WVR_QualityStrategy_AutoFoveation can improve the rendering quality by up to 15% if the rendering bottleneck is GPU fragment processing bound.
• Always build optimized versions of the application for distribution. Even if a debug build performs well, it will draw more power and heat up the device more than a release build.
• Power management is a crucial consideration for Android VR development.
• Wave SDK force disables Adaptive Quality during map loading to increase performance and restores Adaptive Quality status after the map is loaded.
   

Useful Links

To implement Adaptive Quality in your own application, check the documentation pages below:

• Wave Native SDK
• Unity Integration
• Unreal Engine Integration

Also see:

• System Event & how to listen to events
• Foveated Rendering
• Dynamic Resolution

We gave a talk about Adaptive Quality and the new features introduced in the latest Wave SDK during the Virtual Vive Ecosystem Conference back in March 2020. You can watch the presentation below:

What do you think?
Feel free to try this feature and provide feedback from your tests in our forums.
You can also find the complete list of new features for each Wave SDK update in our release notes.