offload_rendering_client

Summary

offload_rendering_client receives encoded frames from the network, and uses a mix of NPPI and FFMPEG, or NVDEC to decode the frames before updating the buffer pool. Setting the environment variable ILLIXR_USE_DEPTH_IMAGES to a non-zero value indicates that depth images are being received, and should thus also be decoded.

When the client is an Android device with its own OpenXR runtime (e.g., Quest 3) you can enable overscanning to allow the onboard timewarp to produce complete images for display. To enable this feature set the ILLIXR_OVERSCAN environment variable in main.cpp to the scale factor you want to use. Testing has found a value of 1.1 works well for the Quest 3. You will also need to supply the same environment variable and value on the server side.

Note that there is a known color shift issue (to be fixed), where the decoded frame's colors are slightly different from the original frame (likely due to the many conversions between YUV and RGBA).

Please refer to the README in the network backends for setting the server and client IP address and port number.

Android Builds

If you are using an Android device for the offload rendering client, you must use NVENC encoding on the server. See the server documentation page for details. The Android decoder supports AV1 and HEVC formats, which must also match the encoder settings in the server. The encoder is chosen by setting the appropriate CMake arguments in app/build.gradle in the externalNativeBuild section. The Android code has been tested on a Quest 3 headset and may contain some code that is device specific.