Monado Integration Dataflow

The dataflow for the ILLIXR Monado integration comprises two steps: 1. getting pose data from ILLIXR, and 1. sending a user rendered frame back to ILLIXR.

In Monado, ILLIXR is recognized as an HMD for Monado, while in ILLIXR, Monado looks like a user application (such as gldemo). After ILLIXR is initialized from Monado, and Monado is registered as a plugin for ILLIXR, the most recent pose information can be easily obtained via Switchboard.

The compositor side of Monado integration with ILLIXR is implemented more subtly. The original Monado compositor primarily performs distortion correction and aberration correction in a Vulkan back-end compositor. The compositor also has two client compositors (one for OpenGL applications and another for Vulkan applications) which pass frame data to the back-end compositor. ILLIXR integration intercepts the frame at GL client compositor and sends it to Switchboard of ILLIXR, which is then used by timewarp_gl component.

To get an OpenGL frame and use it without copying pixels, ILLIXR needs to get the user application GL context. This is done at OpenXR session creation time, where ILLIXR is initialized. Note that, logically, ILLIXR is initialized during OpenXR instance creation, or is otherwise running in the background all the time. Currently, ILLIXR is initialized at session creation time, since ILLIXR only supports single OpenXR session, and requires a user application GL context upon initialization,

The current ILLIXR integration for Monado is a temporary solution and has some drawbacks caused by the concurrent and continued development from both the Monado and ILLIXR projects. The integration:

  1. Does not use the pose that user application declares to use at rendering (using the OpenXR specification). This is due to incongruencies with Monado's internal interfaces and representations. The pose difference used by timewarp is computed using the most recent query for a pose update.

  2. Cannot submit frame data with a depth buffer.

  3. Cannot have poses that make use of OpenXR Spaces. Raw pose data is instead retrieved from the application's SLAM algorithms.

  4. Does not support controller action.

  5. Only supports GL user-space applications.

  6. User-space applications cannot acquire more than one swap chain buffer for each eye during the the processing of a frame.

  7. Must initialize ILLIXR during the session initialization.