Skip to content

Glossary of ILLIXR Terminology

A collection of ILLIXR and ILLIXR-adjacent terms and their definitions can be found on this page your reference.

General

Runtime

The ILLIXR system runtime is responsible for the dynamic orchestration of ILLIXR device resources, system resources, and client applications.

The runtime implementation is located in ILLIXR/runtime/. See the Building ILLIXR and Monado Overiew pages for details about the ILLIXR runtime.

Plugin

A modular component that can be detected and enabled for use by an ILLIXR application. A plugin can be internal or external to the ILLIXR project. Each plugin is compiled and launched dynamically at runtime based on the ILLIXR configuration used. ILLIXR also implements a Monado runtime translation Plugin.

For a list of supported plugins and their details, see the ILLIXR Plugins page. For instructions for how to modify or write your own plugins, see the Modifying a Plugin and Writing Your Plugin pages.

See the Plugin API documentation.

Config(uration)

A file describing the key information required to launch an ILLIXR application. Configurations for ILLIXR are implemented as YAML files. Each configuration comprises an action, a profile, and a list of plugins as defined by our configuration specification Schema.

  • Action (Previously Loader): An action encapsulates a task for Runner.

    • native: The default application launch configuration. Does not use our Monado runtime integration. Defined in ILLIXR/configs/native.yaml.

    • native-lookup: Same as native, but using a ground truth lookup from a file for the pose instead of computing it. Defined in ILLIXR/configs/native-lookup.yaml.

    • headless: Same as native, but using Xvfb to run without a graphical environment. Defined in ILLIXR/configs/headless.yaml.

    • ci: Same as headless, but using Docker virtualization and debug-enabled compilation. Defined in ILLIXR/configs/ci.yaml.

    • monado: Similar to native, but uses our Monado runtime integration. Defined in ILLIXR/configs/monado.yaml.

    • clean: A meta-task that fetches all supported plugins and then cleans up builds across the entire ILLIXR project. Defined in ILLIXR/configs/clean.yaml and supported by ILLIXR/clean.sh.

    • docs: A meta-task that generates and populates the documention subdirectories in the project.

  • Profile: A profile captures the compilation mode used by Runner.

    • opt: Sets Runner to compile the ILLIXR application and plugins with optimizations.

    • dbg: Sets Runner to compile the ILLIXR application and plugins without optimizations, while enabling debug logic and debug logging.

  • Schema: A schema captures the specification describing the allowable structure of a configuration file. Our schema is implemented using the json-schema specification. Defined in ILLIXR/runner/config_schema.yaml.

For more details about the structure of a configuration, see the Building ILLIXR page.

Framebuffer

A region of memory used to hold graphical information to be output to a display or graphics device.

  • Frame: A single frame (image) to be output to a display at a certain instant of time based on the system's frame rate.

  • Frame Rate: The interval period between complete (as defined by the output resolution) frame updates and refreshes. In many systems, the target frame rate is determined by a fixed vertical sync (VSYNC) period.

  • Depth Buffer: A framebuffer representing the depth information of a 3D scene. Depth information is useful for applications such as graphics and SLAM.

  • Eye Buffer: A framebuffer dedicated for display through a HMD lens to be perceived by a user's eye.

For more information, see the Wikipedia article.

Swap Chain

A set of virtual framebuffers to be output to a display. Only one framebuffer in a swap chain is displayed at a time, enabling the other virtual framebuffers to be concurrently modified in memory.

For more information, see the Wikipedia article.

Compositor

A window manager that establishes a framebuffer for each window of a graphical system. A compositor merges information across its windows to construct a unified framebuffer.

For more information, see the Wikipedia article.

Head-mounted Display

A display device worn on the head and face for use with VR and XR applications. Also known as a HMD.

For more information, see the Wikipedia article.

Eye Tracking

The process of measuring the eye movement of a user (who is possibly also wearing a HMD).

For more information, see the Wikipedia article.

Event Stream

A communication interface supporting writes, sychronous reads, and asynchronous reads. For synchronous reads, every value written to the stream is visible to consumers. For asynchronous reads, only the latest values written are guaranteed to be visible to consumers.

Pose

The combination of orientation and position of an object, used for computer vision and robotics applications. ILLIXR applications make use of poses to track the user's HMD within the virtual environment. Internally, ILLIXR has multiple classifications of poses which are used for various purposes.

  • Slow Pose: A slow pose is a ... TODO

  • Fast Pose: A fast pose is a ... TODO

  • True Pose: A true pose is a ... TODO Depracated starting ILLIXR release v2.X.X.

  • Pose Prediction: To improve the user's perception latency experience the time between, pose prediction uses history and current system information to pre-compute the user's next pose Pre-computing the next pose allows for components downstream from the pose output in the event stream dataflow graph to begin computation.

Pose Prediction is implemented in the pose_prediction ILLIXR plugin.

For more information, see the Wikipedia article.

Ground Truth

The most accurate source of measurement available for a data set. Typically, ground truth measurements are provided for the evaluation of sensor data where the sensor or other data source is not as accurate or reliable as the source for the ground truth.

  • Ground Truth Poses: A collection of poses used to evaluate the accuracy of pose generation and prediction algorithms.

  • Ground Truth Images: A collection of images used to evaluate the accuracy of visual processing algorithms, like SLAM and VIO.

See the ILLIXR Plugins page for information about sensors implemented in ILLIXR.

Inertial Measurement Unit

A device that reports its orientation in space and any forces applied it. Also known as an IMU.

An IMU is implemented in the offline_imu_cam ILLIXR plugin.

For more information, see the Wikipedia article.

Simultaneous Localization and Mapping

The computational process of creating a map of an unknown environment, and finding one's location within that space. Also known as SLAM.

For more information, see the Wikipedia article.

Visual Interial Odometry

The process of computing a pose estimate from incoming visual information and measurements from the IMU. Also known as VIO. Often used in combination with SLAM techniques.

See the Wikipedia article.

Asynchronous Reprojection

The processing of rendered video for motion interpolation. Asynchronous reprojection improves the perception of the rendered video to the HMD when rendering misses it target frame rate.

Asynchronous reprojection is implemented in the timewarpgl ILLIXR plugin.

See the Wikipedia article.

Distortion Correction

The processing of visual anomalies in images where rectilinear features have been warped.

For more information, see the Wikipedia artice.

Chromatic Abberation Correction

The processing of visual anomalies in images where colors are diffracted due to imperfect optics or other perturbing factors.

For more information, see the Wikipedia article.

Components

Runner

An ILLIXR tool responsible for preparing the environment, downloading required assets & code, compiling each plugin, and launching the ILLIXR application. The implementation resides in ILLIXR/runner/, and can be launched with the appropriate environment setup via ILLIXR/runner.sh.

Spindle

An ILLIXR component responsible for launching and managing plugin threads. The implementation resides in ILLIXR/runtime/.

See the Spindle API documentation.

Phonebook

An ILLIXR service directory used to introspectively interface plugins and their data. The implementation resides in ILLIXR/runtime/.

See the Phonebook API documentation.

Switchboard

An ILLIXR event stream manager that maintains data pipelines between plugins. The implementation resides in ILLIXR/runtime/.

See the Switchboard API documentation.

Technologies

OpenXR

An open standard for Augmented and Virtual Reality. ILLIXR components target the OpenXR standard and interact with the ILLIXR device via the Application Interface.

For more information, visit the official site from the Khronos Group.

Monado

An open source, modular implementation of the OpenXR standard for GNU/Linux.

See the ILLIXR Monado Overview and Monado Dataflow pages for details about our runtime integration using Monado.

For more information, visit the official Monado development site.

Godot

An open source game development engine. ILLIXR applications targeting the OpenXR use Godot to access the engine's integration with the OpenXR standard via Monado.

For more information, visit the official Godot site.

Xvfb

A virtual framebuffer for the X11 Window Sytem. ILLIXR uses Xvfb to enable running the graphical ILLIXR application without requiring the user to have a graphical environment configured at application launch.

For more information, see the Xfvb man page.

Docker

A platform and containerization framework for deploying applications under virtualization. ILLIXR uses Docker to deploy and test code in a continuous integration and deployment pipeline.

For more information, see the Docker overview and getting started page.

QEMU-KVM

An open source virtulization tool and machine emulator. See the instructions for running ILLIXR under Virtualization.

For more information, see the official QEMU page.

YAML

A markup language and data serilization standard designed to be user friendly. We make use of the PyYAML and pyyaml-include libraries to implement our Configuration implementation.

For more information, visit the official YAML page.

SQLite

A SQL database engine implementation in C designed to be lightweight and easy to use. The ILLIXR project allows user to records application statistics to a local database for efficient processing. See the Logging and Metrics page for usage details.

For more information, see the SQLite development site.

Vulkan

A cross-platform graphics API that allows developers to efficiently target low-level hardware features.

For more information, see the official Vulkan page from the Khronos Group.

OpenGL

A cross-platform graphics API that allows developers to create graphics applications easily and portably. Also known as GL.

  • GL Context: A data structure storing the state of an OpenGL application instance. Within a GL context resides framebuffer data. It is not thread safe to share contexts without appropriate synchronization.

  • GLFW: An open source implementation of OpenGL. Supports Windows, MacOS and, Linux (X11 and Wayland). See the GLFW development site.

For more information, see the official OpenGL page from the Khronos Group.

Ubuntu

An open source GNU/Linux operating system and distribution. ILLIXR currently supports the Long Term Support (LTS) versions of Ubuntu: 18.04 LTS (Bionic) and 20.04 LTS (Focal).

For more information, visit the official Ubuntu site.