Struct ILLIXR::data_format::semantic_frame
ClassList > ILLIXR > data_format > semantic_frame
A single co-captured, HEVC-encoded RGB frame and raw depth frame, each with a matched head pose at their individual sensor exposure time. More...
#include <semantics.hpp>
Inherits the following classes: ILLIXR::switchboard::event
Public Attributes
| Type | Name |
|---|---|
| std::vector< uint8_t > | depth R16_SFloat, inverted NDC, top-down. |
| camera_intrinsics | depth_intrinsics |
| float | depth_near_z = {0.[**f**](namespaceILLIXR.md#function-normalize)} |
| float | depth_pose = {0.[**f**](namespaceILLIXR.md#function-normalize), 0.[**f**](namespaceILLIXR.md#function-normalize), 0.[**f**](namespaceILLIXR.md#function-normalize), 0.[**f**](namespaceILLIXR.md#function-normalize), 0.[**f**](namespaceILLIXR.md#function-normalize), 0.[**f**](namespaceILLIXR.md#function-normalize), 0.[**f**](namespaceILLIXR.md#function-normalize), 0.[**f**](namespaceILLIXR.md#function-normalize), 0.[**f**](namespaceILLIXR.md#function-normalize), 0.[**f**](namespaceILLIXR.md#function-normalize), 0.[**f**](namespaceILLIXR.md#function-normalize), 0.[**f**](namespaceILLIXR.md#function-normalize), 0.[**f**](namespaceILLIXR.md#function-normalize), 0.[**f**](namespaceILLIXR.md#function-normalize), 0.[**f**](namespaceILLIXR.md#function-normalize), 0.[**f**](namespaceILLIXR.md#function-normalize)} |
| int64_t | depth_timestamp_ns = 0XrTime ns (from XrDepthSensorDataMETA) |
| int32_t | frame_number = {0} |
| std::vector< uint8_t > | image H.265 annexb bytes. |
| camera_intrinsics | intrinsics RGB camera intrinsics (includes width/height) |
| float | max_depth = {0.[**f**](namespaceILLIXR.md#function-normalize)} |
| float | rgb_camera_pose = {0.[**f**](namespaceILLIXR.md#function-normalize), 0.[**f**](namespaceILLIXR.md#function-normalize), 0.[**f**](namespaceILLIXR.md#function-normalize), 0.[**f**](namespaceILLIXR.md#function-normalize), 0.[**f**](namespaceILLIXR.md#function-normalize), 0.[**f**](namespaceILLIXR.md#function-normalize), 0.[**f**](namespaceILLIXR.md#function-normalize), 0.[**f**](namespaceILLIXR.md#function-normalize), 0.[**f**](namespaceILLIXR.md#function-normalize), 0.[**f**](namespaceILLIXR.md#function-normalize), 0.[**f**](namespaceILLIXR.md#function-normalize), 0.[**f**](namespaceILLIXR.md#function-normalize), 0.[**f**](namespaceILLIXR.md#function-normalize), 0.[**f**](namespaceILLIXR.md#function-normalize), 0.[**f**](namespaceILLIXR.md#function-normalize), 0.[**f**](namespaceILLIXR.md#function-normalize)} |
| int64_t | rgb_timestamp_ns = 0CLOCK_BOOTTIME ns (from encoder PTS) |
Public Functions
| Type | Name |
|---|---|
| semantic_frame () = default |
Public Functions inherited from ILLIXR::switchboard::event
See ILLIXR::switchboard::event
| Type | Name |
|---|---|
| void | serialize (Archive & ar, const unsigned int version) |
| virtual | ~event () = default |
Detailed Description
Produced by xr_sensor_capture and consumed by the network transmission plugin. Replaces the earlier semantic_data struct.
RGB data
H.265 annexb, encoded on the GPU by Android MediaCodec in Surface-input mode. Camera2 writes directly into the encoder's ANativeWindow; the Snapdragon hardware handles NV12 → HEVC without any CPU copy. The timestamp is recovered from AMediaCodecBufferInfo.presentationTimeUs (CLOCK_BOOTTIME microseconds, promoted to nanoseconds).
Depth data
R16_SFloat (IEEE 754 float16), 2 bytes/pixel, top-down, unencoded. Values are inverted NDC: depth_m = depth_near_z / r_value. CPU-side as delivered by XR_META_depth_sensor.
Poses
Row-major 4×4 float matrices in OpenXR right-handed tracking space. Coordinate conversion to Unity left-handed world space is the server's responsibility, consistent with the existing LhToRh convention.
Public Attributes Documentation
variable depth
R16_SFloat, inverted NDC, top-down.
std::vector<uint8_t> ILLIXR::data_format::semantic_frame::depth;
variable depth_intrinsics
camera_intrinsics ILLIXR::data_format::semantic_frame::depth_intrinsics;
variable depth_near_z
float ILLIXR::data_format::semantic_frame::depth_near_z;
variable depth_pose
float ILLIXR::data_format::semantic_frame::depth_pose[16];
variable depth_timestamp_ns
XrTime ns (from XrDepthSensorDataMETA)
int64_t ILLIXR::data_format::semantic_frame::depth_timestamp_ns;
variable frame_number
int32_t ILLIXR::data_format::semantic_frame::frame_number;
variable image
H.265 annexb bytes.
std::vector<uint8_t> ILLIXR::data_format::semantic_frame::image;
variable intrinsics
RGB camera intrinsics (includes width/height)
camera_intrinsics ILLIXR::data_format::semantic_frame::intrinsics;
variable max_depth
float ILLIXR::data_format::semantic_frame::max_depth;
variable rgb_camera_pose
float ILLIXR::data_format::semantic_frame::rgb_camera_pose[16];
variable rgb_timestamp_ns
CLOCK_BOOTTIME ns (from encoder PTS)
int64_t ILLIXR::data_format::semantic_frame::rgb_timestamp_ns;
Public Functions Documentation
function semantic_frame
ILLIXR::data_format::semantic_frame::semantic_frame () = default
The documentation for this class was generated from the following file /home/friedel/devel/ILLIXR/illixr_release/ILLIXR/include/illixr/data_format/semantics.hpp