Class ILLIXR::offload_rendering_server

ClassList > ILLIXR > offload_rendering_server

Main server implementation for offload rendering. More...

• #include <offload_rendering_server.hpp>

Inherits the following classes: ILLIXR::threadloop, ILLIXR::vulkan::timewarp, ILLIXR::data_format::pose_prediction

Public Attributes inherited from ILLIXR::vulkan::render_pass

See ILLIXR::vulkan::render_pass

Type	Name
VkPipeline	pipeline_ = [**VK\_NULL\_HANDLE**](namespaceILLIXR.md#function-normalize)

Public Functions

Type	Name
virtual void	_p_thread_setup () override Gets called at setup time, from the new thread.
virtual data_format::pose::head_pose_type	correct_pose (const data_format::pose::head_pose_type & pose) override const Correct pose data (no-op in this implementation)
virtual void	destroy () override Cleanup resources on destruction.
virtual bool	fast_pose_reliable () override const Check if fast pose data is reliable.
virtual POSE_TYPE	get_fast_pose () override const Get the latest pose for rendering.
virtual POSE_TYPE	get_fast_pose (POSE_TIME_TYPE future_time) override const Get predicted pose for a future time point (returns current pose)
virtual Eigen::Quaternionf	get_offset () override Get orientation offset (returns identity in this implementation)
virtual ILLIXR::data_format::pose::head_pose_type	get_true_pose () override const Get the true pose (same as fast pose in this implementation)
virtual bool	is_external () override Indicates this sink does not make use of the rendering pipeline in order for the access masks of the layout transitions to be set properly.
	offload_rendering_server (const std::string & name, phonebook * pb) Constructor initializes the server with configuration from environment variables.
virtual void	record_command_buffer (VkCommandBuffer commandBuffer, VkFramebuffer framebuffer, int buffer_ind, bool left) override Record command buffer (no-op in this implementation)
virtual void	set_offset (const Eigen::Quaternionf & orientation) override Set orientation offset (no-op in this implementation)
void	setup (VkRenderPass render_pass, uint32_t subpass, std::shared_ptr< vulkan::buffer_pool< BUFFER_TYPE > > buffer_pool, bool input_texture_vulkan_coordinates, struct illixr_framebuffer * framebuffer_array, VkExtent2D extent) override Sets up the rendering pipeline and encoding resources.
virtual void	start () override A method which Spindle calls when it starts the component.
virtual void	stop () override A method which Spindle calls when it stops the component.
virtual bool	true_pose_reliable () override const Check if true pose data is reliable (always false in this implementation)

Public Functions inherited from ILLIXR::threadloop

See ILLIXR::threadloop

Type	Name
virtual void	internal_stop () Stops the thread.
virtual void	start () override Starts the thread.
virtual void	stop () override Joins the thread.
	threadloop (const std::string & name, phonebook * pb)
	~threadloop () override

Public Functions inherited from ILLIXR::plugin

See ILLIXR::plugin

Type	Name
std::string	get_name () noexcept const
	plugin (std::string name, phonebook * pb)
void	spd_add_file_sink (const std::string & file_name, const std::string & extension, const std::string & log_level)
auto	spdlogger (const char * log_level)
virtual void	start () A method which Spindle calls when it starts the component.
virtual void	stop () A method which Spindle calls when it stops the component.
virtual	~plugin () = default

Public Functions inherited from ILLIXR::vulkan::timewarp

See ILLIXR::vulkan::timewarp

Type	Name
virtual void	setup (VkRenderPass render_pass, uint32_t subpass, std::shared_ptr< vulkan::buffer_pool< BUFFER_TYPE > > buffer_pool, bool input_texture_vulkan_coordinates, struct illixr_framebuffer * framebuffer_array=nullptr, VkExtent2D extent={}) = 0 Setup the timewarp render pass and initailize required Vulkan resources.

Public Functions inherited from ILLIXR::vulkan::render_pass

See ILLIXR::vulkan::render_pass

Type	Name
virtual void	destroy () Destroy the render pass and free all Vulkan resources.
virtual bool	is_external () = 0
virtual void	record_command_buffer (VkCommandBuffer commandBuffer, VkFramebuffer framebuffer, int buffer_ind, bool left) = 0 Record a command buffer for a given eye.
virtual void	update_uniforms (const BUFFER_TYPE & render_pose) Update the uniforms for the render pass.
	~render_pass () override

Public Functions inherited from ILLIXR::phonebook::service

See ILLIXR::phonebook::service

Type	Name
virtual	~service () = default

Public Functions inherited from ILLIXR::data_format::pose_prediction

See ILLIXR::data_format::pose_prediction

Type	Name
virtual pose::head_pose_type	correct_pose (const pose::head_pose_type & pose) const = 0
virtual bool	fast_pose_reliable () const = 0
virtual POSE_TYPE	get_fast_pose () const = 0
virtual POSE_TYPE	get_fast_pose (POSE_TIME_TYPE future_time) const = 0
virtual Eigen::Quaternionf	get_offset () = 0
virtual pose::head_pose_type	get_true_pose () const = 0
virtual void	set_offset (const Eigen::Quaternionf & orientation) = 0
virtual bool	true_pose_reliable () const = 0
	~pose_prediction () override

Public Functions inherited from ILLIXR::phonebook::service

See ILLIXR::phonebook::service

Type	Name
virtual	~service () = default

Protected Types inherited from ILLIXR::threadloop

See ILLIXR::threadloop

Type	Name
enum	skip_option

Protected Attributes inherited from ILLIXR::threadloop

See ILLIXR::threadloop

Type	Name
std::size_t	iteration_no = 0
std::size_t	skip_no = 0

Protected Attributes inherited from ILLIXR::plugin

See ILLIXR::plugin

Type	Name
const std::size_t	id_
std::string	name_
const phonebook *	phonebook_
std::shared_ptr< spdlog::logger >	plugin_logger_
const std::shared_ptr< record_logger >	record_logger_

Protected Functions

Type	Name
virtual void	_p_one_iteration () override Main processing loop for frame encoding and transmission.
virtual threadloop::skip_option	_p_should_skip () override Determines if the current iteration should be skipped.

Protected Functions inherited from ILLIXR::threadloop

See ILLIXR::threadloop

Type	Name
virtual void	_p_one_iteration () = 0 Override with the computation the thread does every loop.
virtual skip_option	_p_should_skip () Gets called in a tight loop, to gate the invocation of _p_one_iteration() __
virtual void	_p_thread_setup () Gets called at setup time, from the new thread.
bool	should_terminate () Whether the thread has been asked to terminate.

Protected Functions inherited from ILLIXR::data_format::pose_prediction

See ILLIXR::data_format::pose_prediction

Type	Name
pose::head_pose_type	_correct_pose (const pose::head_pose_type & pose) const
void	_set_offset (const Eigen::Quaternionf & orientation)

Detailed Description

This class handles: * Frame capture from the rendering pipeline * Hardware-accelerated encoding using FFmpeg/CUDA or NVENC directly * Network transmission of encoded frames * Pose synchronization with the client * Hand tracking data forwarding to Monado

The server receives combined pose_with_hands data from the client, extracts the head pose for rendering, and publishes hand tracking data to the switchboard for Monado's ILLIXR driver to read.

Compile with -DNVENC_ENCODER to use direct NVENC encoding without FFmpeg.

Public Functions Documentation

function _p_thread_setup

Gets called at setup time, from the new thread.

virtual void ILLIXR::offload_rendering_server::_p_thread_setup () override

Implements ILLIXR::threadloop::_p_thread_setup

---

function correct_pose

Correct pose data (no-op in this implementation)

inline virtual data_format::pose::head_pose_type ILLIXR::offload_rendering_server::correct_pose (
    const  data_format::pose::head_pose_type & pose
) override const

Implements ILLIXR::data_format::pose_prediction::correct_pose

---

function destroy

Cleanup resources on destruction.

virtual void ILLIXR::offload_rendering_server::destroy () override

Implements ILLIXR::vulkan::render_pass::destroy

---

function fast_pose_reliable

Check if fast pose data is reliable.

inline virtual bool ILLIXR::offload_rendering_server::fast_pose_reliable () override const

Implements ILLIXR::data_format::pose_prediction::fast_pose_reliable

---

function get_fast_pose [1/2]

Get the latest pose for rendering.

inline virtual POSE_TYPE ILLIXR::offload_rendering_server::get_fast_pose () override const

Extracts the head pose from the most recent pose_with_hands data received from the client. Also forwards hand tracking data to the switchboard for Monado (only once per new data arrival).

Implements ILLIXR::data_format::pose_prediction::get_fast_pose

---

function get_fast_pose [2/2]

Get predicted pose for a future time point (returns current pose)

inline virtual POSE_TYPE ILLIXR::offload_rendering_server::get_fast_pose (
    POSE_TIME_TYPE future_time
) override const

Implements ILLIXR::data_format::pose_prediction::get_fast_pose

---

function get_offset

Get orientation offset (returns identity in this implementation)

inline virtual Eigen::Quaternionf ILLIXR::offload_rendering_server::get_offset () override

Implements ILLIXR::data_format::pose_prediction::get_offset

---

function get_true_pose

Get the true pose (same as fast pose in this implementation)

inline virtual ILLIXR::data_format::pose::head_pose_type ILLIXR::offload_rendering_server::get_true_pose () override const

Implements ILLIXR::data_format::pose_prediction::get_true_pose

---

function is_external

Indicates this sink does not make use of the rendering pipeline in order for the access masks of the layout transitions to be set properly.

inline virtual bool ILLIXR::offload_rendering_server::is_external () override

Implements ILLIXR::vulkan::render_pass::is_external

---

function offload_rendering_server

Constructor initializes the server with configuration from environment variables.

ILLIXR::offload_rendering_server::offload_rendering_server (
    const std::string & name,
    phonebook * pb
) 

Parameters:

• name Plugin name
• pb Phonebook for component lookup

---

function record_command_buffer

Record command buffer (no-op in this implementation)

inline virtual void ILLIXR::offload_rendering_server::record_command_buffer (
    VkCommandBuffer commandBuffer,
    VkFramebuffer framebuffer,
    int buffer_ind,
    bool left
) override

Implements ILLIXR::vulkan::render_pass::record_command_buffer

---

function set_offset

Set orientation offset (no-op in this implementation)

inline virtual void ILLIXR::offload_rendering_server::set_offset (
    const Eigen::Quaternionf & orientation
) override

Implements ILLIXR::data_format::pose_prediction::set_offset

---

function setup

Sets up the rendering pipeline and encoding resources.

void ILLIXR::offload_rendering_server::setup (
    VkRenderPass render_pass,
    uint32_t subpass,
    std::shared_ptr< vulkan::buffer_pool < BUFFER_TYPE > > buffer_pool,
    bool input_texture_vulkan_coordinates,
    struct  illixr_framebuffer * framebuffer_array,
    VkExtent2D extent
) override

Parameters:

• render_pass Vulkan render pass handle
• subpass Subpass index
• _buffer_pool Buffer pool for frame management
• input_texture_vulkan_coordinates Whether input textures use Vulkan coordinates

---

function start

A method which Spindle calls when it starts the component.

virtual void ILLIXR::offload_rendering_server::start () override

This is necessary because a constructor can't call derived virtual methods (due to structure of C++). See threadloop for an example of this use-case.

Implements ILLIXR::plugin::start

---

function stop

A method which Spindle calls when it stops the component.

virtual void ILLIXR::offload_rendering_server::stop () override

This is necessary because the parent class might define some actions that need to be taken prior to destructing the derived class. For example, threadloop must halt and join the thread before the derived class can be safely destructed. However, the derived class's destructor is called before its parent (threadloop), so threadloop doesn't get a chance to join the thread before the derived class is destroyed, and the thread accesses freed memory. Instead, we call plugin->stop manually before destroying anything.

Concrete plugins are responsible for initializing their specific logger and sinks.

Implements ILLIXR::plugin::stop

---

function true_pose_reliable

Check if true pose data is reliable (always false in this implementation)

inline virtual bool ILLIXR::offload_rendering_server::true_pose_reliable () override const

Implements ILLIXR::data_format::pose_prediction::true_pose_reliable

---

Protected Functions Documentation

function _p_one_iteration

Main processing loop for frame encoding and transmission.

virtual void ILLIXR::offload_rendering_server::_p_one_iteration () override

This method: * Captures the latest rendered frame * Encodes it using hardware acceleration * Transmits it to the client * Tracks performance metrics

Implements ILLIXR::threadloop::_p_one_iteration

---

function _p_should_skip

Determines if the current iteration should be skipped.

inline virtual threadloop::skip_option ILLIXR::offload_rendering_server::_p_should_skip () override

Implements ILLIXR::threadloop::_p_should_skip

---

---

The documentation for this class was generated from the following file /home/friedel/devel/ILLIXR/illixr_release/ILLIXR/plugins/offload_rendering_server/offload_rendering_server.hpp