The study utilizes point cloud data, obtained through multiple Azure Kinect RGB-D cameras, to create a detailed and realistic virtual representation of the operational environment.

Point Cloud Mapping with External Calibration

The study proposes using normal vectors derived from point clouds to define extrinsic parameters, eliminating the need for physical markers or additional reference information. This allows a simpler yet accurate mapping of the physical environment into the VR workspace.

Data Integration into VR

The collected point cloud data is mapped into the virtual environment using the Unity game engine, with BAD SLAM (Bundle Adjusted Direct RGB-D SLAM) refining the mapping accuracy. ICP (Iterative Closest Point) is applied to align multiple camera feeds within the virtual space.

Experimental Setup

Two Azure Kinect RGB-D cameras and a Meta Quest Pro VR headset. The point cloud data was processed and visualized using Unity, achieving approximately 16-17 FPS for computationally intensive operations.