GROOT: A Real-time Streaming System for High-Fidelity Volumetric Videos (MobiCom’20)
Kyungjin Lee, Juheon Yi, Youngki Lee, Sunghyun Choi, Young Min Kim
GROOT is a mobile 3D volumetric video streaming system for immersive augmented reality (AR). Volumetric video streaming is challenging as (i) 3D volume data size, in particular, point clouds, is larger than conventional 2D or 360° videos, and (ii) 3D data representation is sparse and unorganized opposed to conventional 2D videos represented in fixed-size grids. Therefore, there is a tight trade-off between the compression rate and the decoding complexity. We develop a novel Parallel Decodable Tree (PDTree) data structure to compress and decode volumetric videos on mobile GPUs to achieve real-time frame rate with minimal resource usage. PDTree allows integrating decoding and rendering in a single pipeline to minimize the decoding overhead and also enables fast modification of the data to apply user-adaptive optimizations. Groot jointly optimizes streaming and rendering which is the main components in volumetric video streaming in AR. Therefore, it can enable real-time decoding while maintaining low motion-to-photon latency on resource-constrained mobile devices.