OmniGS: Fast Radiance Field Reconstruction using Omnidirectional Gaussian Splatting

Photorealistic reconstruction relying on 3D Gaussian Splatting has shown promising potential in various domains. However the current 3D Gaussian Splatting system only supports radiance field reconstruction using undistorted perspective images. In this paper we present OmniGS a novel omnidirectional Gaussian splatting system to take advantage of omnidirectional images for fast radiance field reconstruction. Specifically we conduct a theoretical analysis of spherical camera model derivatives in 3D Gaussian Splatting. According to the derivatives we then implement a new GPU-accelerated omnidirectional rasterizer that directly splats 3D Gaussians onto the equirectangular screen space for omnidirectional image rendering. We realize differentiable optimization of the omnidirectional radiance field without the requirement of cube-map rectification or tangent-plane approximation. Extensive experiments conducted in egocentric and roaming scenarios demonstrate that our method achieves state-of-the-art reconstruction quality and high rendering speed using omnidirectional images. The code will be publicly available at https://github.com/liquorleaf/OmniGS.