Publication

NeRFLight: Fast and light neural radiance fields using a shared feature grid

Conference Article

Conference

IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

Edition

2023

Pages

12417-12427

Doc link

http://dx.doi.org/10.1109/CVPR52729.2023.01195

File

Download the digital copy of the doc pdf document

Authors

Projects associated

Abstract

While original Neural Radiance Fields (NeRF) have shown impressive results in modeling the appearance of a scene with compact MLP architectures, they are not able to achieve real-time rendering. This has been recently addressed by either baking the outputs of NeRF into a data structure or arranging trainable parameters in an explicit feature grid. These strategies, however, significantly increase the memory footprint of the model which prevents their deployment on bandwidth-constrained applications. In this paper, we extend the grid-based approach to achieve real-time view synthesis at more than 150 FPS using a lightweight model. Our main contribution is a novel architecture in which the density field of NeRF-based representations is split into N regions and the density is modeled using N different decoders which reuse the same feature grid. This results in a smaller grid where each feature is located in more than one spatial position, forcing them to learn a compact representation that is valid for different parts of the scene. We further reduce the size of the final model by disposing of the features symmetrically on each region, which favors feature pruning after training while also allowing smooth gradient transitions between neighboring voxels. An exhaustive evaluation demonstrates that our method achieves real-time performance and quality metrics on a pair with state-of-the-art with an improvement of more than 2× in the FPS/MB ratio.

Categories

computer vision.

Author keywords

Training, Measurement, Social networking (online), Metaverse, Games, Rendering (computer graphics), Data structures

Scientific reference

F. Rivas-Manzaneque, J. Sierra-Acosta, A. Penate-Sanchez, F. Moreno-Noguer and A. Ribeiro. NeRFLight: Fast and light neural radiance fields using a shared feature grid, 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, Vancouver, Canada, pp. 12417-12427.