Welcome to the new version of CaltechAUTHORS. Login is currently restricted to library staff. If you notice any issues, please email coda@library.caltech.edu
Published October 10, 2021 | Submitted
Book Section - Chapter Open

Self-Calibrating Neural Radiance Fields

Abstract

In this work, we propose a camera self-calibration algorithm for generic cameras with arbitrary non-linear distortions. We jointly learn the geometry of the scene and the accurate camera parameters without any calibration objects. Our camera model consists of a pinhole model, a fourth order radial distortion, and a generic noise model that can learn arbitrary non-linear camera distortions. While traditional self-calibration algorithms mostly rely on geometric constraints, we additionally incorporate photometric consistency. This requires learning the geometry of the scene, and we use Neural Radiance Fields (NeRF). We also propose a new geometric loss function, viz., projected ray distance loss, to incorporate geometric consistency for complex non-linear camera models. We validate our approach on standard real image datasets and demonstrate that our model can learn the camera intrinsics and extrinsics (pose) from scratch without COLMAP initialization. Also, we show that learning accurate camera models in a differentiable manner allows us to improve PSNR over baselines. Our module is an easy-to-use plugin that can be applied to NeRF variants to improve performance. The code and data are currently available at https://github.com/POSTECH-CVLab/SCNeRF.

Additional Information

© 2021 IEEE. This work was supported by the IITP grants (2019-0-01906: AI Grad. School Prog. - POSTECH and 2021-0-00537: visual common sense through selfsupervised learning for restoration of invisible parts in images) funded by Ministry of Science and ICT, Korea.

Attached Files

Submitted - 2108.13826.pdf

Files

2108.13826.pdf
Files (6.3 MB)
Name Size Download all
md5:38517d4063dd7c77fea579e0e701f8e7
6.3 MB Preview Download

Additional details

Created:
August 20, 2023
Modified:
October 24, 2023