NAN: Noise-Aware NeRFs for Burst-Denoising

Naama Pearl, Tali Treibitz, Simon Korman

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Burst denoising is now more relevant than ever, as computational photography helps overcome sensitivity issues inherent in mobile phones and small cameras. A major challenge in burst-denoising is in coping with pixel misalignment, which was so far handled with rather simplistic assumptions of simple motion, or the ability to align in pre-processing. Such assumptions are not realistic in the presence of large motion and high levels of noise. We show that Neural Radiance Fields (NeRFs), originally suggested for physics-based novel-view rendering, can serve as a powerful framework for burst denoising. NeRFs have an inherent capability of handling noise as they integrate information from multiple images, but they are limited in doing so, mainly since they build on pixel-wise operations which are suitable to ideal imaging conditions. Our approach, termed NAN111Refer to the project website: noise-aware-nerf.github.io, leverages inter-view and spatial information in NeRFs to better deal with noise. It achieves state-of-the-art results in burst denoising and is especially successful in coping with large movement and occlusions, under very high levels of noise. With the rapid advances in accelerating NeRFs, it could provide a powerful platform for denoising in challenging environments.

Original languageEnglish
Title of host publicationProceedings - 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022
PublisherIEEE Computer Society
Pages12662-12671
Number of pages10
ISBN (Electronic)9781665469463
DOIs
StatePublished - 2022
Event2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022 - New Orleans, United States
Duration: 19 Jun 202224 Jun 2022

Publication series

NameProceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
Volume2022-June
ISSN (Print)1063-6919

Conference

Conference2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022
Country/TerritoryUnited States
CityNew Orleans
Period19/06/2224/06/22

Bibliographical note

Funding Information:
Acknowledgements. The research was funded by Israel Science Foundation grant #680/18 and the Israeli Ministry of Science and Technology grant #3 − 15621, the Leona M. and Harry B. Helm-sley Charitable Trust, the Maurice Hatter Foundation. We thank the Interuniversity Institute for Marine Sciences of Eilat for making their facilities available to us; and Deborah Steinberger-Levy, Opher Bar Nathan, and Amit Peleg for help with experiments.

Publisher Copyright:
© 2022 IEEE.

Keywords

  • Low-level vision
  • Physics-based vision and shape-from-X

ASJC Scopus subject areas

  • Software
  • Computer Vision and Pattern Recognition

Fingerprint

Dive into the research topics of 'NAN: Noise-Aware NeRFs for Burst-Denoising'. Together they form a unique fingerprint.

Cite this