Research on neural radiance fields (NeRFs) for novel view generation is exploding with new models and extensions. However, a question that remains unanswered is what happens in underwater or foggy scenes where the medium strongly influences the appearance of objects. Thus far, NeRF and its variants have ignored these cases. However, since the NeRF framework is based on volumetric rendering, it has inherent capability to account for the medium's effects, once modeled appropriately. We develop a new rendering model for NeRFs in scattering media, which is based on the SeaThru image formation model, and suggest a suitable architecture for learning both scene information and medium parameters. We demonstrate the strength of our method using simulated and real-world scenes, correctly rendering novel photorealistic views underwater. Even more excitingly, we can render clear views of these scenes, removing the medium between the camera and the scene and reconstructing the appearance and depth of far objects, which are severely occluded by the medium. Our code and unique datasets are available on the project's website.
|Title of host publication||Proceedings - 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023|
|Publisher||IEEE Computer Society|
|Number of pages||10|
|State||Published - 2023|
|Event||2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023 - Vancouver, Canada|
Duration: 18 Jun 2023 → 22 Jun 2023
|Name||Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition|
|Conference||2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023|
|Period||18/06/23 → 22/06/23|
Bibliographical noteFunding Information:
Acknowledgements. The research was funded by Israel Science Foundation grant #680/18, Israeli Ministry of Science and Technology, European Union’s Horizon 2020 research and innovation programme GA 101094924 (ANERIS), the Leona M. and Harry B. Helmsley Charitable Trust, the Maurice Hatter Foundation, and Schmidt Marine Technology Partners. We thank Matan Yuval for substantial data contribution, Opher Bar-Nathan and Yuval Gold-fracht for help with experiments.
© 2023 IEEE.
- Image and video synthesis and generation
ASJC Scopus subject areas
- Computer Vision and Pattern Recognition