Towards Monocular Shape from Refraction

Antonin Sulc, Imari Sato, Bastian Goldluecke, Tali Treibitz

Research output: Contribution to conferencePaperpeer-review


Refraction is a common physical phenomenon and has long been researched in computer vision. Objects imaged through a refractive object appear distorted in the image as a function of the shape of the interface between the media. This hinders many computer vision applications, but can be utilized for obtaining the geometry of the refractive interface. Previous approaches for refractive surface recovery largely relied on various priors or additional information like multiple images of the analyzed surface. In contrast, we claim that a simple energy function based on Snell's law enables the reconstruction of an arbitrary refractive surface geometry using just a single image and known background texture and geometry. In the case of a single point, Snell's law has two degrees of freedom, therefore to estimate a surface depth, we need additional information. We show that solving for an entire surface at once introduces implicit parameter-free spatial regularization and yields convincing results when an intelligent initial guess is provided. We demonstrate our approach through simulations and real-world experiments, where the reconstruction shows encouraging results in the single-frame monocular setting.

Original languageEnglish
StatePublished - 2021
Event32nd British Machine Vision Conference, BMVC 2021 - Virtual, Online
Duration: 22 Nov 202125 Nov 2021


Conference32nd British Machine Vision Conference, BMVC 2021
CityVirtual, Online

Bibliographical note

Publisher Copyright:
© 2021. The copyright of this document resides with its authors.

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Vision and Pattern Recognition


Dive into the research topics of 'Towards Monocular Shape from Refraction'. Together they form a unique fingerprint.

Cite this