Automated analysis of marine video with limited data

Deborah Levy, Yuval Belfer, Elad Osherov, Eyal Bigal, Aviad P. Scheinin, Hagai Nativ, Dan Tchernov, Tali Treibitz

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review


Monitoring of the marine environment requires large amounts of data, simply due to its vast size. Therefore, underwater autonomous vehicles and drones are increasingly deployed to acquire numerous photographs. However, ecological conclusions from them are lagging as the data requires expert annotation and thus realistically cannot be manually processed. This calls for developing automatic classification algorithms dedicated for this type of data. Current out-of-the-box solutions struggle to provide optimal results in these scenarios as the marine data is very different from everyday data. Images taken under water display low contrast levels and reduced visibility range thus making objects harder to localize and classify. Scale varies dramatically because of the complex 3 dimensionality of the scenes. In addition, the scarcity of labeled marine data prevents training these dedicated networks from scratch. In this work, we demonstrate how transfer learning can be utilized to achieve high quality results for both detection and classification in the marine environment. We also demonstrate tracking in videos that enables counting and measuring the organisms. We demonstrate the suggested method on two very different marine datasets, an aerial dataset and an underwater one.

Original languageEnglish
Title of host publicationProceedings - 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2018
PublisherIEEE Computer Society
Number of pages9
ISBN (Electronic)9781538661000
StatePublished - 13 Dec 2018
Externally publishedYes
Event31st Meeting of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2018 - Salt Lake City, United States
Duration: 18 Jun 201822 Jun 2018

Publication series

NameIEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops
ISSN (Print)2160-7508
ISSN (Electronic)2160-7516


Conference31st Meeting of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2018
Country/TerritoryUnited States
CitySalt Lake City

Bibliographical note

Funding Information:
This work was supported by the European Union’s Horizon 2020 research and innovation program under grant agreement No. 773753 (SYMBIOSIS), the Leona M. and Harry B. Helmsley Charitable Trust, the Maurice Hatter Foundation, and the Technion Ollendorff Minerva Center for Vision and Image Sciences. Research at the University of Haifa was conducted at the Hatter Department for Marine Technologies and the Morris Kahn Marine Research Station, Department of Marine Biology.

Publisher Copyright:
© 2018 IEEE.

ASJC Scopus subject areas

  • Computer Vision and Pattern Recognition
  • Electrical and Electronic Engineering


Dive into the research topics of 'Automated analysis of marine video with limited data'. Together they form a unique fingerprint.

Cite this