Towards automated annotation of benthic survey images: Variability of human experts and operational modes of automation

Oscar Beijbom, Peter J. Edmunds, Chris Roelfsema, Jennifer Smith, David I. Kline, Benjamin P. Neal, Matthew J. Dunlap, Vincent Moriarty, Tung Yung Fan, Chih Jui Tan, Stephen Chan, Tali Treibitz, Anthony Gamst, B. Greg Mitchell, David Kriegman

Research output: Contribution to journalArticlepeer-review


Global climate change and other anthropogenic stressors have heightened the need to rapidly characterize ecological changes in marine benthic communities across large scales. Digital photography enables rapid collection of survey images to meet this need, but the subsequent image annotation is typically a time consuming, manual task. We investigated the feasibility of using automated point-annotation to expedite cover estimation of the 17 dominant benthic categories from survey-images captured at four Pacific coral reefs. Interand intra-annotator variability among six human experts was quantified and compared to semi- and fully-automated annotation methods, which are made available Our results indicate high expert agreement for identification of coral genera, but lower agreement for algal functional groups, in particular between turf algae and crustose coralline algae. This indicates the need for unequivocal definitions of algal groups, careful training of multiple annotators, and enhanced imaging technology. Semi-automated annotation, where 50% of the annotation decisions were performed automatically, yielded cover estimate errors comparable to those of the human experts. Furthermore, fully-automated annotation yielded rapid, unbiased cover estimates but with increased variance. These results show that automated annotation can increase spatial coverage and decrease time and financial outlay for image-based reef surveys.

Original languageEnglish
Article numbere0130312
JournalPLoS ONE
Issue number7
StatePublished - 8 Jul 2015

ASJC Scopus subject areas

  • General


Dive into the research topics of 'Towards automated annotation of benthic survey images: Variability of human experts and operational modes of automation'. Together they form a unique fingerprint.

Cite this