HybridMouse: A Hybrid Convolutional-Recurrent Neural Network-Based Model for Identification of Mouse Ultrasonic Vocalizations

Yizhaq Goussha, Kfir Bar, Shai Netser, Lior Cohen, Yacov Hel-Or, Shlomo Wagner

Research output: Contribution to journalArticlepeer-review

Abstract

Mice use ultrasonic vocalizations (USVs) to convey a variety of socially relevant information. These vocalizations are affected by the sex, age, strain, and emotional state of the emitter and can thus be used to characterize it. Current tools used to detect and analyze murine USVs rely on user input and image processing algorithms to identify USVs, therefore requiring ideal recording environments. More recent tools which utilize convolutional neural networks models to identify vocalization segments perform well above the latter but do not exploit the sequential structure of audio vocalizations. On the other hand, human voice recognition models were made explicitly for audio processing; they incorporate the advantages of CNN models in recurrent models that allow them to capture the sequential nature of the audio. Here we describe the HybridMouse software: an audio analysis tool that combines convolutional (CNN) and recurrent (RNN) neural networks for automatically identifying, labeling, and extracting recorded USVs. Following training on manually labeled audio files recorded in various experimental conditions, HybridMouse outperformed the most commonly used benchmark model utilizing deep-learning tools in accuracy and precision. Moreover, it does not require user input and produces reliable detection and analysis of USVs recorded under harsh experimental conditions. We suggest that HybrideMouse will enhance the analysis of murine USVs and facilitate their use in scientific research.

Original languageEnglish
Article number810590
JournalFrontiers in Behavioral Neuroscience
Volume15
DOIs
StatePublished - 25 Jan 2022

Bibliographical note

Funding Information:
This study was supported by ISF-NSFC joint research program (Grant No. 3459/20 to SW), the Israel Science Foundation (ISF Grants No. 1350/12, 1361/17 to SW, and 258/17 to LC), the Ministry of Science, Technology and Space of Israel (Grant No. 3-12068 to SW), and the United States-Israel Binational Science Foundation (BSF Grant No. 2019186 to SW).

Publisher Copyright:
Copyright © 2022 Goussha, Bar, Netser, Cohen, Hel-Or and Wagner.

Keywords

  • CNN–convolutional neural networks
  • LSTM–long short-term memory
  • animal communication
  • machine learning
  • neural networks
  • social interactions
  • ultrasonic vocalizations

ASJC Scopus subject areas

  • Neuropsychology and Physiological Psychology
  • Cognitive Neuroscience
  • Behavioral Neuroscience

Fingerprint

Dive into the research topics of 'HybridMouse: A Hybrid Convolutional-Recurrent Neural Network-Based Model for Identification of Mouse Ultrasonic Vocalizations'. Together they form a unique fingerprint.

Cite this