Automated video analysis of non-verbal communication in a medical setting

Yuval Hart, Efrat Czerniak, Orit Karnieli-Miller, Avraham E. Mayo, Amitai Ziv, Anat Biegon, Atay Citron, Uri Alon

Research output: Contribution to journalArticlepeer-review


Non-verbal communication plays a significant role in establishing good rapport between physicians and patients and may influence aspects of patient health outcomes. It is therefore important to analyze non-verbal communication in medical settings. Current approaches to measure non-verbal interactions in medicine employ coding by human raters. Such tools are labor intensive and hence limit the scale of possible studies. Here, we present an automated video analysis tool for non-verbal interactions in a medical setting. We test the tool using videos of subjects that interact with an actor portraying a doctor. The actor interviews the subjects performing one of two scripted scenarios of interviewing the subjects: in one scenario the actor showed minimal engagement with the subject. The second scenario included active listening by the doctor and attentiveness to the subject. We analyze the cross correlation in total kinetic energy of the two people in the dyad, and also characterize the frequency spectrum of their motion. We find large differences in interpersonal motion synchrony and entrainment between the two performance scenarios. The active listening scenario shows more synchrony and more symmetric followership than the other scenario. Moreover, the active listening scenario shows more high-frequency motion termed jitter that has been recently suggested to be a marker of followership. The present approach may be useful for analyzing physician-patient interactions in terms of synchrony and dominance in a range of medical settings.

Original languageEnglish
Article number01130
JournalFrontiers in Psychology
Issue numberAUG
StatePublished - 23 Aug 2016

Bibliographical note

Funding Information:
We thank the Braginsky Center for the Interface between Science and the Humanities, at the Weizmann Institute of Science, for support. UA is the incumbent of the Abisch-Frenkel Professorial Chair.

Publisher Copyright:
© 2016 Hart, Czerniak, Karnieli-Miller, Mayo, Ziv, Biegon, Citron and Alon.


  • Doctor-patient interactions
  • Entrainment
  • Non-verbal communication
  • Performance
  • Synchronization
  • Video analysis

ASJC Scopus subject areas

  • Psychology (all)


Dive into the research topics of 'Automated video analysis of non-verbal communication in a medical setting'. Together they form a unique fingerprint.

Cite this