Abstract
Bodily expressed emotion understanding (BEEU) aims to automatically recognize human emotional expressions from body movements. Psychological research has demonstrated that people often move using specific motor elements to convey emotions. This work takes three steps to integrate human motor elements to study BEEU. First, we introduce BoME (body motor elements), a highly precise dataset for human motor elements. Second, we apply baseline models to estimate these elements on BoME, showing that deep learning methods are capable of learning effective representations of human movement. Finally, we propose a dual-source solution to enhance the BEEU model with the BoME dataset, which trains with both motor element and emotion labels and simultaneously produces predictions for both. Through experiments on the BoLD in-the-wild emotion understanding benchmark, we showcase the significant benefit of our approach. These results may inspire further research utilizing human motor elements for emotion understanding and mental health analysis.
Original language | English |
---|---|
Article number | 100816 |
Journal | Patterns |
Volume | 4 |
Issue number | 10 |
DOIs | |
State | Published - 13 Oct 2023 |
Bibliographical note
Publisher Copyright:© 2023 The Authors
Keywords
- DSML 2: Proof-of-concept: Data science output has been formulated, implemented, and tested for one domain/problem
- affective computing
- computer vision
- dance
- deep learning
- emotion recognition
- performing arts
- psychology
- robotics
- video understanding
ASJC Scopus subject areas
- General Decision Sciences