Redefining Multimodality

Research output: Contribution to journalArticlepeer-review

Abstract

The term “multimodality” incorporates visible gestures as part of language, a goal first put forward by Adam Kendon, and this idea revolutionized the scope of linguistic inquiry. But here I show that the term “multimodality” itself is rife with ambiguity, sometimes referring to different physical channels of transmission (auditory vs. visual), and sometimes referring to the integration of linguistic structures with more imagistic, less conventionalized expressions (see David McNeill's work), regardless of the physical channel. In sign languages, both modes are conveyed in a single, visual channel, revealed here in the signing of actors in a sign language theatre. In spoken languages, contrary to expectations raised by defining “modality” in terms of the physical channel, we see that the channel of transmission is orthogonal to linguistic and gestural modes of expression: Some visual signals are part and parcel of linguistic structure, while some auditory (intonational) signals have characteristics of the gestural mode. In this empirical, qualitative study, I adopt the term “mode” to refer solely to specific characteristics of communicative expression, and not to the physical channel. “Multimodal” refers to the coexistence of linguistic and gestural modes, regardless of the physical channel of transmission—straightforwardly encompassing the two natural language systems, spoken and signed.

Original languageEnglish
Article number758993
JournalFrontiers in Communication
Volume6
DOIs
StatePublished - 12 May 2022

Bibliographical note

Publisher Copyright:
Copyright © 2022 Sandler.

Keywords

  • gesture
  • modality
  • mode
  • multimodality
  • sign language

ASJC Scopus subject areas

  • Communication
  • Social Sciences (miscellaneous)

Fingerprint

Dive into the research topics of 'Redefining Multimodality'. Together they form a unique fingerprint.

Cite this