Perception of multi-varied sound patterns of sonified representations of complex systems by people who are blind

Listening to complexity is a long-term research project, which addresses a central need among people who are blind: providing equal access to the science classroom, by allowing them to explore computer models, independently collect data, adapt and control their learning process. The innovative and low-cost learning system that is used in this project is based on the principle of perceptual compensation via technologies, by harnessing the auditory mode to transmit dynamic and spatial complex information, due to its unique affordances with respect to vision. Sonification of variables and events in an agent-based NetLogo computer model is used to convey information regarding both individual gas particles and system-wide phenomena, using alerts, object and status indicators, data representation and spatial audio displays. The paper describes two experiments: (i) auditory perception of varying types of auditory representations, spatial trajectories of a modeled object’s motion, relative intensity, and frequency; and (ii) auditory perception of complex sound patterns, exploring detection and recognition of multiple sound channels at different complexity levels of sound patterns. The research would serve to improve our understanding of the auditory processes by which perception of sound patterns takes place and transforms into a conceptual model. The long-term practical benefits of this research are likely to have an impact on science, technology, engineering and mathematics (STEM) education for students who are blind.