Minding the Borderlands

Mark Koester (@markwkoester) on the art of travel and technology

Language as Imitative, Intersubjective Communication, Pt. 4: Communication as Multimodal, Sensorial Production and Perception

If animals and humans are able to communicate together, then, as I have argued elsewhere, language and communication have some particularly important characteristics that we must not ignore, namely that communication is an intersubjective experience between at least two partners through some sort of communicative medium. Communication requires at least a sender and a receiver.

Most of the time we imagine communication through a written or a spoken form (which in at least human articulated language carries the all-important characteristic of “discrete infinity” through recursion), but clearly communication in living things takes place through several modes or mediums like visual, olfactory, auditory and felt (contact or touch) as well as through magnetic fields or electrical discharges. To a large extent, the type of medium employed places certain limitations and potentials on the communication that is feasible.

Equally important is to distinguish between communicative production and communicative perception. The sender has an organ or structure which is a signal (like a peacock’s tail) or allows the production of a signal (like vocal cords, scent glands, etc.) The receiver has receptors, which are usually some form of cognitive neural structure for sensory perception (rods and cones in the eyes, hair cells in ears, and the olfactory epithelium). These sensory cells are able to “transducer” the signal into nerve impulses which are processed by the nervous system and eventually transferred to a central neural structure (like the brain). It is important for these sensory perceptive systems to be able to decode incoming sensations in order to distinguish between information-carrying or meaningful sensations and extraneous information or noise.

While animals cannot necessarily produce language due physical limitations to their vocalization (specifically their lack of a larynx), this does not mean that they aren’t able to perceive (and perhaps even “understand” to a certain) what you are saying though vocalized words, gestures, facial expressions and emotionally-telling smells.

Leaving aside the question of the evolution of communication, which we will have to look at elsewhere, it is important to see that communication is tied into an entire system of communication through mediums and sensory systems. Even though language is most of the time perceived and experience as instantaneous meaning (for example, when someone says “apple,” we understand instantly what he or she is referring to; there is no need to think or process for the meaning to be “heard”), communication is not just meaningful signals passing to meaningful receptors. In fact, communication and interpretation passes through a whole communicative nexus of sensation, emotion and meaning.

There is an interesting psychological experiment called the McGurk Effect, which clearly shows how speech perception is not simply hearing the words but an interactive cognitive phenomenon between hearing and vision. Check out this short video displaying and subsequently explaining the McGurk Effect.

In the classic example, when we close our eyes we experience a heard /ba/. But when we see and hear at the same time, we experience a heard /da/, which in fact combines a visual /ga/ of the spoken lips and a heard /ba/.

In any case, this shows that speech perception and communication in general is multimodal, meaning that it involves information passing through several sensory modalities. In fact, I would go so far as to claim that an understanding of interspecies communication requires that we accept that much of human-to-human communication occurs through an entire neurobiological nexus of sensory modalities.

Communicative perception is not simply hearing words but being aware through memory of the context of past exchanges and through all of our sensations of the emotions being conveyed. Interspecies communication, while often unaware of the underlying layers to communication, displays a strong tendency to function through emotional and historical exchanges and not simply articulated ones.