Handling Digital Brains: A Laboratory Study of Multimodal Semiotic Interaction in the Age of Computers

An analysis of how fMRI researchers actively involve their bodies--with hand movements in particular--in laboratory practice.

The results of fMRI brain scanning require extensive analysis in the laboratory. In Handling Digital Brains, Morana Alac shows that fMRI researchers do not sit passively staring at computer screens but actively involve their bodies in laboratory practice. Discussing fMRI visuals with colleagues, scientists animate the scans with gestures, and talk as they work with computers. Alac argues that to understand how digital scientific visuals take on meaning we must consider their dynamic coordination with gesture, speech, and working hands. These multimodal actions, she suggests, are an essential component of digital scientific visuals.

A semiotician trained in cognitive science, Alac grounds her discussion in concepts from Peirce's semiotics and her methodology in ethnography and multimodal conversation analysis. Basing her observations on videotaped records of activity in three fMRI research labs, Alac describes scientists' manual engagement with digital visuals of the human brain. Doing so, she turns her attention to the issue of practical thinking. Alac argues that although fMRI technology directs scientists to consider human thinking in terms of an individual brain, scientific practices in the fMRI lab demonstrate thinking that engages the whole lived body and the world in which the body is situated. The turn toward the digital does not bring with it abstraction but a manual and embodied engagement. The practical and multimodal engagement with digital brains in the laboratory challenges certain assumptions behind fMRI technology; it suggests our hands are essential to learning, and the making of meaning.

Morana Alač is Assistant Professor in the Department of Communication and Program in Science Studies at the University of California, San Diego.