Joakinator is a wearable interactive interface that allows users to activate different media materials, such as sound, music, and video, through body gestures. The device was designed in the context of music and performing arts and integrates surface electromyogram, force sensors, and machine learning algorithms with tailored-made software for sonifying muscle-tone and force. This enables the body to reflect expressively the content of the media and the architecture of the device.
As part of the BODYinTRANSIT project, we are investigating the capacity of Joakinator to transform body perceptions. Research has shown that people’s body perceptions are highly malleable, and recent works have demonstrated that perceptual illusions of body transformations can be created using sound. The sonification of body movements with metaphorical sounds has also shown potential to transform body perceptions, movement perception, proprioceptive awareness, and affect movements.
We would like to share our experiences in this workshop and explore the feedback between approaches that reach perception, body, and creative process, specifically using body maps, body storming, post-phenomenology, and autoethnography. We believe that this interdisciplinary approach could provide useful insights for both the art-based research and psychoacoustics communities.To support our presentation, we would like to provide a short video summary and pictorial that explain our methodology and the insights we have already found.