Joakinator: An Interface for Transforming Body Movement and Perception through Machine Learning and Sonification of Muscle-Tone and Force

Joakinator is a wearable interactive interface that allows users to activate different media materials, such as sound, music, and video, through body gestures. The device, designed in the context of music and performing arts, integrates surface electromyogram, force sensors, and machine learning algorithms with tailored-made software for sonifying muscle-tone and force. This allows the body to reflect expressively the content of the media and the architecture of the device. Recently, we have started to investigate the potential of Joakinator to alter body perception in the context of BODYinTRANSIT, a European Research Council Project focused on the transformations of body perception through the use of interactive sound/haptics technology. At NIME-2023, we will showcase Joakinator and invite visitors to experience the device firsthand. Visitors will have the opportunity to try on the device, observe others using it, and reflect on its capabilities to transform body movement and perception through the sonification of muscle-tone and force. Overall, Joakinator is a technology that pushes the boundaries of body-computer interaction and opens new possibilities for human-computer interaction and expression.