by Renee Timmers, Nicola Dibben, Zohar Eitan, Roni Granot, Tim Metcalfe, Andrea Schiavio, Victoria Williamson
It is our pleasure to present to you the Proceedings of the International Conference on the Multimodal Experience of Music, held from 23-25 March 2015 in Sheffield, UK. The conference was the first of its kind and brought together researchers from a diversity of disciplines with a shared interest in ways in which different modalities including vision, touch, hearing and kinematics interact when listening to, imagining, composing, editing or performing music. The proceedings contain only a small part of the work presented at the conference, but nevertheless are representative of the kind of questions that were raised, the approaches taken to study multimodality of musical experiences, and the implications offered for music technology, education, cognition, composition and performance. The proceedings papers are ordered according to topic as follows:
Papers 1, 2 and 3 examine the multisensory basis of musical behavior, theoretically and empirically. The first paper, by Glasser, looks at the implications of synesthesia for musical abilities in conservatory students. The second paper (Schiavio & Cummins) argues for innovations in music education, and the third (Van Puyvelde & Fabio) evaluates the role of gestures in infant development.
Papers 4-8 investigate relationships between movement and music perception and performance using experimental methods. Papers 4 (Metcalfe) and 5 (Küssner & Caramiaux) ask participants to walk or move to music, respectively, while Timmers & Li (paper 7) investigate associations between spatial representation of pitch and movements related to pitch production on musical instruments. Roze and colleagues (paper 8) explore relationships between sound production and freedom of movements in cello performance.
Papers 9-14 investigate and develop new multimodal technologies for music. McKenzie, Lennox and Wiggins (paper 9) present an intriguing alternative to audio presentation via air conduction, in the form of conduction through tissue which has potential relevance for listeners with hearing impairment. Papers 10 (Mechel & Altinsoy) and 11 (Jack, McPherson & Stockman) explore the perception and evaluation of tactile presentations of musical audio. The final three papers of this section examine cross-modal aspects of multimedia, looking into the effects of image compression on the evaluated quality of musical audio (Hammerschmidt & Wöllner), haptic or haptic/audio presentation of sonic icons (Borphy et al.) and efficiency of user interface designs for audio mixing (Mycroft, Reiss & Stockman).
Papers 15-19 explore the role of cross-modal associations in the perception and performance of contemporary and electronic music. Hubert & Wang investigate the effect of descriptive imagery on emotional responses to electronic music. Moore explores cross-modal associations with electronic music from a composer’s perspective. Ricketson explores audience responses to contemporary music, and Peñalba and colleagues present developments of compositional tools that translate movements into sounds. Rudenko explores synesthesia and the use of visual imagery in the performance of works by Sofia Gubaidulina, and their application in music education.
Finally, papers 20-22 present theoretical perspectives on multimodal musical experiences. Deng, Kang and Wang (paper 20) argue for the need to recognize the distinctiveness of soundscape compositions as a musical genre. Eyes (paper 21) explores improvisations of a musical score to film from a practitioner’s perspective. And finally, Zhou (paper 22) analyses different types of accents in performances of Chinese opera.
As is often the case with proceedings papers, these articles do not necessarily provide the final answer to issues, or the final conclusive evidence, but rather explore novel avenues of research and highlight directions for further investigation. They provide a valuable snapshot of current research in a rapidly expanding and developing area. The success of ICMEM 2015 demonstrated the primacy of multimodality to understanding music and its place in human experience, highlighting the need for increased attention to and investment in this area of research.
Yours sincerely,
Renee Timmers, Nicola Dibben, Zohar Eitan, Roni Granot, Tim Metcalfe, Andrea Schiavio & Victoria Williamson