by DAMIEN RICKETSON
Background
Multimodal communication, a ‘text’ delivered via multiple integrated sensory modes, has been hailed as becoming ‘the normal state of human communication’ (Kress, 2010). While a great deal of multimodal theory is applied to educational and marketing contexts where the primary concern is the most efficient and direct method of delivering a message, my practice in the creative arts finds interest in the effect of combined media in creating deepened engagement, dynamic comprehension and facilitating – what Jeanne Fahnestock and Marie Secor (2008) might describe as an individualised or subjective experience and meaning.
As a composer and the Co-Artistic Director of Sydney’s specialist new music organisation Ensemble Offspring, I am continually faced with the task of effectively communicating challenging new work to the public. As with all practicing artists, I seek a deep (if not mass) engagement with my audience. Experiments in sound materials, performance techniques and presentation strategies are undertaken, audience reception observed and the apparent strengths and weaknesses of previous work analysed to inform future work. With 20 years of collating and evaluating feedback, particularly in relation to Ensemble Offspring, it is surprising how much commentary has little to do with the actual experience of music. Positive responses regarding the physicality of performances, program notes, in-concert talks, cross-artform collaborations and multisensory immersive experiences form recurrent themes. As someone who is in the business of making sound, it is interesting to note how much the secondary non-auditory aspects of live music impact audience reception.
Aims
In this paper I offer some personal insights on multimodality in music from the perspective of a creator and curator of new music. Implicit in my proposition is that an understanding of multimodal communication theory goes some way towards explaining perceived successful audience engagement, especially when communicating unfamiliar music to the public.
I broach the issue of deep engagement primarily from the perspective of Ensemble Offspring, a Sydney-based arts company dedicated to the performance of innovative new music. The organisation has a growing body of data that highlights the value of extra-musical information in achieving successful audience engagement. This data has informed strategies to enhance concert activities in both pure chamber-music programs as well as actively encouraging the Ensemble to pursue projects with other art forms.
I will then draw on my work as a composer and a presenter with a brief examination of The Secret Noise, a recent show-length work presented by Ensemble Offspring. The work represents the culmination of many ideas associated with multimodal communication and points towards future strategies in the application of multimodal theory in creative practice.
Main contribution
Ensemble Offspring and artistic impact
As the Co-Artistic Director of Ensemble Offspring, I am frequently challenged to measure the impact our performances have on our audiences. This pressure comes from multiple stakeholders but most tangibly from government funding agencies. To justify the spending of tax-payer dollars Ensemble Offspring is required to not only report quantitative measures (such as audience numbers), but is increasingly encouraged to formalise ‘artistic reflection’ mechanisms that make a more qualitative assessment of audience engagement (Australia Council for the Arts, 2014). As with other key arts organisations in Australia, Ensemble Offspring now employs an artistic self-assessment tool that entails the compilation of data on all concert activities (Ensemble Offspring, 2012–15). Repeat analyses of perceived success against the key performance area of ‘audience engagement’ reveals some interesting data suggesting connections between multimodal presentation and deepened audience engagement.
In relation to Ensemble Offspring’s chamber music concerts, positive commentary on the augmentation of music via extra-musical information has clustered around two themes: (1) visual impact, the physicality of performance and; (2) language, the connection between audience education and audience appreciation (eg. program notes and on-stage spoken introductions). A third cluster of positive feedback has centered on multimedia and hybrid events: where other modes of perception go beyond supporting an auditory medium to being integrated in a more equal artistic relationship. The Ensemble now has a significant track record of cross-disciplinary collaborations including experimental film, video, dance and theatre that feature prominently in the artistic assessment reports. The project that has attracted the highest level of feedback is The Secret Noise (Ricketson, 2014).
Case study: The Secret Noise
The Secret Noise is a hybrid show-length production and set of recordings. The live version of the work was premiered by Ensemble Offspring and involved three musicians, two dancers and an actor who created much of the work in a collaborative fashion. The project brings together many aspects of multimodality in music including my own personal compositional motivations (the nexus between movement and music and the open multifaceted work), as well as curatorial motivations with Ensemble Offspring (cross-art form collaboration and integrated experience design).
The audiences’ experience of The Secret Noise arguably begins before the performance with the receipt of an abstract artwork which they are invited to personalise, place in a sealed envelope and submit ‘for consideration’ at the performance venue. The drawings function as both an arcane form of documentation that the audience quickly learns to use to unlock individual one-on-one performances in private glowing tents as well as notation: a graphic score that is interpreted into sound or movement depending on the performer. No two audience-members can experience the opening installation in the same way. The meaning of the visual script is dynamic and only partially constructed by the audience as they layer their different one-on-one experiences together with silhouetted clues from being a voyeur to others’ individualised performances.
The installation gives way to a promenade of scenes exploring music and secrecy that occupy an unusual artistic space between music concert and dance performance. The relationship between music and movement is highly integrated throughout the work. For example, in one scene, the natural movements of the musicians generated the physical vocabulary of the dancers in a somewhat grotesque parody of chamber music conventions. In another scene, the dancers performed newly invented whirling instruments. As such, their choreography generated the sound, which provided the sonic material on which the musicians extrapolated their melodies and textures.
The Secret Noise is also arguably multimodal in that the recorded form of the work is not a recreation of the live experience. Although sharing similar themes and sound-sources, the live version embraces the theatricality of the visual-sonic spectacle, while the recorded version is more ambient in sensibility and the process of composition different: the live version was the result of detailed score-writing and creative development work, while the recorded version was the result of studio production. For example, in the live version, the newly invented whirling instrument sequences were conceived as choreography first, an exploration of the striking visual impact of the instruments, with sound the by-product of choreography. In the recordings, without the visual stimulus, a different compositional strategy was employed that focused on capturing the unique Doppler effect of the instruments in meticulous tempi which were then overlayed and assembled in the studio to allow the creation of complex polymetric sound patterns. Furthermore, the collection of recordings, comprising a CD, vinyl EP and digital-only content, contain different music that is unique to their particular media (spiral grooves, reversible and hidden tracks, etc.) and were approached with subtly different aesthetics (analogue and digital). That is, every different medium in which the work exists was composed differently.
Multimodality in The Secret Noise exists at multiple levels from the hybrid nature of the live performance to the very existence of the work in multiple forms, to the existence of a supplementary supporting website that provides additional content (text, image and sound). From the moment an audience member receives their abstract artwork their potential engagement with the work is varied. Perhaps they’ll just come to the live performance and leave it at that. Perhaps they’ll experience the live version, purchase the recording and experience the work in a different way. Or perhaps even log in to an associated website including a virtual ‘secret chamber’ where there is the option of contributing a creative response. Multimodality, as understood in The Secret Noise, doesn’t guarantee heightened audience impact per se, but does present the opportunity for audiences to engage with the work in varying degrees of depth.
Conclusions
Ensemble Offspring’s mission to present innovative music in an engaging way is measured by an artistic self-assessment tool. With the data currently available, links can be drawn between the non-musical aspects of performance and deepened audience engagement. Such links resonate closely with multimodal communication theory.
The data affirms some of Ensemble Offspring’s current approaches to concert presentation and also suggests some potential changes for future projects. Generally speaking, there is a curatorial move towards a more holistic approach to programming that is concerned with the visual, as well as the sonic, and plans the full audience experience from the moment they enter the performance space.
The experience of collaborating with visual-based artists, particularly a theatre director, has informed the dramaturgy of our pure chamber music programs including a heightened awareness (and sometimes exaggeration) of the natural gestural communication of chamber music and the adoption of a presentation aesthetic that see the musicians as performing from the moment they are on stage (not just while playing music). For example, the Ensemble now makes a point of rehearsing the transitions between works (not just the works themselves).
Positive feedback relating to program notes and spoken introductions will also inform the design of a new digital strategy that draws upon developments in experience design and augmented reality to enhance the concert experience. Although still in the very early stages of development, the Ensemble is planning to allow audiences the use of networked devices during performances (smart phones and tablets) to access a supplementary layer of digital information in real-time during performance (from simple notes about the works, composers and performers, to potential participatory sound and image generation). Multimodality is also increasingly informing artistic planning from the perspective of the work we commission, especially in relation to affirming ongoing forays into hybrid arts practice and the pursuit of interdisciplinary collaborators.
Although implied, it should be noted that drawing direct links between multimodality and deepened audience engagement still remains somewhat speculative and it should not be forgotten that multimodality says more about medium than the message. Untangling the artistic quality of the musical content vis-à-vis the way in which that content is presented in Ensemble Offspring concerts is difficult to gauge when making qualitative claims of perceived success or otherwise. In the context of an organisation devoted to the development of new work, where the repertoire presented has little or no performance history, this challenge is exacerbated by the lack of comparative data to act as control references. Nonetheless, an understanding of multimodal communication theory does suggest interesting strategies for music creators and presenters seeking a deep engagement with their audience.
Notes
Damien Ricketson is a Lecturer in Composition at the Sydney Conservatorium of Music, The University of Sydney and the Co-Artistic Director of Ensemble Offspring.
Email: damien.ricketson@sydney.edu.au.
References
Australia Council for the Arts (2014). Artistic Vibrancy: A way for organisations to talk about artistic impact [E-book]. Retrieved from http://australiacouncil.gov.au/strategies-and-frameworks/artistic-vibrancy/.
Bezemer, J. & Kress, G (2008). Writing in Multimodal Texts: A Social Semiotic Account of Designs for Learning. Written Communication 25(2), 166-195.
Ensemble Offspring (2013). Artistic Assessment Policy. Internal unpublished document.
Ensemble Offspring (2012–15), Artistic Assessment Reports. Internal unpublished documents.
Kress, G. (2010). Multimodality: A Social Semiotic Approach to Contemporary Communication. New York: Routledge.
Ricketson, D. (2014). The Secret Noise [CD & EP recordings]. Sydney: Curious Noise.
Ricketson, D. (2014). The Secret Noise [website]. Retrieved from http://www.thesecretnoise.com.