by SVETLANA RUDENKO
Background
By observation of students of varying ages, over twenty years of pedagogical practice (piano students aged 6-12 in this particular case), it is evident that the reaction to sound is accompanied by stimulation of other senses: feeling of melody shape under fingers, random visual associations of a character or the associations of musical texture with an emotion, taste or smell. Galeyev (2007) states that ‘synesthesia is a normal and common ability of intersensory association, a particular manifestation of imaginative thinking or (when it appears in verbal form) a double metaphor, in which the transfer of meaning inherent in metaphor is accompanied by the transition into another sensory modality.’ (p.285). It is common among students of a young age to confuse perception of ‘fast tempo’ with increased loudness, and softer dynamics with ‘slow’ music.
Thus, the question is raised: what is the underlying reason for this perception? Some hypotheses explain these phenomena by the condition which we are all born into: neonatal synesthesia. As noted by Maurer and Maurer (1988) ‘the newborn does not keep sensations separate from one another’ but instead ‘mixes sights, feelings, and smells into a ‘sensual bouillabaisse in which sights have sounds, feelings have tastes’ (p.51). Cross-modal processing is characteristic to infants’ and children’s auditory-visual pairings. This mechanism assumes that, although early processing of auditory and visual input is parallel, attention is allocated in a serial manner with the modality that is faster to engage attention dominating later processing. An alternative approach to the learning process is suggested here, making the connection between science and pedagogy, which takes into account the child’s physical perception of, and response to, child-like imagery in order to simplify the complex musical and technical challenges of piano playing. The process of sound visualisation may be referred to as ‘mind mapping’, or as described by natural synesthetes as: an ‘inner screen’ of ‘seeing’ a ‘mental image’ with their ‘mind’s eye’ (Ward, 2008, p.14).
Aims
It could be possible to enhance the musical imagination (and therefore learning) of a child by finding ways to stimulate the voluntary synesthetical senses, as this should boost the involuntary senses in return. The aim of this research is to develop and apply a pedagogical method of tactile and sound visualisation based on cross-modal communication of senses in response to a musical texture of an image. The examples chosen are from Gubaidulina’s ‘Musical Toys’, in which image/ character gestures are clearly recognisable in the musical score.
Main Contribution
Analysis
Recent research by Akiva-Kabiri, Linkovski, Gertner, & Henik (2014) has shown that ‘in musical-space synesthesia, musical pitches are perceived as having a spatially-defined array. Previous studies showed that symbolic inducers (e.g. numbers, months) can modulate response according to the inducer’s relative position on the synesthetic spatial form’ (p. 17). Tests were carried out on ‘two synesthetes: SA (25 years old), and AB (23 years old). They were right-handed females with 7 years and 12 years of formal musical training, respectively. SA and AB reported visualizing each pitch of a given octave in a distinct spatial location. Interestingly, both reported that the pitch tones rose up diagonally from lower left to upper right. Synesthesia was assessed using short interview. During the interview the synesthetes were asked to draw a sketch that simulated their pitch organization. The synesthetes did not report other types of synesthesia apart from their musical-space synesthesia’ (p. 19).
Galeyev (1993) mentions that ‘audio space may even have its own co-ordinates: depth (texture), vertical (melodies), horizontal (architectonics of music pieces as a whole)’ (p. 76). Leikin (2011) analysed transcriptions of the piano rolls, which Scriabin (a synesthete) recorded on the Hupfeld and Welte-Mignon reproducing pianos in 1908 and 1910: ‘When Scriabin’s performing tempo fluctuates continuously and widely and yet the average tempo coincides with the published metronome indication, it means that Scriabin keeps concurrently, side by side, two timelines. One is the underlying steady pulsation in the indicated tempo; the other consists of changeable beats’ (p. 30).
It is evident that synesthetic perception is different, not only with respect to the observation of an image and its association with broader senses, but in its influence upon the spatiotemporal organisation of musical textures. Synesthetic minds may deal with image processing faster: keeping layers of musical texture independent but observing them at the same time, enjoying faster emotional responses and richer aesthetic experiences.
Ramachandran and Habbard (2003) note that: ‘one skill that many creative people share is a facility for using metaphor. It is as their brains are set up to make links between seemingly unrelated domains. In other words, just as synesthesia involves making arbitrary links between seemingly unrelated perceptual entities such as colors and numbers, metaphor involves making links between seemingly unrelated conceptual realms’ (p.57).
Garth and Porter (1934) reported the tendency of young children to attend for longer to colours, relative to achromatic stimuli. On the basis of known properties of neurons in the visual cortex (i.e. that different neural populations are selective for different parameters of the visual stimulus (Bornstein, 1978)) Rogers (1996) suggested that ‘the addition of colours to an achromatic symbol such as a note or rest would add to the neural activity as compared to the activity resulting from the shape alone’ (p. 174).
Extracting gestured, textured images by colouring the score helps to build an overall structure of the piece, resulting in a stronger memory based on the mental mind map, and allowing imagination to flow according to the development of a musical ‘story’. Labelling the image may therefore be helpful in generating both musical ‘imagery’ and tactile associations with the fingers, helping to create a ‘sculpture’ from the sound.
A visualisation of musical texture based on the child’s cross-modal processing of an image is a natural way of developing a synaesthetic ‘inner-screen’, as used by natural synesthetes . This process may therefore be thought of as ‘turning sounds into vivid imagery’ (Luria, 1968, p. 97).
Sofia Gubaidulina, a composer whose pieces here demonstrate examples of synesthetic perception, has described her experience of sound/ composition as such: ‘In the first instance, perhaps on a walk, I hear a huge, shapeless, multi-faceted sound, absolutely fascinating, with everything piled up together in a way you could never notate-something which exists outside time. It’s like a present, and I consider it a duty to transform it from vertical to horizontal’ (Lukomsky, 1999, p. 30). The following diagrams attempt to reproduce models of sound-sculpture, according to synesthetic perception. Figure 1 is an example of synesthetic screen of the musical texture; Figure 2 shows Galeyev’s ‘detailed diagram of probable synesthetic ties between external and internal sensations. Point O1 designates interoceptive sensations and Point O2 designates proprioceptive sensations’ (Galeyev, 1993, p. 77); Figure 3 is a model of a secondary sensory musical image association.
Figure 1. (Left) Scriabin’s Deux Morceaux Op. 57, No.1 ‘Désir’.
Figure 2. (Right) Galeyev’s diagram demonstrating the probable synesthetic ties between sensations.
Figure 3. Image processing in a synesthetical mind
Subjectivity of perception
Kevin Mitchell (2010, para. 1) remarked: ‘For millennia, philosophers have mused over the nature of perception, how closely it mirrors “reality” and different people might, quite without knowing it, subjectively perceive the world in very different ways […] the fascinating condition of synesthesia provides a stark example where the quality of subjective experience is very definitively and demonstrably different. This may be due to genetic variants which affect the functional segregation of specialized circuits in the brain.’ In other words, to experience music in one’s very own subjective way may be considered as akin to a personal journey. In the context of child and teacher, this subjectivity is exaggerated further by perceptual differences arising due to infantile synethesia. ‘For people with synesthesia, particular stimuli automatically and involuntarily elicit a characteristic secondary percept or feeling. Thus, hearing particular sounds may stimulate the very real perception of colored shapes in visual field, tasting flavors may induce the tactile sensation of specific objects, and different textures may induce specific emotions’ (Mitchell, 2010, p. 429).
Gubaidulina has said about her Musical Toys that ‘I often thought of my childhood and of the lack in those days, of piano pieces that were able to take one back into the highly imaginative world of toys. At that time I also looked upon toys as material from which I could elicit sounds; they were part of the world of my musical sensations. With this collection, I have paid a late tribute to my childhood’ (cited in Roster, 1995, p. 6).
The fourteen short piano miniatures are studies in expression and style, in which the piano’s various sound possibilities and its techniques of touch-imaginative, sensorial and contrast are of central importance. The poetic, mysterious or amusing pictures suggested by the titles- whether character-portraits of musical instruments, animals, musicians or musical descriptions of resounding or moving tools or of living landscapes. The following examples of scores (Figures 4, 5 and 6) demonstrate the attempts of the author to extract images according to the musical textures that are present. As a result of structured colouring, they aim to model the so-called ‘inner synesthetical screen’.
Figure 4. Gubaidulina’s Musical Toys, No.1: Mechanical Accordion
In Gubaidulina’s Mechanical Accordion (Figure 4) two images may be extracted: 1. Mechanical Accordion (green graphics); 2. Voice/ Whistle (red marking).
Figure 5. The Magic Smith from Gubaidulina’s Musical Toys.
In Magic Smith (Figure 5) the author extracted three images characterised by very individual melodic and textural gestures.
Figure 6. Song of the Fisherman from Gubaidulina’s Musical Toys (1969)
Child/Adult Perception
Trehub (2003) has discussed adult-infant parallels in music processing: ‘Infants recognize the invariance of melodies across shifts in pitch level (transpositions) and tempo. For adults, as for infants, these changes are detectable, but they are irrelevant to the identity of musical pieces. Equally intriguing is the finding that infants are more precise in perceiving diatonic melodies that violate the conventions of known music: melodic contour is a particularly salient dimension of novel melodies for adults as well as infants. For diatonic melodies, infants detect pitch changes of a semitone or less, even when the melodic contour is unchanged’ (p. 669). Concerning the functional specialisations for music processing in the human newborn brain, Perani et al. (2010) state that: ‘In adults, specific neural systems with right-hemispheric weighting are necessary to process pitch, melody and harmony, as well as structure and meaning emerging from musical sequences. It is not known to what extent the specialization of these systems results from long-term exposure to music or from neurobiological constraints […] Music modulates infants’ attention and arousal levels evoke pleasure or discomfort. Infants with casual exposure to music possess the abilities for relational processing of pitch and tempo; for the differentiation of consonant vs. dissonant intervals; for the detection of variations in rhythm, meter, timbre, and tempo as well as duration of tones and musical phrases’ (p. 4758).
Natural synesthetes tend to display great sensitivity to different timbres and textures, since their ‘neonatal system of perception’ has survived into adulthood, facilitating cross-modal multisensory experiences of music. What is important in pedagogy is to exploit the inherently synesthetic tendencies of the infant in order to support learning, and not to close down this natural source of creativity.
Bor, Rothen, Scwartzman, Clayton and Seth (2014) have reported that ‘adults can be trained to acquire synesthetic experiences’. This presents a valuable opportunity to enrich adults’ music experience via cross-modal associations. Ramachandran and Hubbard (2001) suggested that ‘synesthesia causes excess communication amongst brain maps […] depending on where and how widely in the brain the trait was expressed, it could lead to both synesthesia and to a propensity toward linking seemingly unrelated concepts and ideas – in short, creativity’ (p. 32).
Conclusions
Cross-modal associations may be beneficial in teaching music to young children. Adult perception and childlike imagery are different, however. In order to stimulate and utilise the creativity of a young brain to its maximum potential, the teaching process should aim to communicate using a variety of methods including: the psychology of multisensory perception, image visualisation, tactile visualisation, sound visualisation and mind mapping.
Notes
Address for correspondence: Svetlana Rudenko, Candidate PhD in Performance, RIAM, 36-38 Westland Row, Dublin 2, Ireland.
Email: ssrrudenko@gmail.com.
References
Akiva-Kabiri, L., Linkovski, O., Gerthner, L., & Henik, A. (2014). Musical space synaesthesia. Consciousness and Cognition, 28, 17-29.
Bor, D., Rothen, N., Schwartzman, D., Clayton, S., & Seth, A. (2014). Adults can be trained to acquire synesthetic experiences. Scientific Reports, 4, 7089.
Bornstein, M. H. (1978). Visual behavior of the young human infant: relationships between chromatic and spatial perception and the activity of underlying brain mechanisms. Journal of Experimental Child Psychology, 26(1), 174-192.
Galeyev, B. M. (1993). Synesthesia and musical space: on Yavorsky’s forgotten hypothesis and a proposal for an experiment in zero gravity. Leonardo, 26, 76-78.
Galeyev, B. M.(2007). The Nature and Functions of Synesthesia in Music. Leonardo, 40(3), 285-288.
Garth, T. R. & Porter, E. P. (1934). The color preferences of 1032 young children. The American Journal of Psychology, 448-451.
Kurtz, M. (2008). Sofia Gubaidulina, a biography. Bloomington: Indiana University Press.
Leikin, A. (2011). The performing style of Alexander Scriabin. Farnham, Surrey: Ashgate.
Luria, A. (1968). The mind of a mnemonist. New York: Basic Books.
Lukomsky, V. (1999). ‘Hearing the subconscious’: interview with Sofia Gubaidulina. Tempo, 209, 27-31.
Maurer, D. & Maurer, C. (1988). The world of the newborn. London: Viking.
Maurer, D. & Mondloch, C. J. (2004). Neonatal Synesthesia: A Reevaluation. In L. C. Robertson (Ed.), Synesthesia: perspectives from cognitive neuroscience. Oxford: Oxford University Press.
Mitchell, K. J. (2010). Book review: Wednesday is indigo blue: discovering the brain of synesthesia. American Journal of Human Biology, 22(3), 429-430.
Mitchell, K. J. (2010, March 30). Synesthesia: crossed wires or free association? Retrieved from http://www.wiringthebrain.com/2010/03/synesthesia-crossed-wires-or-free.html.
Perani, D., Saccuman, M. C., Scifo, P., Spada, D., Andreolli, G., Rovelli, R., … & Koelsch, S. (2010). Functional specializations for music processing in the human newborn brain. Proceedings of the National Academy of Sciences of the United States of America, 107, 4758-4763.
Ramachandran V. S. & Hubbard, E. M. (2003). Hearing colors, tasting shapes. Scientific American, 52-59.
Ramachandran, V. S. & Hubbard, E. M. (2001). Synesthesia — a window into perception, thought and language. Journal of Consciousness Studies, 8, 3-34.
Rogers, G. L. (1996). Effect of colored rhythmic notation on music-reading skills of elementary students. Journal of Research in Music Education, 44(1), 15-25.
Roster, D. (1995). [Liner notes]. In Sofia Gubaidulina: the complete piano music [CD]. Åkersberga, Sweden: BIS Records.
Trehub, S. E. (2003). The developmental origins of musicality. Nature Neuroscience, 6, 669-673.
Ward, J. (2008). The frog who croaked blue: synesthesia and the mixing of the senses. London: Routledge.