'TranSonic' Perception in Interactive ChoreoSonic Performance Practice
This paper reflects on the creation and perception of an interactive spatial ChoreoSonic1 performance environment. The term ‘Transonic Perception’ is introduced to describe the intimate bodily experience of the dancer who creates the 3D spatial sonic environment within the technologically enhanced environment. I will discuss conceptual issues that outline a synchronicity between the spatial elements of dance movement and 3D spatial sound perception that can be used artistically in the creative process. The discussion focuses specifically on the human perception of the visual, tactical and auditory space in the digitally enhanced performance environment.
Both the individual and collective ‘living architecture’ (Laban 1966:5) of the dance performer(s) is used as the starting point for a real time transformation of dance movements into a 3D spatial digital sound composition. I will discuss artistic strategies for transcending the conventional perception of musical form in dance performance into a perception of the sonic environment as an interactive ‘space-rhythm-movement’ dimension. The creation of the ChoreoSonic performance environment is realized with the application of five wireless ultrasonic sensors positioned on the dancer’s body. These sensors are able to measure the spatial position, direction, speed, rotation and proximity of the moving body(parts) within a sensitive area of approximately 25m2. This configuration makes it possible to shape the real time ambisonic surround sound environment through body movement in using the visual programming environment Max/MSP/Jitter3. The end of the paper proposes that ‘TranSonic’ perception creates an interactive ChoreoSonic relationship between the dancer as a ‘living architecture’ and ambisonic surround sound as a ‘moving sonic architecture’.
When creating interactive choreographic sound Verstraete (2005:202) remarks that ‘It shifts our attention from visual geometric space to acoustic space’. He (2005:6) keeps the ChoreoSonic relationship close to the body when he states that ‘sound can add an auditory “geography” like a second skin to the dancing body’. Sound becomes an almost tactile and sensual experience for the dancer suggesting ‘dance as an embodied sound’. I argue that in the ChoreoSonic environment a virtual spatial sound body outside the dancing body is created suggesting ‘sound as a disembodied movement’. Sonic space is not an inactive background, but the interactive director of the dancer through the responsive emergence of spatial sound.
Diana Theodores points out in her introduction to the ‘Connecting Bodies’ Symposium (1996)4 that: ‘interactive immersive computer technologies extend and transform the shape of movement and choreography, and if digital media can penetrate the materiality of the body, then our perceptual and ontological notions of embodiment are profoundly affected (quoted in Birringer 1998:125). Considering this issue, Chrissie Parrott refers to the fact that technology can have a positive influence on the dancer’s perception. Concerning the use of Motion Capture technology and the software Life Forms5, she observes that: ‘The technology redefines the principles of space and time that we’ve always looked at as choreographers, and we will continue to look at that, but it helps us redefine them and it helps us redevelop those ideas’ (quoted in McKechnie & Potter 2005:105). Following the ideas of Parrot, I will firstly investigate space from the various viewpoints of dance and ambisonic surround sound in the ChoreoSonic performance area to underpin the choices for the movement measurements in the ChoreoSonic performance research.
Space and Dance
Space is a hidden feature of movement and movement is a visible aspect of space (Laban 1966:4)
Space as a medium for movement has been conceptualized and articulated by movement theorist Rudolf von Laban in the late 1920s as introduced in his principles of ‘Space Harmony’. His study of (classical ballet) movement, the Laban Movement Analysis (LMA), deals with the spatial order of the paths or traceforms that the dancer’s limbs make in space, taking in consideration the connection between ‘the outer result of movement and the mover’s inner attitude’ (Laban 1966:27). In dance this traceform is constructed out of changing spatial and rhythmic tendencies. Laban stated firstly that ‘equilibrium in dance is never complete stability or a standstill, but the result of two contrasting qualities of movement’ (Laban 1966:6) and secondly that ‘in movement each reaction has an equal but opposite reaction’ (according to Lovell et al. 1996).
Throughout his book, Laban (1966) proposed that movement of the body is made up of pathways in which the movement phrase changes bodily positions as well as the combined relationships and connections within the structure of the body. Laban considers the fact that limbs are only able to move in certain restricted areas of the kinesphere (the so called body ‘zones’). The term ‘kinesphere’ can be defined as:
Several neuro-physiological sensations are associated with the first definition of the kinesphere. First of all, the so-called ‘sixth sense’ that is defined as ‘pro-prioception’, the sense of motion and position that ‘bind[s] our sense of agency with our embodied selves at an emotional level’ (Cole et al. 2007). The term ‘kinesthesia’ is interrelated with ‘proprioception’ and is similarly defined as ‘the sense that detects bodily position, weight, or movement of the muscles, tendons, and joints’7. Secondly, the peri-personal or ‘near’ space which is defined as the space closely around the body which spatial coordinates are initially perceived by the brain with reference to the sensory organs8. The technology evokes a dynamic representation of the position and movement in the ChoreoSonic environment by the trained pro-prioception, sensitivity of the peri-personal space and kinesthetic sense of the dancer.
The second definition of the kinesphere is derived from the abovementioned LMA. Within his principles of ‘Space Harmony’, Laban (1966:10) defined the ‘kinesphere’ or ‘personal space’ as ‘the sphere around the body whose periphery can be reached by easily extended limbs without stepping away from that place which is the point of support when standing on one foot [...]’. In other words, the kinesphere is defined as the space around a dancer’s body limited by the maximum space that the limbs can reach. The centre of the kinesphere is the pelvis, defined as the dividing point of the three possible movement directions, height, breadth and depth (ibid:11). It has been proposed by Kirstein et al. (1953) that limbs also have their own individual dynamic kinespheres (fig. 1).
Fig. 1 Space Modules of the Arms and Legs I (Kirstein et al. 1953:2).
These individual kinespheres define the spaces of the two legs and the two arms as the ‘space modules’ of movement in which the ground of style and technique resides. These movement spaces were chosen as the trigger space for the spatial sound in the ChoreoSonic environment. I attach four sensors on the hands and feet that determine the kinesphere of the four limbs and one on the head (fig. 2) that measures the position of the centre of the body in the performance space with a simple mathematical calculation. Each sensor spatially directs its individually allocated sound. In this way, the body and the limbs direct their allocated sounds to the distance ranges of the surround sound set up. Thus a certain synchronicity between the spatial elements of dance movement and 3D spatial sound perception is created.
(Fig. 2) Dancer S. Spasic with 5 sensors of the LCIPS in a pentagonal structure.
Space in dance can be defined in terms of dimensions, planes (vertical, horizontal, saggital) and diagonals. If the space is experienced through the body centre of the kinesphere there is a multiplicity of directions that these main dimensions can relate to referring to geometry and boundary of human movement. My creative process concentrates on two main spatial and dimensional issues of the ‘Space Harmony’ principles9 as described by Laban (1966):
The combination of these two issues directs us to the fact that the dancer’s body is bound to the kinesphere, but the kinesphere is mobile in the context of general space. Laban referred to the dancer’s movement as a ‘living architecture’: ‘movement is, so to speak, living architecture- living in the sense of changing emplacements as well as changing cohesion’ (Laban 1966:5). I relate these spatial views to the interactive ChoreoSonic research in the following way:
Space and Sound
Västfjäll highlights above an important reason for me to use spatial sound when he showed that spatial sound reproduction increases the emotional perception of sound. I noted earlier that the moving body has been considered a ‘living architecture’ in space. Therefore, a question arises - can interactive spatial sound in a ChoreoSonic performance environment also be considered as a moving ‘sonic architecture’ in space?
Ambisonic surround sound10 can be defined as true 3D sound information and reproduces sound in both vertical, horizontal and depth directions around a centrally positioned listener. The ambisonic method was initially invented to archive a better way of the spatial representation of sounds recorded by microphones. Technically speaking, ambisonic encodes and decodes sound through the use of several equations and assigns a precise X, Y and Z cartesian coordinate11 to every sound. The speakers used in an ambisonic environment should all be full range and preferable the same. The speaker lay out can vary from stereo, hexagon, octagon to cube or any other symmetrical configuration. Michael Gerzon, the inventor in Oxford of the mathematical codes needed for ambisonic sound (early 1970s) refers to ambisonic sound as ‘full sphere sound’ or ‘periphony’ (Gerzon 1980). Full sphere sound ‘requires speakers to be placed above and below the height of the listeners’ ears’12. When more speakers are used, the listening area is larger and a more stable sound localization is realized because the 'sweet spot' (the ideal listening spot in which the ambisonic sound field is reproduced accurately due to the algorithmic decoder process) becomes wider. In this way, listeners that are not positioned in the exact centre will hear more output from more speakers. In a full sphere ambisonic environment the geometry of the surround sound can be categorized as a cubiform in which the sound boundaries extend beyond the lines created by the speaker setup depending on certain parameters of the sound13.
Following this technical description of the background and the operation of ambisonic sound, I take a closer look at the spatial perception of full sphere ambisonic audio in the ChoreoSonic environment.
Considering the development of 3-D audio, Lennox et al., from the Signal Processing Applications Research Group at York University, stress that spatial perception is not an isolated feature but created by an interconnected relationship of all the senses. In line with this view, Blauert (1997:193) observed: ‘[T]he assumption underlying visual theories may be stated as follows: What the subject sees during sound presentation, and where the subject sees it, are factors determining the position of the auditory sound event’. Hofmann (2002) imagines himself as a ‘sonic architect’ who works on extending his practice through generating single sounds, placing them in an x, y, z three dimensional coordinate system. Ambisonic (surround sound) would make it possible to define sound in terms of its quality, time and 3D space. He goes on to propose the design of a generation of a whole ‘environment of sound’, like an architect who would create a building from the elements he works with.
In the ChoreoSonic environment 8 speakers are placed in a cubiform at the corners of the sensitive performance space. I argue that it is possible to create interactive ambisonic surround sound as a ‘moving sonic architecture’ inside this ChoreoSonic environment: a performance space is created that makes it possible to synchronise the living architecture of the dancer(s) with the movement of the sonic architecture. In this way the dancer’s two geometric architectures (the architecture of the body and the architecture of the space around it) are blended with the sonic architecture of the sensitive space created by the distance reach of the ultrasonic tracking system and the spatial sound system14.
However, it should be taken into account that spatiality in the ChoreoSonic environment is experienced differently when we observe from the outside as a viewer than when felt from the inside by a dancer who is actually directing the visible- and audible movements in space. The latter being able to choose to hear and experience sound from all spea-kers in the ChoreoSonic performance space more easily by participating to move in the direction of the spatial sound that s/he is guiding. In a live situation only a small minority of the audience may ever be in the most ideal spot to get the full spatial audio effect despite the influence of head movements and visual cues. A sound that is located in one speaker may simply never be heard by a part of the audience. However, this problem is partly solved during my research by giving the audience the freedom to move around in the sensitive ambisonic space to get the best spatial perception. The improvising dancer is trained to interact with the movements of the audience participants in the ChoreoSonic space. Andy Hunt & Ross Kirk (2000:385) remark that in an interactive environment ‘The control mechanism is a physical and multi-parametric device [that] must be learned by the user until the actions become automatic’ and ‘Further practice develops an increased control intimacy and thus competence of operation’. My research has shown that the same learning process also applies to the listener, who needs time to get accustomed to the moving sound as well as to the fact that s/he is able to walk around in the performance space.
During my practical PhD research (2007-2010), I have realized demonstrations and numerous experiments in the created interactive ChoreoSonic environment (Wijnans 201015). During these sessions, the improvising dancer was free to choose and trigger the sounds within the designed interactive environment. I (Wijnans 2009) concluded in an earlier writing that: ‘[...] the aspect of a technology performance improvisation becomes even more important, being able to interact, ‘play’, freely with a technology …’. In my thesis it was concluded that the dancer’s movements might become surrounded, and therefore overwhelmed, by the 3D sound. Sounds directed by more than one dancer started to blend at certain positions in the space. This raised questions of the identity and individuality of the dancers, whilst separated in space.
Theodores pointed us earlier to the fact that if ‘digital media can penetrate the materiality of the body, then our perceptual and ontological notions of embodiment are profoundly affected’ (quoted in Birringer 1998:125). In a technologically enhanced environment, the term ‘embodiment’ is frequently used as the bodily relationship of the dancer to the world. The term ‘disembodiment’ is used in these environments to refer to the ‘ideal’ relationship of humans to the computer that is one without any physical restraints. However, the term ‘disembodied’ quickly became contested in the artistic, technological, philosophical, neuro-physiological, and perceptual field. From a dancer’s point of view, Carolien Herman (2002) for example questions if: ‘New technology has created the ultimate, invisible body: the anti gravitational body, the multi-layered, the vanishing, the inside-out bodies’. In line with Herman, Gloria Mark (1997:221) poses the following question: ‘Should we really speak about disembodiment, or rather should we imagine a background-foreground relationship with our bodies where they exist more in the background as we enter a digital environment’? In her writing she argues that ‘[...] in a virtual world sensory information is restricted, either through a single or very few channels’ (ibid:223).
Embodiment from a philosophical point of view16 has been described by Maurice Merleau-Ponty (1962) as the perception by ‘a “system” of meanings by which the phenomenological process of recognizing and “sensing” objects takes place, and it is through the medium of the body that we get to “experience” and “perceive” the world’ (quoted in Ajana 2005:2). Perception is only possible through the body. Btihaj Ajana (2005:3) approaches the terms ‘embodiment’ and ‘cyberspace’ from a phenomenological point of view and goes on to state that conceptual ‘disembodiment’ is a ‘transcendence of body limitations through electronic prosthesis’. The term ‘transcendence’ means ‘exceeding usual limits of ordinary experience’ and ‘self-transcendence’ means ‘surpassing the conscious boundaries of oneself’17.
Herman (2002) relates this notion to the interactive performance ‘Telematic Dreaming‘ (1994) by Paul Sermon in which performer Suzan Kozel was transformed into a virtual image projected on a bed in another room. A visitor could approach and touch this image. Kozel stated that she felt physically present on the bed and felt physically hurt when people started to elbow her virtual image in the stomach. The virtual image was not disembodied but became a transcendental perception of the physical body. Herman observes that ‘The virtual body [of Kozel] is in this case the extension of the real body: in VR the virtual body becomes the scope and active radius of the touch. We think and perceive from the point of view of the virtual body’. Herman concludes:
Following the above discussion of ‘disembodiment’ and ‘transcendence’ in technologically enhanced dance environments, in the following section I would like to relate the views cited to movement based interactive spatial sound and introduce the term ‘TranSonic’ perception. This form of perception defines the experience of moving in general space from a dancer’s point of view while s/he is simultaneously creating ambisonic surround sound in the earlier defined abstract space necessary for creating spatial sound.
Mark (1997) favoured above a ‘disembodiment’ as a background-foreground relationship between the performer and the visual imagery. In an interactive sonic environment, Verstraete (2005:6) stated earlier that ‘sound can add an auditory “geography” like a second skin to the dancing body’. He mentions the interactive dance solo ‘Mes Jours et mes Nuits’ by sound designer Todor Todoroff and dancer/choreographer Michèle Noiret (200218) and the interactive installation ‘Sensuous Geographies’ by Sarah Rubidge and Alistair MacDonald (200319) as examples of projects in which sound directly affects the movement creation. Both environments use a multi speaker set up to create interactive spatial sound20. In this way the sound acts as an active spatial element that is able to motivate and contextualize (Stiefel 2002:12) the movements of either the performer (in ‘Mes Jours et mes Nuits’) or the audience as performer (in ‘Sensuous Geographies’). Duerden interprets this sensation of the dance-sound relationship as follows:
Sound becomes an almost tactile and sensual experience for the dancer. Kozel’s experience, mentioned above, was similar. In line with Duerden, I would like to introduce the term ‘TranSonic’21 perception to establish this experience: sound is going beyond the prior form of the human auditory perception. The ChoreoSonic perception exceeds the usual limits of ordinary experience by moving the movement-sound relationship closer to the body by adding a second (auditory) skin to the dancing body (Verstraete 2005:75). The neuro-physiological sensations mentioned earlier are valuable for this perception as they suggest the possibility of transcending the conscious body limitations of the dancer to be able to realize a ‘TranSonic’ experience. I propose to add to this observation that this experience realises the perception of a virtual spatial sound body outside the dancing body in the interactive ChoreoSonic environment: ‘sound as a disembodied movement’ and ‘dance as an embodied sound’.
Stan Wijnans is a Dutch interactive ChoreoSonic artist, performance developer and MAX/MSP/Jitter programmer. She holds a PhD (Bath Spa University, UK 2010) in Interactive Choreographic and Sonic Performance Art and an MA (Middlesex University, UK 2003) in Interactive Robotics- and Sound Performance Art. Her work investigates the human-machine relationship in interactive sound performances exploring and applying (3D-surround) sound, choreography, visuals, robotics and sensor systems.
1 A term coined during a research collaboration between Rubidge and myself in 2006 (see also Rubidge & Wijnans 2008).
2 The used Low Cost Indoor Positioning System (LCIPS) has been developed at the Department of Computer Science, University of Bristol, UK and is still in development (Randell & Muller 2001, Randell et al. 2002 and 2006). It is at the time of this writing able to track up to 6 RF/US sensors individually and synchronously.
4 At the ‘Connecting Bodies Symposium’ (1996) Theodores coined the term ‘technography’ as a way ‘to help focus on the mutually informing processes of technology and choreography’. See: http://art.net/~dtz/diana.html [accessed 12.05.11].
5 Life Forms ‘Studio Animation’ and ‘DanceForms Choreography’ are commercial software packages with tools for editing motion captured data.
6 From: http://dictionary.reference.
9 It is beyond the scope of this writing to fully discuss Laban’s movement theories. I therefore refer the reader for further information to the numerous books that have been publicized about Laban.
research on Ambisonics is done at the Universities of York, see: http://www.york.ac.uk/inst/
11 ‘Cartesian coordinates,
also called rectangular coordinates, provide a method of rendering graphs
and indicating the positions of points on a two-dimensional (2D) surface
or in three-dimensional (3D) space’.
12 From Ambisonic Surround Sound FAQ, see: http://members.tripod.com/
13 For more information on this subject see my PhD thesis (Wijnans 2010).
14 The creation of a choreographic scale, much comparable to a musical scale, to realize a real time transformation of dance movements into a 3D spatial digital sound composition has been published in an earlier paper: ‘A Choreography of a Spatial Sonic Disembodiment, Development of the Three Dimensional Data Interpreting Methodology’ (Wijnans, 2009).
15 See the visual documentation in
chapter 6 of my PhD research: http://www.mudanx.nl/PhD/6%
16 A further philosophical discussion is beyond the scope of this writing. For more information on the subject ‘disembodiment’ I refer the reader to Merleau-Ponty (1962) or Ajana (2005).
17 From: http://www.britannica.com/
18 See: http://www.michele-noiret.be/
20 Please note that the spatial sound in these environments is applied as a horizontal (2D) moving element with the speakers set up horizontally around the audience.
21 I originally proposed the term ‘TranSoniscendence’, but I am grateful to Dr Sher Doruff who advised me to change the name in the simpler term ‘TranSonic’ perception.
Ajana, B. (2005) ‘Disembodiment and Cyberspace: A Phenomenological Approach’. ElectronicJournal of Sociology, 7.
Birringer, J. (1998) Media and Performance along the Border. London: John Hopkins University Press.
Blauert, J. (1997) Spatial Hearing, The Psychophysics of Human Sound Localization. Revised ed. Massachusetts, USA: The MIT Press.
Cole, J. & Montero, B. (2007). ‘Affective Proprioception’. Janus Head 9: 299-317.
Duerden, R. (2005) ‘Dancing in the Imagined Space of Music’. Proceedings of Sound Moves: An International Conference on Music and Dance 2005: 26-33.
Gerzon, M.A. (1980) ‘PRACTICAL PERIPHONY: The Reproduction of Full-Sphere Sound’. Audio Engineering Society (AES) Convention. Preprint 1571.
Herman, C. (2002) ‘Embodiment: the
flesh and bones of my body’ [online] available from: http://du.ahk.nl/mijnsite/
Hofmann, J.J. (2002) ‘Towards Spatial Music’. Earshot [online] available from:
Hunt, A.M. & Kirk, R. (2000a) ‘Mapping Strategies for Musical Performance’. CD-ROM ‘Trends in Gestural Control of Music’, Wanderley, M & Battier, M. eds. Paris: Ircam, pp.382- 427.
Kirstein, L, Muriel, S. & Dyer, C. (1953) The Classic Ballet, Basic Technique and Terminology. Revised ed. New York: Longmans, Green & Co.
Laban, R. (1966) Choreutics. Annotated and edited by Ullmann, L. London: MacDonald and Evans.
Lennox, P., Myatt, A. & Vaughan, J. (1999) ‘From Surround-sound To ‘True 3-D’. AES 16th International Conference on Spatial Sound Reproduction: 126-135.
Lovell, R.E., Denney, J. & Mitchell, J.D. (1996) ‘Virtual Space Harmony: The Value of Rudolf Laban's Theory of Space Harmony as a Means to Navigate a Virtual Stage Environment (VSE)’.
Mark, G. (1997) ‘Flying Through Walls and Virtual Drunkenness: Disembodiment in Cyberspace?’ Proceedings of the 6th International IFIP Conference ‘Women, Work and
Computerization, Spinning a Web from Past to Future’: 221-223.
McKechnie, S. & Potter, M. (2005) ‘In the Air: Extracts from an Interview with Chrissie
Parrott’. In: Grove, R., Stevens, C. & McKechnie, S. eds. Thinking in Four Dimensions,
Creativity and Cognition in Contemporary Dance. Melbourne, Australia. Melbourne University Press: 95-106 [e-book 22.03.2007].
Merleau-Ponty, M. (1962) Phenomenology of Perception. London: Routledge.
Randell, C. & Muller, H.L. (2001) ‘Low Cost Indoor Positioning System’. Ubicomp 2001:
Ubiquitous Computing: 42-48.
Randell, C., MacColl, I., Muller, H. & Rogers, Y. (2002) ‘Exploring the Potential of Ultrasonic Positioning Measurement as a Research Tool’. First European Workshop on Location Based Services: 171-174.
Randell, C. & Rowe, A. ‘Come Closer: Encouraging Collaborative Behaviour in a Multimedia Environment’ (2006). Interactive Technology and Sociotechnical Systems. 12th International Conference, VSMM: 281-289.
Stiefel, V. (2002) ‘A Study of the
Choreographer/Composer Collaboration’ [online] available from: http://www.princeton.edu/~
Västfjäll, D. (2003) ‘The Subjective Sense of Presence, Emotion Recognition, and Experienced Emotions in Auditory Virtual Environments’. CyberPsychology & Behavior, 6 (2): 181-188.
Verstraete, P. (2005) ‘Interfacing Dance. Choreographing (by) gestural controls’. Sound
Moves: An International Conference on Music and Dance: 197-203.
Wijnans, S. (2009) ‘A Choreography
of a Spatial Sonic Disembodiment, Development of the Three Dimensional
Data Interpreting Methodology’. Proceedings of Sound, Sight, Space
and Play 2009: 1-16. [online available from: http://www.mti.dmu.ac.uk/
Wijnans, S. (2010) The Body as a Spatial Sound Generating Instrument, Defining the Three Dimensional Data Interpreting Methodology (3DIM). PhD. thesis. UK, Bath Spa: Creative Music Technology Department.