Heightened Awareness through Simulated Movement in Multimedia Performance
In the 1960s and 1970s, American artist Bruce Nauman made installations and video works that prompt the observer to explore the awareness of her presence in relation to the object. Art critic Susan Cross suggests that the effect of these works is related to Michael Fried’s notion of the 'stage presence' of Minimalist works. The works by Nauman that are discussed in this paper are primarily based on the observation of pre-recorded videos or the active participation of the observer in the work. In response, the author suggests a way to develop Nauman’s strategy for enhancing awareness of presence by confronting the observer with the visual and aural perception of moving sounding objects in combination with simulated aural motion cues generated by means of digital sound technology. This approach is illustrated by an analysis of some aspects of his performance installation Box Piece.
Multimedia Performance, Digital Sound Synthesis, Spatial Music, Installation Art, Post-Minimalism.
BRUCE NAUMAN AND THE INFLUENCES OF MINIMALISM
A number of Bruce Nauman’s works from the 1960s and 1970s deal with the awareness of the observer’s perception of the presence of herself and objects around her. Examples of these works are Lighted Performance Box (1969), Walk with Contrapposto (1968) and Performance Corridor (1969). Art critic Susan Cross describes the experience of these works in context of the 'stage presence' of Minimalist works that Michael Fried suggests in his essay Art and Objecthood. Fried suggests that this stage presence results from the anthropomorphous qualities of Minimalist objects. He gives three reasons for this anthropomorphousness: 'First, the size of much [Minimalist] work [...] compares fairly closely with that of the human body. [...] Second, entities or beings encountered in everyday experience in terms that most closely approach the [Minimalist] ideals of the non-relational, the unitary and the wholistic are other persons. [...] And third, the apparent hollowness of most [Minimalist] work – the quality of having an inside – is almost blatantly anthropomorphic.' (Fried, 2003: 839) The anthropomorphous perception of the object will result in the observer experiencing the relation between object, the space it is in and herself. Thus the Minimalist work incorporates the subject and the space around her into the work.
Cross suggests that the works by Nauman that I mentioned above evoke a comparable experience by emphasizing the observer’s relationship to the work. Lighted Performance Box consists of a rectangular metal box that is about the size of a human being and a light that is installed inside the box (Cross, 2003: 13). The size of the box invites the observer to identify with it, whilst its shape and material are likely to give her a feeling of alienation. The top of the box is open so the light illuminates the space above the object. The light emphasizes the environment of the object and thus makes the observer aware of the object’s relationship to the space around it (which also includes the observer herself). In Walk with Contrapposto Nauman filmed himself whilst walking through a narrow corridor in a manner referring to the use of posture in classical sculpture. The contrapposto (the torso angled away from the hips) was used to create an illusion of motion. Because Nauman’s head is outside the frame, his image will be perceived as an anonymous person (Cross, 2003: 15). On the one hand, the observer encounters a familiar situation (a person walking); on the other hand, the situation is uncanny (the person cannot be recognized and the manner of walking is unfamiliar and exagerated). This will lead to a 'simultaneous recognition and misrecognition' (Cross, 2003: 17) of the situation by the observer. The body on the video will become 'an abstracted body [...] and a surrogate for the viewer' (Cross, 2003: 14), but at the same time alienates this body from the observer. Thus, Walk with Contrapposto creates a similar experience of the relationship between the work and the observer as Lighted Performance Box does.
After Walk with Contrapposto and other, similar video works such as Bouncing in the Corner, No.1 (1968), Nauman started to further integrate the observer’s presence in his works. In 1969, he exhibited the walls he used to create the corridor in Walk with Contrapposto as Performance Corridor. Instead of watching an anonymous performer on a video performing the action, the observer was now invited to perform the action herself. This is also the case in Video Corridor for San Fransisco (Come Piece) (1969). In this work, the observer sees herself on a videoscreen. However, the image has been manipulated (part of the camera is covered and the angle is changed) and the observer is filmed from behind. Therefore the observer will be disoriented when she is first confronted with her image on the video screen. She recognizes that it is her on the video and that the movements of the image on the screen are related to her own movements, but she does not immediately understand the relation between the two. This will draw the observer’s attention to focus on her presence in relation to her surroundings. In other words: this is also an experience of 'simultaneous recognition and misrecognition' that makes the observer realize her presence in relation to the object, the space around it and herself.
Mobile Observer vs. Moving Object
By exhibiting the walls used for Walk with Contrapposto as Performance Corridor, Nauman shifted his focus from letting the observer relate herself to an anonymous performer on a video to the active participation of the subject; the mobility of the observer had now become a necessity to experience the work. I suggest that an equally interesting possibility lies in a development in a different direction: instead of a mobile observer, one could take the use of moving objects as a basis for the enhanced experience of presence in a work. Rather than letting the observer perform the actions previously executed by the anonymous performer, one could substitute the anonymous performer on the video by an anthropomorphously perceived moving object in the observer’s presence and evoke the experience of 'simultaneous recognition and misrecognition' by confusing the observer's sensory perception of the behaviour of this object in space. This could be achieved by providing the observer with contradicting information on the object’s movements. This would be possible when the mentioned anthropomorphous object emits sound. When an object’s movements can be perceived both visually and aurally, it would be possible to manipulate one of these two sensory inputs with technological means in order to create the desired contradicting information on the object’s movement. This strategy will result in an experience related to the discrepancy between the performed movements and the observed movements on the video screen that the observer experiences in Video Corridor. In the remainder of this paper, I shall discuss how simulated aural motion cues can be used in this context.
Perception of Sounding Objects
It is possible to register information on the movements of an object visually and aurally. The aural perception of the position of a sounding object is mainly based on the perception of interaural differences in timing and amplitude of the sound, and differences in the perceived sound spectrum caused by spectrum transformations in the outer ear (Wightman et al., 1997: 1). However, in the case of a moving sound source, information derived from monaural perception becomes more important (Perrott et al., 1997: 289-290). When a sounding object is moving, the observer will register changes in the amplitude of the sound, changes in the perceived frequency of the signal caused by the Doppler effect, a change in the timbre of the sound caused by reflection of the sound on the (stationary) objects around it and, if there is an additional stationary sound source emitting the same sound, amplitude differences caused by changing phase differences between the two signals. It is possible to electronically simulate these acoustic changes. Thus it is possible to create aural information on movement that does not correspond with the visually perceived movements of a certain object. This strategy can be used to generate the contradicting information on the object’s movement that I mentioned above.
SOUNDING OBJECTS IN BOX PIECE
The performance installation Box Piece (Ploeger, 2009) consists of three large cardboard boxes (71 x 71 x 71 cm) (see figure 1) that are attached to the ceiling with ropes. This enables the performer to move the boxes like pendulums. The boxes are held together with parcel tape. Inside boxes 1 and 2, loudspeakers are installed. The performer, who has a microphone taped to his body, takes position in between box 2 and 3 whilst the observers stand in between box 1 and 2. The sound material of the work consists of electronic transformations of the sound that was produced by the performer covering his body with parcel tape. The performer moves box 2, which will cause a change of the perception of the emitted sound as I described above. As soon as there is movement, changing phase differences between the sound emitted by the loudspeakers in box 1 and 2 cause changes in the perceived amplitude of the sound. During the performance, the sound of the parcel tape is transformed into pitched sounds that are composed of only a limited number of sine waves. This makes the sound changes that are caused by the changing phase differences between box 1 and 2 easily perceptible.
As I showed in the beginning of this paper, anthropomorphous perception is important in the experience of 'simultaneous recognition and misrecognition' described by Cross. The anthropomorphous perception of the boxes in Box Piece is evoked in three different ways:
Figure 2: Box Piece, still from video registration of performance.
The sound material of Box Piece is processed with a program written in programming language Max/MSP. The sound of the performer’s handling of the parcel tape is registered by the microphone attached to the performer’s body. This sound is played back through the loudspeakers inside box 1 and 2, and gradually transformed into pitched sounds with a simple harmonic spectrum by means of amplitude modulation. As I mentioned above, the movements of box 2 will result in a change in the perception of the emitted sound. This change will mainly result from changing phase differences with the sound emitted by the loudspeaker in box 1, a change in overall amplitude and a change in pitch caused by the Doppler effect. In order to create aural information on movement that does not correspond with the visually perceived movements, it would be necessary to simulate these effects electronically. Electronic simulation of aural motion cues would also enable the synthesis of aural information that is related to movements of the actual object that would be impossible in practice (the movement of the object at a very high speed for example). The program designed for Box Piece generates simulated aural motion cues belonging to the aforementioned pitched sound material. Three cues are simulated: Firstly, the perceived pitch changes caused by the Doppler effect are simulated using:
Where f is the perceived frequency, vs is the speed of sound, v is the speed of the sound source, vr is the speed of the subject and f0 is the emitted frequency. Secondly, the overall change in amplitude caused by the changing distance between the observer and the loudspeaker in box 2 is simulated by:
Where p is the sound pressure and rbox2 is the distance between the sound source in box 2 and the observer. Thirdly, the amplitude differences caused by changing phase differences between the signals emitted by the loudspeakers in box 1 and 2 are simulated by changing the delay of the signal of the loudspeaker in box 2 according to
Where Dt is the delay of the signal emitted by loudspeaker 2, rbox1 and rbox2 are the distances between the observer and box 1 and box 2 respectively, and vs is the speed of sound.
In the beginning of the performance, the naked performer starts to wrap up his body with the parcel tape. After the wrapping up is completed and the sound is transformed into aforementioned pitch material, the performer starts moving box 2. Initially, the pitched sounds emitted by the loudspeakers in box 1 and 2 are not manipulated. Thus, the observers visually and aurally perceive the actual movements of box 2. The observers cannot perceive the exact nature of the performer’s action since their vision is partly blocked by the box. This facilitates the continued perception of the box’s movement as (partly) autonomous (and thus as anthropomorphous). A situation is created where it is not really clear whether it is the box’s movements that dictate the performer’s movements, or the performer that controls the movements of the box. Gradually, the emitted sound is electronically transformed in such way that the aural cues of the box's movement are simulated. Towards the end of the performance, box 2 is given an initial impulse and then left moving freely. The amplitude of the pendulum movement will be damped naturally, which will also cause the naturally occurring changes in sound material to decrease. Simultaneously, however, the intensity of simulated aural motion cues increases gradually. In effect, this means that the observer sees the movement of the object decreasing, whilst she hears an overall increase of the object’s movement. About 2 minutes before the end of the piece, box 2 is stationary, whilst simulated aural motion cues of the box moving with a very large pendulum amplitude continue to sound.
In this paper, I suggested that it is possible to develop Bruce Nauman’s strategy for enhancing awareness of presence, by confronting the observer with the simultaneous visual and aural perception of moving sounding objects. Nauman's use of contradicting visual information on the position of the observer's body or objects surrounding it, can be replaced by a juxtaposition of contradicting aural and visual information on an object's location in relation to the observer. In my performance installation Box Piece, I used this hypothesis as one of the theoretical starting points for the conception of the work. A moving sounding object in combination with computer generated aural motion cues was employed to facilitate the occurrence of contradictory visual and aural motion cues. During and after performances of the piece, it became apparent that two issues concerning the strategy described in this paper require further investigation: In addition to the aural motion cues described in this paper, aural motion cues related to interferences with sound reflections in the performance space also turned out to play an important role. Accordingly, research into the simulation of these additional motion cues has been undertaken in the Laboratory of Acoustics at the Helsinki University of Technology (Ploeger, 2008) and will be applied in future work. Furthermore, informal evaluations of performances by observers suggested that the perception of aural and visual motion cues described in this paper was somewhat reduced by the simultaneous presence of other strong visual stimuli in the work. Therefore, experimentation with performance environments with fewer additional visual stimuli will be undertaken in future work.
Finally, I would like to remark that the discussion of Box Piece in this paper is intended as an illustration of the suggested approach to moving sounding objects and should not be regarded as an overall analysis of the work. In Box Piece, the strategy discussed above was applied in conjunction with other aspects of the work, such as the possible semiotic connotations of the materials used and the perception of agency concerning the performer and the digital technologies involved.
The cardboard boxes and the parcel tape were chosen for their affordance as reminiscences of modes of industrial production, which may be considered obsolete in contemporary Western post-industrial society. Cardboard boxes, and the parcel tape that holds them together, serve as packaging material for the industrial products that are nowadays mainly produced in non-Western countries. At the same time, they are omnipresent in post-industrial office environments. In the performance, the material aspects of these potential signifiers of industrial production are thematized in a post-industrial, information technologized context by means of alternating real movement initiated by the body of the performer with the computer-aided movement simulations described in this paper.
Simulated aural motion cues were also applied in context of a strategy to thematize the perception of agency concerning digital technologies. On the one hand, the performer apparently has control over the parcel tape that he is wrapping around his body; he could decide to interrupt the action at any moment. On the other hand, however, the sound material, which can be regarded as a digital representation of the tape, evolves independently from the performer’s actions. Towards the end of the performance, simulated aural motion cues are added to this sound material, whilst the performer is not controlling the box’s movement anymore. This may lead the spectator to question the extent to which the computer program that generates the sound should be considered an autonomous agent.
Cross, Susan (2003) ’Bruce Nauman: Theaters of Experience’, pp. 13-21 in Bruce Nauman: Theaters of Experience. New York: D.A.P.
Fried, Michael (2003) 'Art and Objecthood', pp. 835-845 in Charles Harrison and Paul Wood (eds) Art in theory 1900-2000: an anthology of changing ideas. London: Blackwell.
Heider, F. and Simmel, M. (1944) ’An experimental study of apparent behavior’, American Journal of Psychology, 57: 243-259.
Perrott, David R. and Strybel, Thomas Z. (1997) 'Some Observations Regarding Motion Without Direction', pp. 275-294 in Robert G. Gilkey and Timothy R. Anderson (eds) Binaural and Spatial Hearing in Real and Virtual Environments. Mahwah NJ: Lawrence Erlbaum Associates.
Ploeger, Daniël (2008) ‘Simulating
Loudspeaker Movement in the Median Plane’, COST SID Short Term Scientific
Mission Posters, http://trac.sme-ccppd.org/SID/
Ploeger, Daniël (2009) ‘Box Piece’ (video registration of performance), http://www.vimeo.com/7708075 [accessed 30 November 2009]
Wightman, Frederic L. and Kistler, Doris J. (1997) 'Factors Affecting the Relative Salience of Sound Localization Cues' pp.1-24 in Robert G. Gilkey and Timothy R. Anderson (eds) Binaural and Spatial Hearing in Real and Virtual Environments. Mahwah NJ: Lawrence Erlbaum Associates.
Daniël Ploeger is a DPhil candidate in the School of Media, Film and Music at the University of Sussex. After initial training as a musician, he is now primarily working as a performance and installation artist and researcher. His research is focused on digital performance, sound art and media theory.