Three Interactive Scenes of The Crystal Cabinet
Claudine Ulrich, Jan Vesala, Johan Chistenssen, Kristina Hansson and John Erik Eleby
Choreography is a field where corporealities are at play. Corporeality does not simply depend on the materiality of the body, but rather on the imagination that stimulates the sensations. Choreography and digital technology create a space for experimentation where new corporealities can be produced and explored in multiple ways. This article will focus on the use of interactive performance technology to produce the three scenic subjects of the stage performance The Crystal Cabinet (2008) that also constitute the first part in my artistic research project exploring multistable bodies and shifting corporealities in the encounter of choreography and technology. The idea was initiated by the composer, the choreographer and the video designers, and proposed to Piteå Chamber Opera, who produced the performance that toured in seven cities in the north of Sweden. This performance took the form of a dream play opera in twelve scenes, including texts and images from William Blake’s (1757-1827) illuminated books (2000) performed by twelve musicians, three opera singers and two dancers connected to moving computer animations and interactive sound technology.1 The libretto, developed in various kinds of collaboration between the composer, the director and the choreographer, shifts between two different kinds of scenes: a fictional frame story about William Blake and ambiguous visits to his illuminated poems. While the frame story scenes are composed and staged in a relatively traditional way, the in-between passages use the connection of choreography and interactional staging techniques to transform the characters to become ambiguous creatures influenced by the ones found in Blake’s images and poems. To create his books of compound text and image, Blake invented a printing-machine with which he could print his handwritten texts and pictures. After printing, he coloured the pages by hand. We transformed this idea into an interactive area where the connections among bodies, technology, voices and words work in several directions. To explore the singular corporeal experiences found in his metaphorical poems and images, the performers in those passages leave their frame story characters to become human-machine subjects transformed, deconstructed and elaborated by the rhythms and dynamics of a virtual voice reading Blake’s texts. Throughout the performance computer-animated versions of his images and words move across the stage, and in the vision scenes the acoustic of the room is transformed via microphones connected to a computer. The title of the opera is borrowed from one of Blake’s poems, and the concept ‘the crystal cabinet’ also became our name for the shining globe that the performers pass through, in order to reach the interactive area.
As the choreographer and part of the libretto-team of The Crystal Cabinet, my artistic concerns in the interactive scenes lay in the development of scenic subjects that challenge traditional ways of developing opera characters from psychological identification and interpretation of a pre-conceived role. When working out the crystal cabinet scenes, the visits in to Blake’s poems, we decided to connect the performers’ movements to a computer that processes the words, rhythms and dynamics of a play-backed voice reading his texts. The rhythms and dynamics of the voice are at the same time circled back to the performers’ performance of gestures and poses generated from Blake’s poems and images. Beyond the notion of technology as a tool for command and control we wanted to use it as a way to acquire new rhythms and discover new kinetic possibilities in a non-hierarchic connection of body, movement and text.
This article will focus on how unthought-of effects, new rhythms and scenic subjects can be derived from the various body-movement-word-technology connections in the interactive scenes of The Crystal Cabinet. Rather than being a tool, for imitation and extension of already known human powers, the technology of these scenes was developed to interrupt habitual linear ways of performing text and choreographed movement. Some of the areas that will be explored concern how the integration and/or disturbance of technology generate new choreographic methods and artistic concepts. Below, a presentation of body, space and technology concepts will be followed by a performance overview that leads to a more detailed description and discussion of the three interactive scenes.
Body, technology and space concepts
According to Elisabeth Grosz, the human body must be seen as a series of processes of becoming rather than as a fixed state of being (1994, 11-12). Its unity is not a function of its machinal operations as a closed system but arises from a sustained sequence of states in a unified plurality. The body cannot be definitively known since it is not identical with itself across time.
Animate bodies, including
humans, stretch and extend the notion of physicality that dominates
the physical sciences. Grosz describes the body as a sign system subject
to an endless rewriting and inscription. Human bodies are the centers
of perspective, insight, reflection, desire and agency. To avoid many
of the common metaphors that have been used to describe the interaction
of mind and body she proposes the model of a Möbius-strip as a way
to re-think the relations between surface and depth. The Möbius-strip
is an inverted three-dimensional eight where surface become depth in
a continuous outside-in and inside-out process (1994, xi-xii). I will
argue that the depth of the scenic subjects of the interactive scenes
is produced in such a continuous outside-in and inside-out process.
Rather than being developed from a psychological identification with
or the performers interpretation of a preconceived character described
in a libretto, these scenic subjects are
produced in the moment of performance as an effect of choreography-body-technology-
The sensory device in the three interactive scenes consists of a digital camera hanging from the ceiling ”seeing” the body moving underneath. The camera is connected to a computer programme where the video signals are transformed into MIDI-messages.2 In our Max/MSP-application,3 the programme is configured to extract objects in the video image – the colour, brightness and size of chosen objects differentiated from the background. Several objects can be traced at the same time and their position is determined in relation to chosen kinds of ”hot-zones”. The hot-zones are demarcated and grouped by dividing lines, drawn by the user. The object and the hot-zones can interact in three different ways; the zone sees the appearance of a new object, the zone sees an ”old” object move or the zone sees the object disappear. The MIDI-messages then can be programmed to have an influence on qualities and events in the playback voice, music or sound. Technology can’t perceive or experience human bodies as such, as distinguished from a background or non-human bodies. It has no experience of human bodies as different from non-humans if the colours are the same. For the computer our bodies are of the same kind as non-human bodies – we are all objects. What we easily perceive as one body, the programme may register as several bodies if the colours of the body parts are different. The programme is not able to read the body as one if we haven’t calibrated the sensitivity so that all the actual nuances are grouped into one defined object.
The technological definition of the body as one object is constituted of a great number of nearby pixels in the image. There must be a human to make this grouping. The digital camera captures the visual world as a fragmented multiplicity of pixels, each of the same value. The computer can’t tell the object from the background if they are of the same colour. When some spot on the floor suddenly reflects the light in the same colour as the dancer, the spot and the dancer become the same object from the programme’s point of view. To make the programme see humans as separated from the environment we have to define the difference as a difference in colour or brightness. The contrast between the depicted colour of the body (or costume) and the floor had to be as sharp as possible to make the interface work. For example, when a dazzling light falls from above, the colour of the body, seen from the camera, becomes almost white, and the programme can’t separate the body from the floor. From the computer’s point of view the object body has disappeared, even though the dancer’s body is still there. For the computer the dancer’s body has become another body – when the reflected colour is changed. When developing and rehearsing the interactive scenes the performers had to be aware of the object body perspective of the technology which meant that they had to hear their spatial position in the responding computer processed voice rather than thinking of it as a certain spot on the floor.
Space is not the empty container surrounding physical objects that one usually thinks of as space: it is a virtual plenum of intensities that unfold actualised, extensive material entities (Deleuze 1993, 3). The fold is an ontology of becoming, multiplicity and differentiation while maintaining continuity. The multiple is not only what has many parts but also what is folded in many ways.
The interactive area could be described as a space divided into folds of potential words, rhythms and movements, distributed all over the stage. It is a space where the performance of the choreographed movements and Blake’s text become deterritorialized – non-linear and infinite. The space of folded words and movement creates a situation where a continuous and reversible transformation of audible and bodily rhythms can occur. A folding across choreography and text to create uncertain boundaries, instead of defined boundaries of separation. These uncertainties constitute a flow from the performance of the choreography to the performance of the text, where neither is fixed but rather in constant exchange. It is not the line that is between two points, but the point that is at the intersection of several lines. In the crystal cabinet passages the spatial position of the performer points to the intersections of choreography and text. The lines of choreography intersect with lines of words so that the boundaries of the body and the voice become blurred. Movements and rhythms pass forth and back between the performer and the digital voice, creating a field of sensations, intensities and affects – a torsion around which a new subject is generated. Suddenly the body-space-technology-voice figure starts to “live” on its own, producing a scenic subject of continuously varying kinetic-textual connections. Rather than emphasizing the digital outcome of the interaction, this article aims to highlight how the body-space-voice connection through technology unfolds new rhythms in the performers’ bodies and their performance of movements.
The Crystal Cabinet
In The Crystal Cabinet, the poet, graphic artist and inventor William Blake has been transferred to a time close to our own. The frame story scenes take place in the physical world, while the in-between passages stage visits to Blake’s illuminated poems. Throughout the twelve scenes the audience follow Angela when she exposes herself to an experiment that aims to find out if humanity on a voluntary basis could give up the material body and enter a virtual one, in order to save the planet. On the surface of earth nobody can any longer live. Down in a subterranean laboratory Master Blake works on his innovative printing machine. In the first scene his assistant enters the machine to take part in testing, tuning and adjusting it. Throughout this first interactive passage the audience follows the Assistant when he loses control of his body to slowly transform into shapes and expressions generated from Blake’s printings. Suddenly two representatives from a political commission, who have heard about the powers of this machine, enter the laboratory hoping to make use of it as a drastic solution to the acute global need to reduce the bodily consumption of energy. When Angela, who has volunteered to test his machine, later enters the crystal cabinet she finds herself transformed in to different characters of Blake’s visionary poems. When she returns to the laboratory her arm has gone numb. As a side effect of her recurrent visits in the cabinet, Angela by degrees loses contact with her physical body, and in her last visit Angela hybridizes with the crystal cabinet, so that her body and the machine-voice become amalgamated. From this condition she cannot any longer return to her earlier bodyliness. Even though Blake expresses his doubts about his machine, he in the third interactive passage enters the machine to grasp the words of a new poem from the air. Angela happily rests in the crystal cabinet.
While the movements of the first interactive “solo” were choreographed from the expressive body poses found in Blake’s images, the gestures in the other two scenes were generated from the words and expression in his texts. Just as the poems are connected to images in his books, the words of the digital voice and the movements on stage interfere with each other in a way that deconstructs them both. The interfaces and interaction rules of the three scenes are framed so that the influences work in several directions. In the following three examples I will elaborate on how bodies are moving words and how words are moving bodies throughout the interactive scenes.
The Assistant: Jan Vesala
When the overture has died away, the Assistant steps out of the crystal cabinet towards Blake. The touch of his fourth step starts the machine, which makes a virtual voice read the first word One, in Blake’s text:4
With his left foot centre stage the dancer, Jan Vesala, slowly rotates down to the floor where he suddenly extends his gaze and hand to the ceiling to grasp the next word. A rectangular floor-surface lights up, defining the interactive area. In a drawn out movement the hand seems to pull the voice down to the floor, where he, by taking hand steps, word by word, moves towards the audience. Using the body as a ruler and his limbs as a pair of compasses, the dancer moves across the rectangle, like a geometrician measuring the world. Throughout the solo the voice follows his movements, and on the wall behind him Blake’s critical portrait of Isaac Newton measuring his universe, piles up, dissolves and disappears. In the computer the video artists, Lene Juhl and Mark Viktov, made the image of Newton’s body and universe elastic and stretchable. Absorbed in thoughts Master Blake sits on his stool in the front corner. At the beginning of this scene, the dancer seems to control the virtual voice but throughout the solo the relation become more and more ambivalent. Gradually the dancer abandons control of his body, letting it float around, shaped by the fluctuating density heard in the varying velocity of the virtual voice. Suddenly he awakes to become Blake’s assistant. He gets a sight of his employer and advances across the space to grasp the word in the left front corner. Blake parallels The Assistant’s movements and again they seem to control the voice together. Blake falls back to his stool, and by degrees the voice seems to absorb the dancer so that his body cracks, undermined by the extremely slow pronunciation of the words. The sound of the steps of the commission coming down the stairs interrupts the session. Blake and his assistant hasten to get prepared to meet them.
In the computer program, the recorded voice reading Blake’s text5 intertwines with one of the crystal sounds.6 The degree of convolution as well as the velocity of the played back voice and the crystal sound depends on the dancer’s spatial position underneath the camera serving as a sensor. When he moves close to the audience, the words can be heard clearly. Further back, the voice begins to slow down, and at the back of the stage the reading become so drawn out that the words become impossible to perceive. The voice turns into a dark rumble so that the words fall apart like a wall cracking in slow motion. To stage The Assistant’s process of reading and learning Blake’s poem, we designed an elastic space where the fluctuating velocity of the sound makes the words of voice come easily in the front, while in the back they are almost impossible to draw out from the machine. The idea was to make the air, through which the dancer moved, seem transparent in the front – sluggish, heavy and almost impassable in the back. The interaction concept was to create a space where the density seems to differ in various places – a fluctuating density heard in the variation of voice’s velocity and seen in the dancer’s movements. To follow the performer’s continuous motion across the room, the technology was programmed to register the body’s position in a two-dimensional area where the width and depth constitute the x- and y-axes of a co-ordinate system. While the dancer’s in-depth movements influenced the pitch and velocity of the virtual voice, his sideways movements had an effect on the crystal sound.
The human part of the interaction consists of a choreographed series of movements performed by the dancer, Jan Vesala, who at the same time lets his performance of them be affected by the transformations of voice following his movements. The series of movements were developed by the dancer from a movement score of body poses that I chose from Blake’s images. To shape his moves across the floor I drew a motion map, which also defined the approximate time for each transition and stay. The motion map with given times was also followed by the video artists when they made the digital deformations of Blake’s Newton image projected behind the dancer. When Vesala throughout this solo transforms his body from pose to pose, I wanted his body to move like a computer programme morphing between pictures. At the same time his moves make the voice morph between the words in a similar way. When reaching each body pose he was asked to accentuate the pose as if bumping in slow motion – like in a room of a continuously shifting gravity. The physical feeling of how to bump into each body pose, he captured from the voice, while moving.
Master Blake: John Erik Eleby
In the second interactive scene the interactive area becomes a poetry-machine in which Blake, impersonated by the opera singer John Erik Eleby, captures the words and images of a new poem. The character William Blake walks across the room grasping the words of a new poem with his gestures.7
To see the world in a grain of sand
This time the interactive area consists of a grid of expressive words and sounds. The grid is divided into 15 squares and when the performer enters them the machine responds with the words or sound of that square.
A – Hold infinity
The voice, reading the words, has been acoustically transformed so that they seem to be born out of this poetry machine, and the sounds are crystal sounds from earlier scenes in the opera. The performer’s movements consists of a series of choreographed gestures derived from the expressions found in the text fragments, and the interaction rule says that, when hearing a word, he has to respond with the gesture corresponding to that specific expression. This momentary voice-word-gesture reaction is similar to the simultaneous ear-hand relation that we experience when hearing someone speak, and we find ourselves able to write the words almost at the same time. When reaching a crystal sound his body reacts more spontaneously to the memory of the scene where the sound was first heard. The words also write themselves on the back wall, while he moves between the squares. In this scene the idea was to use the technology as a contemporary version of Blake’s printing machine – a body-voice-machine that pronounce and perform the words and images of his poems.
Angela: Claudine Ulrich
In Angela’s last visit to the crystal cabinet, the dancer Claudine Ulrich interacts with a virtual version of her own voice.8 In this passage I wanted the dancer to seem to rub and loop the words in her limbs and joints, getting in contact with each letter of the text, phoneme of the voice and vice versa. The intention was to frame a scene where the body and the machine become so intimate that you couldn’t tell the one from the other – the (e)motions of the one from the (e)motions of the other: desire, despair, resignation, resistance and delight. Through small changes of position she could move between the words, and while staying in one place she was able to scratch and itch the vowels and consonants of the voice. The dancer’s movements and the words of the voice became crystallized and repeated in a similar way; chopped into pieces and looped until the performer moved to another place – the place of another word and another gesture.
I dreamt a dream, what can it mean?
When the scene starts Angela is left alone on stage. The interactive area lights up underneath her and the machine voice slowly starts to move her body, saying; delight, delight, my only delight. The words float into her torso to become meandering motions transmitted into the limbs. She then searches around the room to be captured by other words: rose, turned away and jealousy. The further back she moves, the more chopped and stuttering the voice becomes; thorns, thorn, orn-orn-or-or-or-r-r-r-ns. The crystallizing of the text into pieces of words is transformed into a similar mincing of her movements and in the back of the stage the shivering voice makes her whole body tremble. Close to the audience the words become longer phrases and for a while her body stays in a loop saying: What can it mean? What can it mean? She decides to move up stage left and in that corner the first words of the poem I-I-I dr-dream-eam-eamt wash over Angela and the trembling movements of the voice spread through her body. From this position she continues her shaky ”reading” of the entire text by moving sideways across the room. Filled by delight, she gives herself into this body-word erotic rubbing condition, and stays in the cabinet, physically paralysed.
Tiny text fragments in the back
In this scene the position of the dancers object body is directly connected to each word of the voice. Her spatial position could be described as a needle pointing to each word or even letter of the text. By moving through space she opens “windows” of the text and when she stops, the voice continuously repeats the word – or phoneme – over and over again. To the left, seen from the audience, she opens a window reading the first word I, and to the right the last word delight is heard. If she moves continuously from the left to the right, a linear reading of the entire text can be performed by the voice. On the depth axis, her position defines the length of the loop. Close to the audience the loops consist of several words each, while in the back she opens smaller and smaller windows looping single words, parts of words or just one phoneme. In the back, at the edge of the camera eye, also the sounds of the silences in between the words can be looped.
Once again the dancer’s movements are choreographed from a text, and consist of a series of gestures and bodily expressions derived from Blake’s words and phrases. The interaction rule says that, when hearing the repeated loop of a word or part of a word, she has to oscillate the rhythm of the gesture fragment corresponding to that word. The scene was worked out so that she moved back and forth through the text, jumped between the words and read the poem in a non-linear way – as decomposed object words. This fragmentized reading of Blake’s poem was mirrored in the dancer’s immediate shifts between gestures connected to different words. When the machine reads entire words and phrases, the dancer performs the gestures in “normal speaking” rhythms. In other positions, the gestures and words are minced into grains of sound and motion so that the semantic understanding of them crumbles away. I wanted her body to connect to the sounds of the voice on a micro-level – a level where the rhythms of the voice and the body intermingle in a mutual body-space interface.
Interactive performance technology creates a space where bodies and movement are at play. The interfaces between choreography and technology and the interaction rules we elaborated to interrupt habitual linear ways of performing choreography and poetry – creating situations that acquire new rhythms and unthought-of effects in the performers movement. In these interactive scenes the performers have to deal with at least three different kinds and layers of movement – three corporealities approaching the body in different ways. The choreographed gestures constitute a first movement layer that has been rehearsed and incorporated so that the performers enact them almost as a conditioned reflex when hearing a word or phrase. The second kind of movement consists of the rhythms and dynamics of the voice sweeping through their limbs as they perform the gestures. These rhythms arise as another movement layer, as an improvised response to the rhythms heard in the voice. The third kind of movement is the performers’ moves of the entire body across the stage, moves that in a radical way also change the rhythms of the voice. This interference of three kinds of movement in one and the same body was something that in the beginning of the rehearsal period was confusing and disturbing for the performers to deal with. But after a while the situation became more familiar so that they know how to shift between the different states of mind/body. In Angela’s scene it was especially hard to shift into the third kind of movement, to make the decision to move to another place when being totally occupied by the rhythms of the current gesture. The dancer therefore had to learn how to make a mind/body shift to move her feet across the stage while at the same time continue her rhythmic elaboration of the gestures in the upper body.
Regarding space as a plenum the interactive area provides a space of enfolded poetry and movement where the smallest elements of the pre-recorded voice and the choreographed gestures constitute a multiplicity of folds within folds. This body/movement/space/technology interface makes the potential of a folding and unfolding of movement and rhythm where neither the performance of the text or the choreography is fixed but rather in constant exchange. Due to the performer’s spatial position underneath the camera, the body points to various intersections of choreography and text. At each point the lines of the words and the lines of the choreography also intersect in the dancer’s body. This folding across lines creates an uncertainty between body, voice and space. The interactive potential constitutes a prism that continuously crystallizes new rhythms in the voice and in the body due to the light reflected from the performer. As an effect, the interaction produces continuously varying scenic subjects – crystal cabinet subjects of fluctuating body, voice, space and technology connections.
In this article I have focused on the use of interactive performance technology to produce three scenic subjects of The Crystal Cabinet. Rather than regarding the elaborations of the digital voice as a final outcome of interaction, we wanted to develop a situation where the movements and rhythms of the voice and of the bodies mutually influence and interfere with each other. Following Grosz’s model of a Möbius-strip, to describe how subjects are produced as an effect of an outside-in inside-out process, the depth of these scenic subjects are produced as an effect of rhythms, words and movements that pass through body, space and technology.
Blake, William (2000). The Complete Illuminated Books. London: Thames & Hudson.
Deleuze, Gilles (1993). The Fold: Leibnitz and the Baroque. Minneapolis: University of Minnesota Press.
Grosz, Elisabeth. (1994). Volatile Bodies: Towards a Corporeal Feminism. Bloomington: Indiana University Press.
The Crystal Cabinet, Piteå Chamber Opera, 2008. Music: Carl Unander-Scharin. Libretto (based on texts by William Blake): Carl Unander-Scharin, Keith Turnbull & Åsa Unander-Scharin. Choreography: Åsa Unander-Scharin. Video design and animations: Lene Juuhl & Mark Viktov). Director: Keith Turnbull. Angela, Dick: Claudine Ulrich (dancer). The inventor Blake, The Father: John Erik Eleby (bass). The Assistant, The snake: Jan Vesala (dancer). Chairman, The Girl, The Mother: Kristina Hansson (soprano). Secretary, The boy, Tom, The Angel: Johan Christensson (tenor). Interaction design and software application: Mateusz Herczka, Carl & Åsa Unander-Scharin, updated version by Nesa Nenad Popov, 2008. Costumes: Katarina Wiklund. Conductor: Mats Rondin. Musicians: Norrbotten NEO Duration: 80 minutes. Video recording: Johannes Oscarsson. Photographer: Staffan Nygren
Video excerpt 1 "Short overview", The Crystal Cabinet
Video excerpt 2 "Man has no Body Distinct from his Soul", The Crystal Cabinet
Video excerpt 3 "To see the World in a Grain of Sand", The Crystal Cabinet
Video excerpt 4 "I Dreamt a Dream", The Crystal Cabinet
Åsa Unander-Scharin (Luleå University of Technology/ Dep. Music, Dance and Theatre), choreographer and artistic researcher (PhD), creates choreographic installations and stage performances in collaboration with dancers, musicians, visual artists, computer programmers and robotic researchers. In 1999 The Lamentations of Orpheus, performed by an industrial robot, was awarded an honorary mention from VIDA 2.0 and in 2006 Petrushkas´ Cry received the special prize in VIDA 9.0 Madrid. In 2008 she choreographed the interactive video Rintrah Roars, The Crystal Cabinet at Piteå Chamber Opera, Desire, Chaos and Geometry for The Vietnam National Opera Ballet, and in the same year her doctoral thesis "Human mechanics and soulfull machines" was published. In November 2010 the robot bird dancer Robocygne and the giant mechatronical marionette Olimpia was premiered in her exhibition Opera Mecatronica in Stockholm.