back printable version
 
SUE BROADHURST

Intereaction, Reaction and Performance: The human body tracking project

As a result of new technological advancements in performance practice, I am arguing that new liminal spaces exist where there is a potential for a reconfiguration of creativity and experimentation in performance practice.

Liminality, from limen (Latin: literally threshold) is a term most notably linked to Victor Turner who writes of a no-man's-land betwixt-and-between, a site of a 'fructile chaos ... a storehouse of possibilities, not by any means a random assemblage but a striving after new forms ' (Turner, 1990: 11-12). My own use of the term includes certain aesthetic features described by Turner, but emphasizes the corporeal, technological, and chthonic (Greek: back to the earth) or primordial (Broadhurst, 1999a: 12).

These spaces are liminal in as much as they are located on the 'threshold' of the physical and virtual. As a result, tensions exist within the spaces created by new technological art practices, such as, motion capture and artificial intelligence. Since no body not even a naked body escapes (re)presentation altogether (Broadhurst, 1999a: 103), the virtual body (as any other body) inscribes its presence and absence in the very act of its performance leaving gaps and spaces within its wake. I suggest it is within these tension filled spaces that opportunities arise for new experimental forms and practices.

Within my current research I explore performance practices which explode the margins between the physical and virtual and what is seen as dominant traditional art practices and innovative technical experimentation. This involves collaboration with Richard Bowden, a systems engineer from the Vision and Virtual Reality group at Brunel. The VVR group research into methods to allow both humans and objects to be located and tracked seamlessly and in real time. The applications of this technology range from Visual Surveillance to Virtual Reality of the media and production industries. For instance, constructing a smart room that can anticipate the needs of the user and the creation of tele-virtual newsreaders.

What I was particularly interested in, initially, for my own practice was markerless motion capture which allows the 3D pose of an individual and its motions to be extracted without the need for sensors, calibration or preparation. Magnetic or optical motion capture has been used widely in performance and art practices for some time now. This involves the application of sensors or markers to the performer or artist’s body. The movement of the body is captured and the resulting skeleton has animation applied to it. This data projected image or avatar (Hindu: manifestation of a deity or spirit) then becomes some part of a performance or art practice. For instance, in the most recent work of Merce Cunningham, Biped, an absolutely stunning but ultimately distancing performance, pre-recorded dancing avatars were rear projected onto a translucent screen giving the effect of a direct interface between the physical and virtual bodies.

However, the use of markers and the accompanying technology cause quite a few restrictions to the performer’s movements, for instance, mainly having to remain within certain spatial constraints. There is also the need for extra preparation and calibration and if the physical and virtual interface is in real time then aesthetically there are some problems. So with the development of markerless motion capture, these restrictions are removed. This particular research is still being developed.

Our main collaborative project that we are currently working on, ‘Interaction, Reaction and Performance’, involves direct interaction between an avatar and a physical performer. The physical work was initially a text and movement based piece which was performed at Brunel University in 2000. The text titled Blue Bloodshot Flowers and written by Phil Stanier involves the remembrance of a love affair. There is some ambiguity on whether it is between two adults or an adult and a child. Also if the narrator is dead – the ex lover is obviously long gone. The performer, Elodie Berland, is French and we used a French voiceover as a memory device in the initial performance with good effect. When we decided to combine the piece with interactive technology we initially wanted an avatar which would be female and perhaps a child to represent the child of the affair or the inner child. However, this all seemed too literal and when we saw Jeremiah we immediately wanted him in the performance and decided to leave it to the audience on how they would interpret this virtual presence. Though, of course, most people would assume it was the image of the departed lover.

Jeremiah is a computer generated animated head based upon Geoface technology. He has a simple bone structure which allows him to express himself and emotions with which he can become angry or sad. But most importantly he has eyes with which he can see. He doesn’t only interact but also reacts. In fact he possesses artificial intelligence to the degree that he can demonstrate several emotions as a reaction to visual stimulus. Jeremiah is unique in that he embodies intelligence that is no way prescriptive. Therefore, the performance is a direct interaction between performer or audience and technology.

One of the most interesting aspects of this project is how much the spectator projects into the avatar. Jeremiah, as we know, is computer technology programmed with some artificial intelligence and has the ability to track humans or objects. However, the interaction with him is anything but objective. Most people when they first see Jeremiah find him fairly spooky. After the initial contact people tend to treat him in the manner of a small child or a family pet and behave accordingly. Usually trying to make him smile and generally please him. His face becomes so sad when he is left alone that it is becomes quite difficult to walk away. Although he is programmed with emotions to react to certain stimuli, he can demonstrate fairly random behavior that can be fairly disruptive during a performance adding a further dimension to experience of working with a virtual body.

‘Interaction, Reaction and Performance’ is a pilot scheme for future projects. And it is a feasibility study in as much as if this is successful we would want to develop the technology even further. At the moment we are discussing the introduction of hearing and speaking to the avatar and another head is being constructed which can morph between male and female. Though, we are all very reluctant to lose Jeremiah at least in this project.

From a technological perspective (Bowden, 2001), Jeremiah is based around two subsystems; a graphics system which constitutes the head and a vision system which allows him to see. There is also built in a simple emotion engine which allows him to respond to visual stimuli via expressions or emotions. We are currently working on a new head model called Rachel (see above) who is a deformable eigen head providing more lifelike appearance and motion.

The system is capable of running on a single PC but for speed of operation each subsystem runs on a dedicated PC connected via a network crossover and is, therefore, self standing and also now, due to the construction of a flight case, truly portable.

The head is based upon the Geoface articulated bone model with prescripted expressions for key emotions. It contains a simple Newtonian model of motion with random elements of movement and random blinking and ambient motion.

The vision system is based around a per pixel Gausian mixture model of colour distributions, using expectation maximization within the Grimson tracker framework with additional shadow suppression and noise removal algorithms. This allows static background scenes to be learnt dynamically providing a robust segmentation of foreground objects. Jeremiah’s attention is randomly distributed between these objects weighted by their size and motion. Co-ordinates of objects within the field view are sent to the head model for animation.

The emotion engine determines the current state of emotions from simple parameters extracted from objects of interest within the visual field. This simple set of rules allows chaotic behaviour in a similar fashion. For instance, Jeremiah likes visual stimulus – high rates of movement make him happy. He likes company – no stimulus makes him sad. He doesn’t like surprises – high rates of change in the size of objects make him surprised. Jeremiah doesn’t like to be ignored – if objects exist but don’t move then he assumes they are ignoring him and hence gets angry. Also, if Jeremiah experiences too much pleasure due to too much of any particular stimulus, he will get bored and reduce its influence on him.

Since this research project is a science and art collaboration, there are marked difference in the research rationale and questions.

For Richard, the Turing test describes a system as artificially intelligent if a human user cannot distinguish the system from another human in conversation. He is attempting to test this concept of intelligence by providing an interactive human avatar with simple rules and chaotic behaviour. Richard believes the interactivity and human embodiment of Jeremiah is sufficient that individuals see him as a living entity. Therefore, Richard’s foremost question is ‘How real can Artificial Life become? How do we interact with A' Life?’ (Bowden and Broadhurst, 2001).

I, on the other hand, am interested in a more arts related research rationale and questions. Since, as I have argued elsewhere, language without the body does not ‘mean’ at all, as corporeality provides language with meaning under socio-cultural and thus temporal constraints (1999b: 17), what then are the implications for a virtual body? Therefore my overall research question is:

Due to such developments as artificial intelligence and motion capture becoming increasingly prominent in art practices, does this interface between the physical and virtual body give rise to a new aesthetics? What then are the theoretical and practical implications of this?

My aim is to explore and analyze the effect these new technologies have on the physical body in performance. Especially in relation to the problem of re) presenting the 'unrepresentable', that is the sublime of the physical/virtual interface (or liminal space). Underpinning this is a series of specific research questions:

  • What are the effects of new technologies in the analysis of the performing body?
  • What are the theoretical implications of virtual performance for the body and space?
  • What are the implications of, and how do we theorise, the resultant de-stabilisation of identity and origin?
  • What is the potential for participation and interactivity, inter-performer and spectatorship, within this new art practice?

In conclusion, this is an ongoing project with the first public performance to be staged at the 291 Gallery, London in August of this year and a private performance to be presented at the Media Lab, MIT later in the year. The performance process has proved extremely stimulating and in some ways may prove ultimately more beneficial for research purposes than the actual finished product.

Finally, although much interest is directed toward such new technologies as Jeremiah, it is my belief that technology’s most important contribution to art practices is the enhancement and reconfiguration of an aesthetic creative potential which consists of the interaction and reaction with a physical body, not an abandonment of that body. For, it is within these tension filled liminal spaces of physical and virtual interface that opportunities arise for new experimental forms and practices.

REFERENCES

Bowden, Richard. ‘The Human Tracking Project’ (Unpublished Paper), 2001

Bowden, Richard and Broadhurst, Susan. Interaction, Reaction and Performance URL: http://people.brunel.ac.uk/~pfstssb/, 2001.

Broadhurst, Susan. Liminal Acts: A Critical Overview of Contemporary Performance and Theory, London: Cassell/New York: Continuum, 1999a.

---- 'The (Im)mediate Body: A Transvaluation of Corporeality', Body & Society 5(1), March 1999b: 19-27.

---- (Dir). Blue Bloodshot Flowers. Brunel University, June 2001.

Merce Cunningham Dance Company. Biped. Barbican, London, November 2000.

Stanier, Philip. ‘Blue Bloodshot Flowers: Text for Performance’. Body, Space, & Technology www.brunel.ac.uk/bst (Vol.1 No.2), 2002

Turner, Victor 'Are there universals of performance in myth, ritual, and drama?’, pp. 1-18 in Richard Schechner and Willa Appel (eds) By Means of Performance. Cambridge: Cambridge University Press, 1990.


Blue Bloodshot Flowers:
Text for Performance.

PHILIP STANIER

1.
I remember everything all too well. I am speaking of departure, severance. As we were wedged together at the knees so frequently so hard. There was little dividing us, so, severance. This is departure again. Another split. Doubled over. We were broken down and drained, he more than I. We parted once he left me laid out in the flowers and I loved him for it. This is his corpse what I now make with my tongue, these are his remains raked through, ploughed up, and cried over. His breath rattled, mine he called quiet algebra. He could only make out my face this close enough to see my lack of lines, to see the difference between the two of us between the mushroom soft escarpments underneath his eyes and the drum tight lids of my eyes, but too close to make out the whole. He could never see me at a distance. It was always too close. He would never know I was on the horizon to call and retrieve me, too close or nowhere at all. This is my dismissal of him, my move to the horizon. His redemption. I say this in the cold. Anger has no place here. Any emotion would be synthetic. This is nostalgia gone rotten like my insides all leather and dust. I am speaking of a glint in his eye that was unsuitably framed. A context that made what happened unpleasant to other eyes, the speaking of it uncomfortable, it made peoples skin crawl. I am speaking of an attempt to gain distance, from a fatal event. I won’t be able to repeat this enough. I had him inside me, I had him deep inside me, So deep in my heart, he is still really a part of me, I had him inside me. I won’t be able to repeat this enough. It was natural for us both, so much for childhood reasoning. He thought more childishly than I. No style or rhetoric of thought. Just patterns in actions. The dispersal of flower seeds in the air, the spiral of geometry in nature. /Hold hands he said we did. A walk he said we did. Your smile he said I did. I’m lost he said I was. I’ve fallen he said we had. No more he whispered I left. That’s it all, how it began how it ended. It could have seemed so much like chance, coincidence. But this is no eternal story, there is neither chance nor anger at work here. Just the pedestrian fear of an airtight alien logic. I have to confess there were more complicated requests later the arrangements of bodies and movement with numbers and names but it was the same uncomplicated thought. /Kiss me he said I did, lick me he said I did, suck me he said I did, fuck me he said I did. Simple for the mucous mind. Thick and slow but liquid and impossible to damage. Inevitable. We had already begun before he spoke as much as he spoke before we began. Like the snake that eats itself tail to head the vanishing circle chicken before egg before chicken before cart before horse head over heels. He sent me spinning, reeling occasionally with the back of his hand. I have too tell you that, it is my duty to tell you that. I did not live too many days after that. But I can’t repeat this enough. I tried so not to fall in, I told myself this could never end well, how could I resist, when I knew so well that I had you inside me. But I can’t repeat this enough.

2.
He marked me out, and mapped my surface. His hands moving over my skin, fingertips rough like tiny moving cancers stretching out, trying to leap his broken down frame into mine. There was a calm arithmetic in his palm. Strange how the body in repeating itself cell by cell, breeding, doubling, strange how these little clones can reject the persistence of the pattern towards age. Set up their own little colony. I’m sorry I’ve lost you, my metaphor was cancer as a symptom of a fear of death, that simply hastens it, hastens death that is, the result of a paranoid body. Was he my cancer. I was his, he was more afraid of death. while he was with me, I reminded him of his age and we wore each other out, but while I fed and grew there was less of him each time. The skin on his hand hung bare like a glove, each day the bones were thinner. I was against his skin but there was no touch, the surface was cold. His breath rattled more. We walked. Our daily average 5 miles I was only a child, I would get tired. But those walks led to calm, great calm in the time of sleeping or half sleep with him, when it was twinned with my exhaustion after he wore himself out on me in the shelter of the forest. The weather was not often kind, we walked in bent double downpours, the result of overzealous butterflies pollinating orchids in Brazil. The pelting downpour covered several darkening equators. Windless day and night. Through fields of a million blue bloodshot flowers. Battered shattered by the rain, broken glass fault-lines etched in the petals. Sap and mud like blood underfoot, always cold. Wet hair in thick coils, fingers running through. As I went down on him. Those that went like worms through blades of grass. And the wet ground cradled us and we were wedged together at the knees snapping buttercup stems. Eating the flowers, despite the blankets there were too few for sustenance. I starved with a full belly. Now I walk my daily average 1000 miles internally. I have returned to zero so many times, I lost track the distance covered, I deal in speed and daily averages. I don’t know when he was ever silent or still. Always fumbling for intimacy. Always fumbling the anatomy of my disgrace. The other things future quiet crying and empty returning searches could not have been known in view of time and shade. Everything was too close to see the horizon. I won’t repeat this enough. Cause I’ve got you deep inside me. I won’t repeat this enough

3.
When it happens the echo was noticeable in my skull. Ringing. When he took me with delayed communication. When he panicked. When he shook my head with that rock. He sometimes seems weak and distant, though he was strong that time. The sweat off his body stank of his age and ran down the crevasses in his skin. Mine still made its own random path across marble, and smelt only of salt. But I didn’t live so many days after that. The violence of the idea had germinated the first time, but now it blossomed. Blue green blooms on my skin, red roots from my nose, mouth and between my legs. Pretty flowers from my skin. My body aged no more. My skin did not let go of the muscle, the muscle did not let go of the bones, the bones did not let go of each other. At least not through any living process. I did not invite decay, it made itself a guest. The glint in my eye suitably framed by thin dry skin tight on the near hollow socket. I did at the very least welcome that on the inside where time and his old body were laid bare and raw. On the inside where neither time nor his old body hung bare. Please note that when I said he hit me I lied at least not before he brushed my hair with earth and granite. Please note when I said we ate flowers I lied though I chewed them with my face in the ground. Please note when I said that there were a million blue bloodshot flowers I guessed there could have been more there could have been less. I’ve got you inside me. I’ve lost you inside me. I’ve left you inside me. I’ve kept you inside me. I remember everything all too well and so I am speaking of severance, departure. I can’t repeat this enough. My fumbling progression through counting weeds those unwanted flowers keeps me occupied. I am in the earth. I am roots and creepers. I am inside you.