Intereaction, Reaction and Performance: The human body tracking project
As a result of new technological advancements in performance practice, I am arguing that new liminal spaces exist where there is a potential for a reconfiguration of creativity and experimentation in performance practice.
Liminality, from limen (Latin: literally threshold) is a term most notably linked to Victor Turner who writes of a no-man's-land betwixt-and-between, a site of a 'fructile chaos ... a storehouse of possibilities, not by any means a random assemblage but a striving after new forms ' (Turner, 1990: 11-12). My own use of the term includes certain aesthetic features described by Turner, but emphasizes the corporeal, technological, and chthonic (Greek: back to the earth) or primordial (Broadhurst, 1999a: 12).
These spaces are liminal in as much as they are located on the 'threshold' of the physical and virtual. As a result, tensions exist within the spaces created by new technological art practices, such as, motion capture and artificial intelligence. Since no body not even a naked body escapes (re)presentation altogether (Broadhurst, 1999a: 103), the virtual body (as any other body) inscribes its presence and absence in the very act of its performance leaving gaps and spaces within its wake. I suggest it is within these tension filled spaces that opportunities arise for new experimental forms and practices.
Within my current research I explore performance practices which explode the margins between the physical and virtual and what is seen as dominant traditional art practices and innovative technical experimentation. This involves collaboration with Richard Bowden, a systems engineer from the Vision and Virtual Reality group at Brunel. The VVR group research into methods to allow both humans and objects to be located and tracked seamlessly and in real time. The applications of this technology range from Visual Surveillance to Virtual Reality of the media and production industries. For instance, constructing a smart room that can anticipate the needs of the user and the creation of tele-virtual newsreaders.
What I was particularly interested in, initially, for my own practice was markerless motion capture which allows the 3D pose of an individual and its motions to be extracted without the need for sensors, calibration or preparation. Magnetic or optical motion capture has been used widely in performance and art practices for some time now. This involves the application of sensors or markers to the performer or artist’s body. The movement of the body is captured and the resulting skeleton has animation applied to it. This data projected image or avatar (Hindu: manifestation of a deity or spirit) then becomes some part of a performance or art practice. For instance, in the most recent work of Merce Cunningham, Biped, an absolutely stunning but ultimately distancing performance, pre-recorded dancing avatars were rear projected onto a translucent screen giving the effect of a direct interface between the physical and virtual bodies.
However, the use of markers and the accompanying technology cause quite a few restrictions to the performer’s movements, for instance, mainly having to remain within certain spatial constraints. There is also the need for extra preparation and calibration and if the physical and virtual interface is in real time then aesthetically there are some problems. So with the development of markerless motion capture, these restrictions are removed. This particular research is still being developed.
Our main collaborative project that we are currently working on, ‘Interaction, Reaction and Performance’, involves direct interaction between an avatar and a physical performer. The physical work was initially a text and movement based piece which was performed at Brunel University in 2000. The text titled Blue Bloodshot Flowers and written by Phil Stanier involves the remembrance of a love affair. There is some ambiguity on whether it is between two adults or an adult and a child. Also if the narrator is dead – the ex lover is obviously long gone. The performer, Elodie Berland, is French and we used a French voiceover as a memory device in the initial performance with good effect. When we decided to combine the piece with interactive technology we initially wanted an avatar which would be female and perhaps a child to represent the child of the affair or the inner child. However, this all seemed too literal and when we saw Jeremiah we immediately wanted him in the performance and decided to leave it to the audience on how they would interpret this virtual presence. Though, of course, most people would assume it was the image of the departed lover.
Jeremiah is a computer generated animated head based upon Geoface technology. He has a simple bone structure which allows him to express himself and emotions with which he can become angry or sad. But most importantly he has eyes with which he can see. He doesn’t only interact but also reacts. In fact he possesses artificial intelligence to the degree that he can demonstrate several emotions as a reaction to visual stimulus. Jeremiah is unique in that he embodies intelligence that is no way prescriptive. Therefore, the performance is a direct interaction between performer or audience and technology.
One of the most interesting aspects of this project is how much the spectator projects into the avatar. Jeremiah, as we know, is computer technology programmed with some artificial intelligence and has the ability to track humans or objects. However, the interaction with him is anything but objective. Most people when they first see Jeremiah find him fairly spooky. After the initial contact people tend to treat him in the manner of a small child or a family pet and behave accordingly. Usually trying to make him smile and generally please him. His face becomes so sad when he is left alone that it is becomes quite difficult to walk away. Although he is programmed with emotions to react to certain stimuli, he can demonstrate fairly random behavior that can be fairly disruptive during a performance adding a further dimension to experience of working with a virtual body.
‘Interaction, Reaction and Performance’ is a pilot scheme for future projects. And it is a feasibility study in as much as if this is successful we would want to develop the technology even further. At the moment we are discussing the introduction of hearing and speaking to the avatar and another head is being constructed which can morph between male and female. Though, we are all very reluctant to lose Jeremiah at least in this project.
From a technological perspective (Bowden, 2001), Jeremiah is based around two subsystems; a graphics system which constitutes the head and a vision system which allows him to see. There is also built in a simple emotion engine which allows him to respond to visual stimuli via expressions or emotions. We are currently working on a new head model called Rachel (see above) who is a deformable eigen head providing more lifelike appearance and motion.
The system is capable of running on a single PC but for speed of operation each subsystem runs on a dedicated PC connected via a network crossover and is, therefore, self standing and also now, due to the construction of a flight case, truly portable.
The head is based upon the Geoface articulated bone model with prescripted expressions for key emotions. It contains a simple Newtonian model of motion with random elements of movement and random blinking and ambient motion.
The vision system is based around a per pixel Gausian mixture model of colour distributions, using expectation maximization within the Grimson tracker framework with additional shadow suppression and noise removal algorithms. This allows static background scenes to be learnt dynamically providing a robust segmentation of foreground objects. Jeremiah’s attention is randomly distributed between these objects weighted by their size and motion. Co-ordinates of objects within the field view are sent to the head model for animation.
The emotion engine determines the current state of emotions from simple parameters extracted from objects of interest within the visual field. This simple set of rules allows chaotic behaviour in a similar fashion. For instance, Jeremiah likes visual stimulus – high rates of movement make him happy. He likes company – no stimulus makes him sad. He doesn’t like surprises – high rates of change in the size of objects make him surprised. Jeremiah doesn’t like to be ignored – if objects exist but don’t move then he assumes they are ignoring him and hence gets angry. Also, if Jeremiah experiences too much pleasure due to too much of any particular stimulus, he will get bored and reduce its influence on him.
Since this research project is a science and art collaboration, there are marked difference in the research rationale and questions.
For Richard, the Turing test describes a system as artificially intelligent if a human user cannot distinguish the system from another human in conversation. He is attempting to test this concept of intelligence by providing an interactive human avatar with simple rules and chaotic behaviour. Richard believes the interactivity and human embodiment of Jeremiah is sufficient that individuals see him as a living entity. Therefore, Richard’s foremost question is ‘How real can Artificial Life become? How do we interact with A' Life?’ (Bowden and Broadhurst, 2001).
I, on the other hand, am interested in a more arts related research rationale and questions. Since, as I have argued elsewhere, language without the body does not ‘mean’ at all, as corporeality provides language with meaning under socio-cultural and thus temporal constraints (1999b: 17), what then are the implications for a virtual body? Therefore my overall research question is:
Due to such developments as artificial intelligence and motion capture becoming increasingly prominent in art practices, does this interface between the physical and virtual body give rise to a new aesthetics? What then are the theoretical and practical implications of this?
My aim is to explore and analyze the effect these new technologies have on the physical body in performance. Especially in relation to the problem of re) presenting the 'unrepresentable', that is the sublime of the physical/virtual interface (or liminal space). Underpinning this is a series of specific research questions:
In conclusion, this is an ongoing project with the first public performance to be staged at the 291 Gallery, London in August of this year and a private performance to be presented at the Media Lab, MIT later in the year. The performance process has proved extremely stimulating and in some ways may prove ultimately more beneficial for research purposes than the actual finished product.
Finally, although much interest is directed toward such new technologies as Jeremiah, it is my belief that technology’s most important contribution to art practices is the enhancement and reconfiguration of an aesthetic creative potential which consists of the interaction and reaction with a physical body, not an abandonment of that body. For, it is within these tension filled liminal spaces of physical and virtual interface that opportunities arise for new experimental forms and practices.
Bowden, Richard. ‘The Human Tracking Project’ (Unpublished Paper), 2001
Bowden, Richard and Broadhurst, Susan. Interaction, Reaction and Performance URL: http://people.brunel.ac.uk/~pfstssb/, 2001.
Broadhurst, Susan. Liminal Acts: A Critical Overview of Contemporary Performance and Theory, London: Cassell/New York: Continuum, 1999a.
---- 'The (Im)mediate Body: A Transvaluation of Corporeality', Body & Society 5(1), March 1999b: 19-27.
---- (Dir). Blue Bloodshot Flowers. Brunel University, June 2001.
Merce Cunningham Dance Company. Biped. Barbican, London, November 2000.
Stanier, Philip. ‘Blue Bloodshot Flowers: Text for Performance’. Body, Space, & Technology www.brunel.ac.uk/bst (Vol.1 No.2), 2002
Turner, Victor 'Are there universals of performance in myth, ritual, and drama?’, pp. 1-18 in Richard Schechner and Willa Appel (eds) By Means of Performance. Cambridge: Cambridge University Press, 1990.
Blue Bloodshot Flowers: