Design And Performance Lab



(Moveable World)

a new project created in a research collaboration between DAP-Lab and Keio University / Inetdance Japan

. .



Photographic studies by Paul Verity Smith for "Ukiyo"

with Yiorgos Bakalos as "Colonel Brazhinsky" (c) DAP-Lab, 2009

Notes on the creation of interactive images

by Paul Verity Smith


I have been formulating a theory based on something that Mark Coniglio said about introducing the organic into the performance. I think - and this is now the basis of my thesis - that it is this. I feel there is something in the connection between the gesture and the interference with the “image group  or - image batch  by the dancer that can re-emerge as a signature and have poetics. What is paramount is that it must be filmed for the gesture. It is not a movie clip, it is not a slide show or a still image but it is an object that it designed to be interacted with. As such it is incomplete without the interaction.

Almost all of the photographic or film making techniques that are used to make a piece of film or an edit sequence complete or interesting do not apply here because they would simply detract from the interaction. Just as Eisenstein's notion of the montage of attractions is between two pieces of film cut together, the montage of attractions here is between the image that is the live dance and the photographic image behind her/him. The question is not - why would a dancer want to be a video editor, valid though that is; the question is why should we mess up a perfectly good dance choreography with wonderful dancers, in beautiful costumes by projecting video behind it. The notion of the dancer as video editor was simply a bon mot, an attempt to describe a process that took away the notion of the creation of fixed montage making meaning.

This sense of connection between the dancer and the media is not the same as making gestures that click on a particular clip that will give a completely predictable result. I think that Ioanna's film making (for our production of "See You in Walhalla") was beautiful, highly skilled and produces wonderful atmospheric montages of the European City. But from the point of view of interaction it was the very craft of this beautifully finished work that left no room for interactivity. The interactivity should have been the element that finished the work and that work should have been the relationship between the dancer and the image. In this regard the jumps, jitters and occasional blackouts are the same as the flecks from dust on the enlarger or scratches on an old negative. The are its signature and stamp of authenticity.

What then is this image group image / batch image object? I think that it is made up of a clearly easily read single gesture. It is not a wash or a substitute for scenography. It is a drumbeat from the orchestra pit. Think of images from Hitchcock. The freshly cooked fried egg that looks healthy and appetising until a cigarette is suddenly stubbed out in it. Barely 35-30 frames of film but it makes a clear statement and engenders a feeling of revulsion and remains with the audience for long after the film is over. This also defines the medium in a way. Hitchcock s images are purely cinematic – no other medium does this. Look at the work that Antonioni achieves in Blow-Up with repeated analysis of six photographs – taking different contexts depending on the viewpoint or degree of englargement. This concept is reworked in Peter Greenaway's Belly of An Architect where the fat dying architect looks along a row of still photographs that without anyone saying anything allow the audience to figure out that his wife is having an affair with another man. Coppola makes an entire film in The Conversation with a short sound clip that we hear over and over again. In Costa-Gavras' Z a multi-angle shot of a moment of assassination is replayed throughout the film taking on a different meaning as each witness remembers (differently) the same scene. In each of these cases the image or sound is a short sharp clearly defined gesture.

A completely different approach is also that of ambient music. This is creating images that are there but not necessarily observed. I formulated an idea for this earlier which was that we could be creating ambiences – neutral anodyne images that are felt more than read. Using the Sensordress, mild interventions or disturbances can be felt by the audience. The ambience is not the same depending on the dancer's dress. The technique for achieving this I felt would be to use extreme slow motion photography though time-lapse might also be effective. Another possibility is the layering of images in such a way that the presence of the dancer is responded to. Changes in layers depending on movement or position. I think –
and I was completely wrong about this – that measuring the position on stage of a dancer accurately and having a response to this could be as interesting as trying to make a gesture repeat itself on video. In this way the imagery becomes, as you say one of: the other modes of thinking (also dramaturgically), and one of those is derived from digital/interactive arts practices interested in less deterministic (pre-defined) and more open, dynamic, aleatory and undetermined interactive and sensorial/affective experiences.....

Paul Verity Smith's photographic study of gesture motiuon, performed by Helenna Ren (c) DAP-Lab 2008

And your (Johannes) quote from Hansen puts it perfectly into context:
" brings together body, sound, and space into a positive feedback system that creates two kinds of emergence: of new bodily movements and of new frequency interferences. And while both emergences - human and machinic respectively - are only possible through the perturbation introduced by the other, each occurs solely through a reorganization that respects its constitutive principle of operational closure. While both follow the same basic rule - let movement create space - each does so in a manner entirely particular to it."

The next element is the on all the time nature of the video and interactivity. The video would be more effective if it was used sparingly. The idea that we always have to have moving lit images risking an upstaging the dance or confusing the audience about what they should be watching is an overall weakness. In film making, the poetry is between the said and the unsaid. What is edited out is in its way as important as what is left in. The emotional impact is in what the audience fill in for themselves. For this reason one of the old premises of the interactive CD-Rom (a non-lieu if ever there was one) that viewers could chose a happy or unhappy ending was a complete nonsense. The elements that create happy/unhappy and a sense of an ending’ are all in the what is not shown’category. The process of ellipsis means that a cinematic or performed event will not have an emotional effect until the audience member decodes the mystery for themselves. Showing something will not move them. Showing it live and filmed at the same time will not move them and giving the more information will not move them. This is the secret behind Noel Coward's dictum on: Extraordinary how potent cheap music is. Providing the process of decoding is left to the audience member the emotional impact is of the same intensity whether it be Samuel Beckett or a song on a jukebox in the corner of the diner. Sophisticated ideas are interesting but not necessarily moving. Long before the work with sensors Martin and I did a performance with a jazz musician, Andy Sheppard, in which we analysed the frequency of his music and had this influence the images. Martin programmed this but it is now a pre-set in Isadora. The question we were continually asked was what would happen if the power failed and all of our technology was lost. My answer was quite simple. We would send one of the assistants to the Cheap Shop with £5.00 and they could come back with 500 little tea-light candles and Andy would play his saxophone and the evening would be quite magical with that alone and the audience would have a wonderful experience.

The magic of the musician, the dancer, the costume designer and maker, the lighting designer – a much more recent addition to the troupe, the dimmer in stage lighting which comes, I believe, twenty years after the invention of photography - is established and has its own evolving aesthetics. The first time I ever saw full screen video playing on a computer without jumping frames from a programme that permitted any sort of meaningful interactivity was in about 2000 with the advent of the Mac G4.

Johannes' taking the work of Theremin as being the genesis of this form of work is absolutely correct and what is interesting is the way in which the Theremin instrument was immediately adopted. RCA sold them as fast as they could make them to Hollywood. Critics from the time talk about this instrument that could emulate any other instrument and perceived of it as a way in which a whole film could be scored without recourse to whole orchestras. However, if we actually listen to the theramin today, even the lovely Clara s recordings, it does not sound like other instruments at all. It sounds like a theremin. It finds its own voice not when trying to be violins but when used by Jimmy Page in Led Zeppelin's Whole Lotta Love or Brian Wilson in Good Vibrations. By the same logic, the early pictorialist photographers who struggled with callotypes trying to make them look like paintings or engravings were ultimately on a highway to nothing - though some very beautiful images were produced along the way – it required photographers to accept their own medium and explore its own properties before it became an art form in its own right. The same goes for film with the early film makers placing the camera in the best seats of the theatre and simply filming a stage production. By the same logic dance for the camera must have something of dance but also something of film language to be valid.

How does one artistically reconcile the fact that good editing can make a poor clumsy dancer look elegant, and in order to show that it really is the dancer's skill that is creating the magic one has to eschew film technique and editing and simply let the dancer and the dance do the work but this is merely documenting not film-making. The same applies to the use of imagery in interactive dance. Both the dancer s methods and the image makers methods must be adapted to a new medium.

Laurie Anderson s work as well points the way. Her early shows used the same tape-slide equipment that I came from. But here again the eccentric instruments are not just used as a gimmick. She makes say -a violin that plays a 45 rpm record but there is also a continual social comment and wit in her work.
There is also a sort of torch carrying as well in this progression. Bob Moog learned to make theremins (from a kit) and then starting making them commercially. From there he moved to the synthesizer which he invented using Theremin's principles. Bob Bielecki who worked for Moog made many of Laurie Anderson's early instruments. The sensors used in flight simulators – the first joystick control – was invented by Link whose parents had made piano roll machines which was the other half of the synthesizer equation. From dismantled synthesizer parts Michel W at Steim started making his eccentric instruments and experiments with sensors, and it was at Steim that Laurie Anderson made her later instruments. Mark and Dawn went for a residency there and came back with many of their ideas for MidiDancer, and our Infusions System's Axel went for a residency and that is how Infusion Systems was born. A small world.

The aesthetics therefore are positioned between process - it is interesting to look at an art work purely as an example of a new process though this is not necessarily moving or edifying – and cultural context which will provide the ready made framework for emotional impact. The process people are for example: exploring GPS or Biotechnology or concepts of neuroaesthetics. These ideas lend themselves well to grant applications in that the language used suggests some form of scientific enquiry into unknown territories or the creation of some new aesthetic theory based around changing conditions of whatever but these are not elements that are going to reduce an audience to tears. And the interesting elements here in the study of performance going to be based around those elements that move and touch or excite the viewer. The problem with process ideas or academic painting or books by people who write novels without using the letter e is that these are not artistic concepts. I would argue that that the problem with process based ideas is that they attempt to go beyond the idea of an artwork as a cultural artifact. This seems to promise some hotline to a state of consciousness and nirvana that it illusory. The V.R. phenomenon went down with that ship. The fact is one was still just looking at images only this time one felt slightly nauseous while doing so.

Anxd why not mention Michèle's fashion/visual art/live art context where perhaps "extreme fashion" works on visual effect, excess, hyperbole, etc, and not on "comprehensibility"
and yes, premise 1: agreed on the side of transparency and 1;1 or literal mapping. Exactly. Michèle' s costumes are thrilling and exciting. I am particularly interested in the way she uses the textures of the found materials. It is a pity that the audience cannot see them closer. Our problem is to provide an enclosure, a space for the costumes to – leur mettre en valeur . I am not questioning the ideas of Sensordress at all – I am certain that it will the future. Either telecomms or walkman or some forms of identifiers built in. I think that the Sensordress will probably receive more than it transmits. We were transmitting.

My question is how we make a performance that gives this value. Sadly what we learned was that there is very little difference in sewing in an accelerometer carefully into position and having it not work and just putting it in the dancer's pocket and having it not work. In a way for this to work we need to perhaps think of the costuming in the same way as the video. The costume will be finished by the sensors. In a way like the video we should be working with unfinished costumes where the sensor interaction completes the picture. I don't think the comprehensibility issue is the same. People know what clothes are, they don't know what accelerometers do. The quality of the costumes is immediately compressible regardless of one s fashion sense. Even someone with my sartorial elegance can appreciate them.

We can see indications for the direction of work in the paintings of Baconm, and for me in the poems of Eliot – it has not been possible to illustrate Eliot s line on "time past and time present are both perhaps present" &

Evelyn Cobley states:
"Like the camera lens, the eye reconstructs reality metaphorically; the signified referent (the perceived object) is as arbitrary as the signifier (its visual or verbal representation) because it is determined by culture rather than by some natural reality." (p. 108)

The reader of Dispatches must reconstruct reality from the arbitrary signifier in the same way as the eye or camera lens. Christian Metz's exploration of the difficulties inherent in montage is relevant to the discussion of "Illumination Rounds" as Herr's endeavor to achieve its literary equivalent. Metz writes:
"Although each image is a free creation, the arrangement of these images into an intelligible sequence-cutting and montage-brings us to the heart of the semiological dimension of film." (101)
Though he faults Eisenstein's sometimes too zealous attempts to perceive "precinematic" montage in all works of literature and film, Metz cannot but admit the manipulation of syntagmatic paradigmatic significance apparent in montage film.
Frederic Jameson's theory …of total flow and free floating signifiers in video proves useful in interpreting the dynamics involved in the "Illumination Rounds" section of Dispatches. Jameson's theory of videotext, a wall of televisions displaying a variety of images simultaneously, bears a similarity to film montage.
The formulations, he writes: pose two signs of equal nature and value, only to observe that in their moment of intersection a new hierarchy is at once established in which one sign becomes something like the material on which the other one works, or in which the first sign establishes a content and a center to which the second is annexed for the auxiliary and subordinate functions (the priorities of the hierarchical relationship here seeming reversible). (87)

Each vignette in this section has a meaning in and of itself, but each takes on a different meaning when placed in relation to the other vignettes in the section. This postmodern pastiche and exchange of significance provide Herr with a method of ordering and interpreting the Vietnam experience both for himself and for the reader.
We can also see a cinematic approach to writing in Don Delillo' s Underworld where he takes a single incident in a 1920's baseball game where a ball is hit into the crowd and then wraps a narrative around all of the spectators who were in the section of the crowd where the ball landed. There is a Dutch photographer whose name I forget who takes people who have a similar sort of look from the street – common fashion sense – age whatever – and then takes them into a studio with a white seamless background and makes them pose in exactly the same way. He creates exhibitions of almost identical pictures in which one is forced to pick out the tiny differences given that the pose and background are the same.

In this regard we should look more carefully at the se of multiple screens (e.g. triptyc).. Our MatroxTriple-head-to-go is simply used as a splitter. We can for example have an image on screen one interact with an image on screen three. We can have the image part follow the dancer. We can break up the screens into smaller screens either with graphic lines or soft edge masks. Images can break up and then reform depending on the performance needs. This area has not been explored yet working with multi-projector slide equipment we made this happen all the time using little metal masks on the slides. We can have very slow building of an image perturbed by the dancer movement.
The notion of the costumes as characters is also a very strong lead for imagery. The way in which the textures of the clothing could be photographed, perhaps even in closeup so that there can be an inter-relationship between the costume and the character onstage wearing the costume.

I don t think we have tried all our options. We just need five times as much research and development time and a standby electronics person at all times. The feedback that I got from my people was that it was magical. And everyone was overwhelmed with the Suna no Onna costumes. I feel that Katsura's "unfinished dancing" lends itself to dancing with sensors and imagery. The unfinished choreography complements our design of objects precisely to be inter-acted with.


. .


Yiorgos Bakalos as "Colonel Brazhinsky" (with phantom limb) guarding Swiss Réduit, in film shoots on the new movement characters for UKIYO (c) DAP-Lab, 2009


Cinematic movement character studies by Johannes Birringer


Wearable designs & styling on this page courtesy of Michèle Danjoux & Johannes Birringer



For the development of the choreographic installation and its interactive images, go to the February 2009 workshop images:





This research project is funded by a PM12 Connect/British Council Grant and a RDF (Brunel University) grant.

dance tech network site of Ukiyo project


All photos (c) DAP-Lab


Further notes on design and performance concepts are published on this site.

click here

(c) dap 2007-08


Project directors: Johannes Birringer & Michèle Danjoux

Brunel University, West London