Design And Performance Lab
Interactivity: "User Testing" for Participatory Artworks
Although perfectly common in the world of scientific research, engineering, industrial design and commercial product design, the idea of laboratory tests and "user testing" in the world of art appears an anomaly, unsettling our aesthetic assumptions about artworks as well as the established cultural mechanisms of their display and distribution. We generally encounter a new artwork at its first exhibition or premiere on the stage, in museums, galleries, concert halls or cinemas. In spite of numerous boundary-crossings and reformulations of art as art-process over the course of the last hundred years, especially with the historical evolution of live art and conceptual/non-object based art practices, and more recent media and net art, the notion of an original event which initiates reception (and subsequent reconstructions of the "event" and its manifestations and recordings) still governs our thinking about authors, artworks, and audiences.
But in the contemporary context of electronic intermedia art and its interdisciplinary techniques of research on interactive design, telematics, and human-machine relationships, the nature of the event-manifestation changes, while the role of observation and evaluation informing the design development moves to the foreground. A shift in the aesthetic constellation of artistic production and audience reception is presupposed in installations for human-computer interaction, aiming at an active "user" who experiences, not a static completed work, but an intelligent, responsive environment or a self-organizing system. Such a "system" requires the participant to engage the various interfaces which control and mediate the aesthetic as well as psychological processes the work harbors. The "system" is designed in explicit anticipation of its user: it is always becoming and never completed.
Defining such installations in regard to their conceptual design, their interaction models (based on activities or objects), styles and interface metaphors draws from a vocabulary that is closer to prototyping and marketing than to art. But the software development for such work now indeed has a user-centered design focus emphasizing different goals or the appropriateness of different kinds of interfaces, depending on what kind of activities need to be supported, what kind of efficiency and usability or pleasurable enjoyment in the user experience is desired. Usability and evaluation (testing) are inextricably linked. In a standard professional handbook for "Interaction Design" mostly directed at technical and commercial markets (addressing students and scholars in computer science, information systems design, psychology and cognitive science), we learn for example that a user-centered approach to interactive product designs involves three key characteristics:
1. Early focus on users and tasks. This means first understanding who the users
will be by directly studying their cognitive, behavioral, anthropomorphic, and attitudinal characteristics. This requires observing users doing their normal tasks,
studying the nature of those tasks, and then involving users in the design process.
2. Empirical measurement. Early in the development, the reactions and performance of intended users to printed scenarios [ ] is observed and measured. Later on, users interact with simulations and prototypes and their performance and reactions are observed, recorded, and analyzed.
3. Iterative design. When problems are found in user testing, they are fixed and then more tests and observations are carried out to see the effects of the fixes. This means that design and development is iterative, with cycles of 'design, test, measure and redesign' being repeated as often as necessary.(1)
Even though instrumental aspects of optimization dominate commercial product design as well as the so-called "conceptual models" (based on the idea of a person conversing with a system where the system acts as a dialog partner), it is noteworthy that the "Interaction Design" handbook distinguishes between objective usability goals and more complex, subjective user experience goals. The latter pertain to issues of educational and aesthetic experience, fun, or pleasure, and the lines between business, entertainment and art become more blurred, even as computer science literature generally never addresses user experience design in interactive dance or art installations. Again, whereas cognitive science informs the design process from conceptual models to physical design, say in the case of cellular phones, understanding users who carry out tasks with mobile phones may not prove helpful for an evaluation of what constitutes successful or unsuccessful user behavior in an art installation. We also need to separate such user models from the performative dimensions of interactive art which is staged for an audience and does not require the audience to become active itself. An interactive dance or music piece performed for a spectating audience may involve similar interface designs in its programmed environment, but its sensors or motion tracking devices are only there for the expert performers who have trained and rehearsed with them.
In the following, I discuss an example of installations designed explicitly for the experience of the visitor, i.e. which depend on the presence of the visitor inside the "system." I begin with a brief overview of the conceptual field of interactivity and the circumstances - within specific production situations - which have led artists and installation designers to consider treating the development of their work as a sequence of laboratory tests and prototypes opened up for "testing" by expert and non-expert users. It will help to outline a spectrum of aesthetic, critical and sociological theses which respond to the particular principles of collaborative interactive art operating in the current cultural formation of participatory installations. If we follow Roy Ascott's suggestion, we have to consider these principles (for example art as processual system, the activation of feedback loops, the negotiation of behavior contingent on feedback, etc) as a larger phenomenon in today's digital cultures. Interactive, systematic processes of cybernetic art, Ascott proposed already back in the 1960s, can be seen as interconnected components in the larger system of feedback loops that constitute culture. The integration of cybernetics and telecommunicational networks into aesthetics therefore suggests that art, science, and society are interconnected systems of feedback loops.(2) Such an understanding of digital media installations opens up aesthetic notions of the artwork and its inclusion of active audience-collaborators to larger sociological theories of interactive behavior and communicative situations, and of the cultural production of meaning as the result of information exchanges. It is not surprising, then, that some writers urge us to reconsider such artwork as belonging to an evolving history of science and technology (cf. Stephen Wilson's Information Arts ), as well as to various areas of research in neurobiology, the cognitive sciences, communication studies and social anthropology, rather than to an exclusively aesthetic realm.(3)
From the Theatre to the Transmedial Laboratory
Theorizing user testing andthe social aesthetics of inter-acting indicates the changing conditions under which we work. Some of us who design interactive performances or installations used to create pieces that were performed on the stage or shown in visual art contexts or site-specific locations. The notion of "user testing" was unfamiliar, even though there is a time-honored tradition in the theatre to present new work to "preview" audiences (sneak previews are also known in the film industry, but the movies have already been completed and what is shown is the final cut, to test audience reactions). Yet previews are more of a warm up for the final premiere presentation, allowing director and actors some small changes or finishing touches in a production which is practically completed after months of rehearsal (as far as dramaturgy, scenic, sound, lighting, and costume design, or acting choices and role interpretation are concerned).
However, there are forms of experimental contemporary theatre and performance art which use intermedial strategies and flexible dramaturgies that can be adapted to particular situations or even need to be adapted in order to effectively include the social and political matrix surrounding the action (besides historical avant-garde examples of the kind of happenings, environmental and participatory theatre we remember from the 1960s or from politicized activist performances that utilize various modes of intervention into public space or everyday life, a practice already theorized by the Situationists in France or by theatre pedagogues such as Boal or Freire who believed in performance-action as social transformation which, by necessity, implicates the audience). Such adaptive performances are relevant to our discussion here since they often explicitly "stage" a real, anti-illusionist encounter with the audience (Felix Ruckert's 1999 piece Hautnah, for example, required each viewer to spend time one on one with a dancer in a small tentlike space), thereby highlighting playful or provocative demands of participation and instruction for the audience which reflect on issues of control, surveillance, privacy or the very concept of "direction" - cinematic, theatrical, choreographic - involving processes of selection, arrangement and exclusion.
Furthermore, in the history of performance theory the theatricalized real "encounter" or action signals the shift from aesthetic contemplation of the object to an aesthetics of experience locating the essential affect of a work in the transformation of a situation and the perceptional and emotional experience of the recipient, also pointing beyond the ancient notion of catharsis in Greek tragedy. It already implies an interactional scenario of exchange. This scenario shifts from the inherited model of the theatrical system of representation, with its hierarchical maps for the production of closed, completed and autonomous stage works and one-way systems of communication in which the audience is the passive receiver, to alternate models of transmedial, hybrid and networked performance in which, ultimately, no single authorship is attributable since they depend on dialogical processes of interaction. The current production model of transmedial performance process is the laboratory. One could argue that all interactive art is performative, even in the case of installations or programmed environments exhibited in visual or electronic art contexts. Our notion of "interactivity" is adopted from media culture and the sciences (cybernetics), drawing attention to the ways in which interactivity as a participatory model moves between theatre, dance, installation art, net.art, telematics or more straightforward uses of the internet, and computer games.
The research context for my example is the DAMPF Lab. Dance and Media Performance Fusions (DAMPF) was conceived as an interdisciplinary collaborative initiative supported by a partnership involving tanzperformance köln, the Ars Electronica Futurelab Linz, V2_Lab Rotterdam and the Animax Multimedia Theater Bonn. This European partnership facilitated a series of ongoing events in 2003-2004 within the framework of two complementary strands: (1) co-productions and (2) research labs. It is important to emphasize this particular production scenario: the co-productions were aimed at the nurturing and realisation of specific art works that integrate interactive computer technologies with performing arts practices, in this case involving a collaboration (Rebound's Lab ) between choreographer François Raffinot, percussionist Roland Auzet, and the programmers Yan Philippe, Sven Mann, Niels Elburg, and Bernd Bleßmann; a project led by composer/director Klaus Obermaier working in close collaboration with programmer Christopher Lindinger and the Ars Electronica Futurelab to investigate the effective implementation of interactive technologies in the context of a new stage performance (Apparition); the third was led by choreographer Angelika Oei collaborating with the V2_Lab on the creation of a series of prototypes heading towards the production of a large scale interactive media installation work (Kurort ).
Scott deLahunta organized a research component for the DAMPF initiative, inviting a team of practitioners and scientists to help generate a range of diverse and shareable outcomes to include: drawing questions from and in turn supporting the creative work of the co-productions; the development of reusable technology solutions (found in extant or newly developed hardware and software); and the devising of unique dramaturgical and user testing approaches to the artistic process. In addition, the research labs aim to disseminate documentation and writings that will contribute to conceptual, theoretical and educational approaches to this area of work (http://dampf.v2.nl).
Three research labs took place over a period of months, the first in the context of the Ars Electronica Pixelspace conference 2003 in Linz. As deLahunta points out, the theme of the Pixelspaces conference focused on the relation between the programmable (as different from the navigable) interface and interactive physical environments, and thus offered several DAMPF related research lines to be developed within this theme; e.g. connections between code and choreographic processes, complex perceptive/receptive modes, generative performing systems, etc. In the second lab, held at the Animax Multimedia Theater in Bonn, Raffinot performed with Auzet on the intelligent stage of the Animax. The third lab was held at V2 in Rotterdam in early 2004. In June/July 2004, a presentation of resulting artistic prototypes and previews took place in the context of the DAMPF Festival organised by tanzperformance köln.
The Laboratory and Programmable Interactivity
Besides the computer science approach to interaction design, the conceptual aspects of live art/performance and interactivity for a theory of "inter-acting" inhabit a wider context of practices. For a critical understanding of the contemporary nexus between performance and technology, it is important to define the parameters for the aesthetic and social construction of multimedia work promoted as "interactive" or "networked performance," especially at a time when art academies are beginning to introduce courses in "digital theatre" based on design concepts generally derived from gaming, 3D animation and VR (virtual reality) design. Such design concepts are quite removed from the performing arts and dramatic theatre. The shift towards computer design and the programming of malleable environments, as we also see it in artificial life research, the biological sciences, and the computer-generated models of contemporary architects (FOA, Diller + Scofidio, Frank Gehry, Peter Eisenman, Greg Lynn, etc), clearly signals the impact of computing on all areas of cultural and scientific production.
It is not surprising, then, that artists and programmers now work in laboratories or design studios in order to develop prototypes of the programmed environment which will eventually become the sensory architecture for the visitors who will walk into it. This prototyping is a form of system design, using the process from conceptual model to physical design. The interfaces in this architecture, whether they involve screens and keyboards, mouse, joystick, or magic wand, microphones, wireless sensors or tracking cameras, need to be tested for functionality in order to allow for the development of the desired effect. Testing for functionality and for aesthetic effect may overlap by necessity; we will come to back to the question whether "aesthetic testing" is in fact possible under laboratory conditions rather than in "the field," whether artistic design teams use low-fidelity and high-fidelity prototyping, whether it is possible and useful to distinguish different levels or dimensions in the design process which may require specific testing methodologies in order to determine accuracy of functionality and a variable range of "user satisfaction" in the aesthetic experience. When I recently worked with Canadian artist Renn Scott who had studied interaction design at the Royal College of Art in London and now works for a firm in Toronto specializing in mobile telecommunications devices, she pointed out that different prototyping cultures are emerging in corporate organzations. Her task is that of a "user experience architect," and her observations, user tests and analyses contribute to the constant refinements made in the product design process.
The design process is thus inherently dependent on the user's real and imaginary relationship to the form and appearance of a device, in addition to functional use, which in our case creates a difficult paradox, since on the one hand interactive art must rely on hardware, software, and advances in engineering, while on the other hand any art form which does not liberate itself from its carrier, the technological parameters and their operational or instrumental aesthetics, risks to be judged in instrumental terms and to fall into "decay" - the necessary ageing and cultural devaluation of a carrier as we have seen it throughout the history of technological features which end up (at best) as sentimental objects in science museums or are simply forgotten in the course of constant upgrading. Emergent technologies of interactive design thus belong to industrial culture and its inevitable cycles of decay and redundancy. This throws a shadow on aesthetic and conceptual presumptions, if programmable interactivity indeed can also be read as a scientific communications project or a data base project which banks its conditions of artistic effectivity on, say, a sociological or psychological method of operation. The end-user of artworks is probably not interested in interfaces that malfunction or do not withstand the test of time. In a revealing and quite sardonic report on his visit to the 1992 Melbourne Experimenta, an exhibition of film/video, performance and interactive installations, Warren Burt points out numerous glitches, practical problems and perceptional dilemmas in the way in which many of the programmed art environments bogged audiences (and gallery staff) down as they tried to figure out how each work "worked," how it structured the time/duration of interaction, or why a piece like Bill Seaman's The Watch Detail might have been very effective, "but like a computer game, you had to be willing to spend considerable time learning how it worked to get the maximum pleasure out of it. I wondered about the seeming contradiction of placing a single-user intensive interactive piece like this into a multi-user public space such as an art gallery, though. Maybe we need to develop the art world equivalent of the video-game arcade, where one can engage with a whole series of interactive electronic environments without being surrounded by the contradictions of the gallery."(4)
The elevation of the "laboratory" in current discourse on performance and "new media" reveals another risk: a hundred years after Duchamp it seeks to redefine again the aesthetic criteria for, in this case, not a found object but the coding of an environment, the speculative research and anticipatory design of a (data)space which may or may not also be a perceptible aesthetic product yet announces itself as art. Within the conceptual currents of a history of art that has recently embraced performance, "liveness," and embodiment, we can assume that interactive art inherently states itself as performance, as action, locating its aesthetic operation in the provisions for social interaction as such, for emotional and sensual communication between persons or, in the case of Kurort (DAMPF), for emotional and sensual communication between a person and the "system." From this definition of performance interaction we could deduce different types of interaction design based on scenarios (which may involve touching or not touching, or which are only visual or sonic, or both), activities such as movement or physical action, or communication.
Interactive art and networked performance, two areas of computer-mediated production that are most relevant to our expanded notion of performance, often overlap but are not equivalent. The conceptual differences - in the use and understanding of interactivity - between performers and digital media/network artists can be significant, especially in the case of highly trained dancers who apply their extensive somatic knowledge to interacting with a wireless sensor or a pressure sensor which may or may not provide a response challenging the dancer's kinetic awareness and consciousness of movement. Dancers also arrive at interactive design-environments with a choreographic or dramaturgical sensibility that needs to be reoriented towards a complex improvisational engagement of sensors and tracking devices, towards what Kenneth King calls "writing in motion" or "kinemorphic transcription" engaging multiple and simultaneously extended motion signals, scapes, images and meanings. (5) On the other hand, recent workshop experiments with multi-user online performances and games, described by Sher Doruff as "collaborative culture," indicate that the deployment of an aesthetic of "play" or immediate communication here entails using webcams, archives and databases for creative play with media, with the goal of developing interactive, dynamic artworks in which such databases and cooperative multimedia software carry an active role in the creation of the work. (6)
Programmability and immediate playful communication (between users) are at opposite ends of the interaction design spectrum. From the point of view of the interaction designer, a precise set of usability and user experience goals initiates the prototyping to assess effectiveness. But even a well-known design principle such as constraints (determining ways of restricting the kind of user interaction that is possible at a given moment), classified by D. Norman (1999) into the categories of "physical," "logical" and "cultural" constraints, may not be effective for time-based installations designed for the art world in which reception is highly contingent on context and in which it might be better to exceed users' expectations than to fall below them, or for Doruff's players who might prefer inconsistency over logical consistency and a non-commonsensical attitude towards actions and the effects they trigger.(7) If one is concerned with effectivity, it stands to reason that designers would want to offer training and hands-on demonstrations on how to use the (prerelease) system: user testing of alpha and beta versions of a new work thus becomes very time-consuming since it involves the organization, management and control of user involvement and problem solving ("frequently asked questions").
The spectrum widens. It is frequently noted that artists and computer scientists/programmers don't necessarily speak the same language or understand their work on a common aesthetic and technical ground. There is also no reason to assume that artistic visions and programmability (with code and software applications) can find solutions that satisfy engineers and artists equally. However, the transcultural development of interactive genres in the current context of globalization, and the integration of interactive tools into theatre, dance and music production, suggests that we look at the historical conditions for the current use of interactive design processes. A focus on design implicates both aesthetic-political and technical questions regarding the contingencies of the performing body in its coupling with technological systems. It allows us to formulate new criteria of inter-acting which offer alternatives to representational theatre and to the dominance of the image in contemporary media performance. In particular, such alternatives concern the role of kinetics, synesthesia and "behavioral play" in transmission or in processes of "transduction," as Adrian Mackenzie has described them. (8) Playing through movement, or movement interfacing with distributed, virtual spaces (multiple rooms, the internet), implies new phenomena of proprioception and feedback in displaced actions. Such alternatives imply low constraints, low viscosity. To put it differently, if the system is designed with high constraints, it will require extra work from the player.
Interaction: Beyond Spectacle
The notion of "interactivity" - connecting bodies to digital interfaces - gains its aesthetic sense if we read it in the context of design processes which build an extended, transindividual nervous system with high variables. In artistic contexts, a theory of inter-acting has peculiar consequences for interaction design, if I am looking at the "nervous system" from a dancer's point of view, for example, who wants to move in it not based on choreographic principles of constraints but on low-viscosity principles of free improvisation and kinetic exuberance. Interactivity points to a new understanding of environments of relations and a relational aesthetics based on interhuman exchange or physical interaction as well as a new technological kinesthetics. Designing digital interfaces in dance, for example, means organizing a sensory and intelligent space for communicative acts that are inherently changeable and unpredictable. The space is not "set" for a fixed choreography, but programmed for potential interactions and movements in which partners behave within a network of relays and responses, and in which media generate perceptions of reality. Interaction thus involves the whole environment, and the partnering maps the "world" through the continuous biofeedback it receives via direct sensory stimuli which are also technically mediated (sound, image projection, tactile sensors, wearable computing built into textiles, etc). (9)
In terms of compositional operation and outcome, interactive art is predicated on process. Beyond the live art connotations, process as interactive playing or playful ritual can be associated with a wide spectrum of behaviors in social, pedagogical, therapeutic, gaming and sports contexts as, for example, in the activities of online game players participating in gaming worlds where numerous people playing roles make up the plot in the process, trading and chatting, joining or building separate clans, adding to the environment or altering it, while the software of the game engine provides the parameters or proposes rules of play. Interactivity here means tinkering and community-building. The modernist notion of composition no longer applies; rather, programming of an environment resembles a kind of continuous postproduction of recording/recorded data, as the interactive system uses input from the tools of connection and manipulates, mixes, and remixes the samples, which in the case of dance includes bodily movements, gestures, sensations. The same open-space concept of interactivity in dance is now also explored in the gaming world, with EyeToy and Playstation programs where there is a dynamic environment for physical interaction. The emphasis has shifted entirely from the object of representation to the emergent situation or playground (playstation), and the materialization of technology, itself. Interactive real time computing in installations relocates "process" in the physical-emotional involvement of the user, and thus alters all conventional distinctions between "artwork" and "observer," which is not the case in interactive performances staged for an audience, as in Klaus Obermaier's DAMPF project Apparition : his aesthetic is still indebted to the notion of spectacle, even though he argues that he would like his audiences to enjoy the interactive play between dancer and system and not just the aesthetic appearence of the dance. Interactivity, in general, offers and assigns roles to the users when interacting becomes an essential component in the condition of the situation, its actualization and reception.
As a consequence, interactivity undermines the aesthetics of spectacle. I define inter-acting as collaborative performance with a control system in which the performer-action or play is tracked by cameras, sensors or other interface devices and thus used as input to activate or control other component properties from media such a video, audio, motion capture and midi-data, text, graphics, scanned images, etc. The latter scenario I call an interactive performance system that allows performers and computers to generate, synthesize and process images, sound and text within a shared real-time environment. The real-time processing differs from the historically evolved understanding of multimedia performance, either based on a dramaturgy/choreography or more open-ended constellations like the chance operations Merce Cunningham has used in his dance collaborations (most recently in Split Sides and his "event dance" in the Tate Modern Turbine Hall in late 2003). Historically, interactivity as a category does not derive from artistic or political contexts: it has no radical or subversive presumption but basically denotes an economy of exchanges, technical processes and boundaries/interfaces between living bodies and technological networks. Interface design is a fundamentally commercial activity (cf. computer games), and it of course trains us to live in a culture of technical apprehension. Doruff might argue that it allows us a new creativity of collaborative communication, sensitizing us to the fragile beauty of emergent realities.
Whereas distancing separation of stage/audience is required by the spectacle, Oliver Grau in his study of virtual reality traces the concept of immersion through a long history of image spaces (panoramas) of illusion, and argues that virtual techniques attempting to overwhelm the senses and fuse the observer with the image medium are not new, but that today's real-time computation and sensorial interactivity, linked with telepresence and distributed networks, infinitely expand the "processual variability of the work" the status of which is challenged by interactivity even as intervention is only possible within the framework of the program. (10) Compared to immersive installations, interactive performance with computer-assisted design cannot claim such a long history, nor has it resolved its relationship to the spectator as user. The proscenium is the dilemma, as long as theatre practitioners remain committed to presentational stagings of multimedia works for the consumption and aesthetic contemplation of the audience. Another unresolved question might be how audiences will behave, and understand the theatricalization of a social-communicative space, once intervention into the performance is expected and becomes conventional. At this point, we can observe how interactivity is explored on stage, and how it is explored ostensibly without a stage.
Collaborative performance design brings actors, dancers, filmmakers, composers and musicians together with programmers and engineers, the process requiring new concepts of "script" and "dramaturgy." Developing a new piece, we worry about the interface design since it decides how meaning and affect become manifest. The process involves intimacy with digital materials, algorithms, physical artifacts, but also with cognitive aspects such as attention, perception, recognition and memory: the potential players adopt to the mediated environment, and affect the evolution of image and sound design. According to Erving Goffman and other social anthropologists who have studied social behavior, ritualized actions and gestures, the "players" become agents in a social scenario in which behavioral action-patterns are available to be used and - on occasion - be re-interpreted and altered. Specific cultural contexts influence the values and beliefs that determine the behavioral patterns. After Marshall McLuhan's media theories, such physical and psychic behavioral role-playing has been examined in relation to media and how media, as extensions of human behavior, affect our environment and our interactions.
It seems that Goffman's ideas of "interaction rituals" and face to face behavior - especially his fascinating analysis of the extraordinary repertoire of maneuvers humans employ in social encounters to "save face" - deserve to be recuperated for a methodological examination of the kinds of behavioral patterns we notice in our audiences' interaction with programmed environments.(11) One could argue that an interactive installation inevitably sets up both a social-interactive and a theatricalized context, since the visitor who enters an installation, marked as an artwork, enters an extra-ordinary space outside of the conventional everyday context, and she will to some extent behave according to the rules of heightened self-consciousness that apply in aesthetic contexts. The particular venue for such work (gallery, museum, theatre, club, etc) will undoubtedly influence the reception behavior, even if the installation seeks to suppress the aesthetic and emphasize the social or technological context. Of course there might be strategies to create the illusion that one has entered a particular space which is not perceived aesthetically (a playground, a chill-out room, a library, a bathroom, etc). Another important consideration in such installations is the element of time, i.e. the temporal dimension of such work and the durational aspect of the encounter. The visitor enters into the "time" of the interactive work, and many intermedia installations today use the loop and hyper slow-motion as structural elements of filmic or sonic textures. Becoming immersed in different (and self-repeating) flows at the same time, encountering different kinds of simultaneous interfaces, generates an increasingly complex interactive environment filled with streams of material and immaterial data: the user needs to create an awareness of where she is and what she can do, what might be expected of her, how she will relate to the potentially infinite duration of the loop form (cf. Bill Viola's The Passions or Douglas Gordon's Psycho ) or to potential changes her behavior might cause in the sequencing of digital media.
This inevitably creates complicated psychological reactions, since understanding the world, in this case, does not simply mean understanding what digital databases or sensors can or cannot do, but the environment will entice the visitor to explore it in certain ways or it may alienate, frustrate, amuse, titillate, undermine, distract or bore the visitor, along with innumerable other possible reactions we can think of. From the point of view of the interactor, there will be choices that are induced, there will be expectations that are deduced from the way the environment reacts to or sensorily and intellectually affects the visitor. The user may act automatically and intuitively, following already formed and learned behavioral patterns, or she may act like an explorer or as someone who enjoys playing with (tampering with) the interfaces, perhaps using them against their "intended" function. Other users may be inexperienced, unfamiliar with such interactive installations and unsure about the degree to which their active participation is required, too self-conscious or shy, or too embarrassed. It is interesting to speculate to what degree such installations have a therapeutic dimension built in, since as a social space, especially if there are groups of people engaging together or at the same time with the interactive environment, the behaviors will inevitable become theatrical to the extent that they mirror public and private gestures, codes, expressions, conduct, emotions, skills, intentional actions and unanticipated responses, etc. If numerous people are inside an installation, there will inevitably be some who are more active and boisterous than others: some will play and others will become observers, and these roles of course are exchangeable and fluid.
This entire discussion of interactive behavior tends to belong to a psychological or sociological domain; it warrants the use of ethnography in design, even though it has been pointed out that observation of detail and cultural affect is not easily compatible with the system designer's need for useful abstractions. The technical and aesthetic architecture of the installation is always going to be affective in some way and generate psychic responses. From the point of view of the technical and artistic design, therefore, we need to inquire whether and how interactive art aims at such subjective and social responsiveness, locating the "content" of the work in the social interaction, and whether the programming of interfaces (which may set in motion a complex array of media images, sounds, texts, voices, etc) and the architectural design of the place for the interaction constitute the shared artistic vision behind a prototype which can be tested. What kind of "use" is implanted into the design, what kind of expressive or involuntary user is expected to become incorporated into the sensorial environment to act with it or be "acted" upon? What kinds of "bodies" of information are stored to be generated in the user's improvisation with cognitive and kinesthetic processes?(12)
Christopher Lindinger, working with Obermaier on the design for Apparition, suggested the example of game design's use of different levels which seek to hook the user to become familiar with the game environment. This model may not be equally appropriate for artistic or educational purposes, but in game design the programmer generally follows the logic of three levels: (1) the attraction level, allowing the user to become interested and slowly comfortable with the environment and the rules; (2) the engagement level, which draws the user into the game and allows quicker and more competent actions; (3) the experience level, involving increasing complexity to sustain curiosity, energy and excitement with the player. This model is built like a pyramid; it challenges the designer to start with a very modest complexity that can be progressively enhanced to achieve "higher" levels. The task for interactive installations, thus, could be to find the initial "attractor" or hook which makes the visitor curious and involved enough to want to progress to more complex levels of interaction.(13) The "user experience architecture" thus intrinsically harbors a learning curve.
The Expert Performer as Interactor
Obermaier's artistic vision for Apparition is a departure from his previous work, which extensively integrated digital video projection into the choreography but was not interactive and relied instead on a precise dramaturgy, prerecorded film and exact cueing for the dancers. In D.A.V.E., the dancer moves between back-screen projections and front projection onto his/her own body: the digital images and animations are mapped very precisely on top of the body, connecting the physical presence of dancer under/into the projection. The same approach was used for Vivisector, except that in this piece there was no back screen and no other light source except the video projector. The dance was visible only through the light of body-projection, working with very real bodies but streaming, so to speak, the video into the body. The performances have a very strong osmotic and anamorphic quality, and in this sense they play of course with theatrical and filmic illusionism (the two-dimensional surface oscillating with the three-dimensional bodies of the dancers). These dance works are spectacular, literally, and there are some extraordinary sequences of projections of (images of) stomachs on real stomachs, gender disturbances, virtual images on real bodies, which confuse our perception of space and time, especially as Obermaier manipulates the time code of video to accelerate the speed of images, reverse the flow, or insert quick animations and numerous jump cuts to disrupt the linearity of the image movement. Physical and image movement are in perpetual conflict. It could be argued that Obermaier's dancers interact with the digital video system, yet their choreography is fixed, it is itself a precise system, and the dancer follows the projection script without feeding back to it (website: www.exile.at).
Obermaier's goal for Apparition is to work with Lindinger on the design of a new interactive stage-work, extending further his aesthetics of the mutual penetration of body and digital media but allowing the dancers the freedom to improvise and play with the system in real time. Planning to have two dancers on stage with the interactive design, he is now interested less in precise image superimpositions than in the physical possibilties of movement feeding into the "physical movement possibilities" of computing, exploring physical modeling of the computer, as well as expressive behavior and personality interrelating with expressionist graphic projections (models generated in real time) which will be seen on a screen behind or above the dancers. Obermaier and Lindinger are also interested in using multiple projections with the EyesWeb interface, so that some rendering can happen on the screen, while other real-time images are mapped onto these images until no edges remain. Working on design problems and the first protoype, they ask themselves how much one can get out of the human body (which data), and how one can integrate these data captured from a camera which stands in the same position (frontally) as the projector. The design involves a series of parameters, and the research focuses on the correlation of data to the aesthetic vision of apparitional bodies. During a discussion of these plans in Rotterdam, Obermaier repeatedly pointed out that he prefers to focus first on simple outlines, needed for masking, and that the pure physical motions are the basics for him, before beginning to hang the goals higher and develop interfaces for gesture recognition and more complex and subtle tracking choices (from multiple angles). Both Obermaier and Lindinger also agreed that all the key questions in the development of the work involve both levels, the technical and the content, always at the same time. Obermaier's choreographic aesthetic is visual, and when Anne Nigten (director of V2) asked him whether technology always coincides with aesthetic questions, he admitted that he finds it difficult to speak about user testing before knowing if his artistic vision can be satisfied. "Usually I work alone, in this case I depend on programmers, and as a team we need to know how far we can go. Our strategy is to define an achievable goal. In the new work, the dancers have to be in a dialogue with the responsive environment, and the audience will have to perceives the difference, namely that the dancers are not restricted and are not following a prescribed choreography."
What is often missing in design discussions is the point of view of the dancer, so rarely articulated in the available literature on interactive dance created by a slowly growing number of artists, including Troika Ranch, Pablo Ventura Dance Company, Company in Space, half/angel, Palindrome Inter-media Performance Group, Barriedale Operahouse, INTERFACE, Sarah Rubidge, Sue Broadhurst, Isabelle Choinière, Christian Ziegler, Nik Haffner, Susan Kozel, Gretchen Schiller, Todd Winkler, Kondition Pluriel, Lisa Naugle, Carol Brown, Ariella Vidach and others.(14) Dawn Stoppiello, in a recent essay composed with her Troika Ranch partner Mark Coniglio, speaks eloquently about the slow learning process she underwent, adapting to wearing flexible plastic sensors attached to the joints of her body and a smaller transmitter box that allowed less restricted and more fluid movements: "On a technical level, we wanted my gestures to control the musical score, the playback of images from a laser disc, the movement of a robotic video projector, and the theatrical lighting for the piece. We realized that this was ambitious, but we wanted to see how far we could go. We wanted to find out how much media one performer could play... During the process of creating and rehearsing In Plane, I became acutely aware how information would flow back in the other direction. I would see the video move in response to my gestural control, and my dancing would be influenced by my playing...Each day felt a bit like my first dance class, overwhelming because I was not yet familiar enough with the instrument to keep track of all of its parts. But perhaps the most important experience for us both came late in the creation process, when the elements had begun to coalesce."(15) She argues that eventually she began to feel as if the video images were not an external object (to which she was linked by some interface) but seemed like "a hand or a torso or some other part of my body. The medium wasn't separate from me any longer." Stoppiello speaks of "entwinement" with the projected images; they become like phantom limbs.
Addressing such interaction as kinetic, temporal, spatio-architectural and emotional performance experience means shifting the interpretive emphasis away from choreography and representation to the dynamics of intermediality and interconnection, and to an immersive imagination, the immersive "unconscious." Moving bodies become hyperextensive and hyperintensive. This notion, in my mind, is indebted to the plastic sculptural process that dancers, visual artists, media artists, programmers and architects have explored - a plastic process of "designing" fluid space through movement which incorporates projected space. Aesthetically speaking, it is transformative space, allowing for the integration of "nervous" or sensitive media presences.(16)
The relational performance architecture involving the dancer becomes a laboratory for technical apprehension, which means that a dancer, in this interface, is like a musician who plays a remote instrument and learns to interact with virtual image-spaces and projected video images that may bear no resemblance to familiar physical (self)images in the mirror. Furthermore, Bodo Lensch (director of ANIMAX Theater) pointed out that Raffinot danced on a floor that had multiple low-resonance speakers built into it, while being surrounded by a 28 channel sound system affording the spatial dislocation of sound and highly complex manipulations of acoustico-dramatic events. The distributed soundscape allows playing with the effects of virtual space and its narrative and emotional implications for the dancer's psychogeography. Performers, composers, and designers interpret the relational architecture of interactive systems in many different ways, depending on a work's emphasis on nonlinearity, gesture-to-music synthesis, gesture-to-video synthesis, or depending on the dancer's dialogue with moving images and sound geographies.
Given such complexity, we must ask how performers and musicians regard the physical relations, the plasticity between performance and "controlled" parameters, and how dancers can see their movement as a form of topological "mapping" of the body's experience and proprioception within the interface. The programming goal is to integrate an image-based recognition system (e.g. a computer running BigEye, EyeCon, VNS, or EyesWeb) or a motion sensor interface (e.g. the MidiDancer, a wearable device Mark Coniglio built for dancers) into a unified MAX, VNS or Isadora environment.(17) But what does "technical" integration mean to the dancers, and how do dancers integrate diverse or parallel parameters into their movement intelligence and their increasing awareness of tactile image projection spaces (as we use them in extreme close-up scenarios for telematic performance) and image-movement as partners in the performance?
From a choreographic point of view, the dancer within an interactive environment familiarizes herself with the response behavior of the sound and video parameters, and both dancer and composer/programmer will strive to create an exponentially more sensitive, articulate and intuitive system. In a shared environment this could mean refinements in sensors, filters, and output processors, but also an attenuation of the performer's spatial-temporal consciousness. Her gestures tap into tactile, spatial, acoustic and kinetic sensations that go beyond the immediate kinesphere of the body. If she performs in conjunction with video or 3-D projections, she is also doing a kind of contact improvisation with virtual image-spaces, with entire artificial worlds. Her movement, if it affects and responds to the image movement, crosses real and virtual spaces. She moves in-between. Such movement literacy cannot necessarily be expected from a non-expert "user," and thus the design of an emergent environment which can "learn" from the unsuspecting audience presents a considerable challenge.
User Test in the Detached Lab
During the weekend research lab with the Kurort team in Rotterdam, many of these questions about "user-testing" were raised before and after each of the invited guests visited the prototype. I now sketch our collective findings, in particular the list of user testing criteria we assembled.
The Kurort/ team installed the third prototype of Kurort in the Rotterdam headquarters of V2_ Institute of the Unstable Media for one week of user testing. Angelika Oei and René Verouden, who had shown an earlier version in Glasgow, collaborated with the V2 designers Erik Kemperman and Stock. The motivation for inviting visitors was based on the idea that in the development of complex interactive installations like Kurort, user design and testing is a new practice which is not very well developed in the artistic field. Can more effective methods of "testing" participants in complex interactive installations be derived from other areas such as the product design field? Or maybe from dramaturgical approaches adapted from the world of theatre and dance? And is it possible for sociologists and psychologists to contribute useful ideas to more effective artistic development processes? The DAMPF Lab explored these questions from different angles, inviting input from three researchers from the fields of psychology, sociology, and communications studies. Their input included presentations of their research activity to provide a context for the art and science interfaces.
Gerd Ruebenstrunk ( http://www.ruebenstrunk.de/) spoke about emotional computing and gave an overview of its possibilities in the content industry. Johan de Heer (http://www.telin.nl/) addressed cross modal interactions, attentive environments and usability testing with some references to participatory design. Loet Leydesdorff (http://www.leydesdorff.net/) sketched his theories of interhuman communication systems, knowledge production, observed and expected communication. In a reader prepared by deLahunta, there were also some pages on "participatory design" from the CPSR website (http://www.cpsr.org/program/workplace/PD.html), an essay by David Mamet on directing film and the viewer's attention to story/narrative, and articles from the New Media Reader (Wardrip-Fruin/Montfort) on historically important materials like Myron Krueger's original essay on "responsive environments."(18)
In wide-ranging discussions we tried to link performance with interface-design and the scientific ideas sketched by Ruebenstrunk, de Heer, and Leydesdorff, but we were also caught in a paradox, trying to delineate methodological approaches to "user testing" before we had actually seen the prototype of Kurort or spoken with the programmers to find out what they themselves were expecting to observe during our tests. We lacked an initial framework of requirements (why was it important to involve users at all, what degrees of involvement, what organizational environment, what are the characteristics of the intended user, in this case various experts from the fields of art and science, etc) and a clearly articulated set of data-gathering techniques (e.g. on-site observation, interviews, questionnaires, focus groups, procedures and rules, etc). This led to some confusion since interpretation of the prototype was not based on a specific task analysis. After each of us had experienced the prototype (for the relatively short amount of time of 20 minutes), we met again on the following day to give our feedback to the design team and clear up the confusion caused by the lack of guidance concerning the data-capture technique the programmers needed for their evaluation of the system functionality. Most of the invited users approached the installation with an expectation that all dimensions of the prototype - technical, aesthetic-artistic, psychological, sociological - would be experienceable to some extent. The feedback session showed that most of us had responses to the technical and psychological dimension of Kurort but did not have an aesthetic judgement other than allowing that we could imagine the work's potential as a mise en scène.
One reason for this was the emplacement of the Kurort prototype in the unaltered office space of the V2 building - Oei and Verouden had not chosen, at this point, or were not given the opportunity or funding to build a specific environment. Thus the appearance or form of the work existed only as rudimentary interface stations (monitors, screens, keyboard, mouse, microphone, desks and a few objects such as paper, broom, chocolate). There were no visual metaphors, no landscape, no atmosphere and no inszenierter Raum in the sense in which, for example, Ilya Kabakov defines various aspects of "total installation" (ein vollständig bearbeiteter Raum ), "doubled" or "open" installation, or in the sense in which other aesthetic characteristics of site-specific, kinematographic or spatial design could be assessed.(19) The laboratory set looked detached from any physical design vision. The withholding of aesthetic responses among the users was also mingled with a certain frustration in regard to the banality or simplicity of the current interface structure and especially the apparent lack of the system's responsiveness. It was generally noted that the system's voice (as the main "attractor") was commanding and director-like rather than inviting or seductive. Regarding the banality of the office environment, deLahunta reminded us that at an earlier stage of the prototype design (at the Banff Center in Canada), Oei and Verouden in fact deliberately chose an everyday prosaic space for their prototype space, using the banal as a tactic in their effort to work with social behavior as a material.
Following Anne Nigten's suggestion, we sketched a list of criteria or "keywords." As a set of requirements they may have been too comprehensive or vague, crossing too many domains, to be practical for the stakeholders in the project:
- design of environment, effects of the setting
- states of the environment
- evaluating/assessing the aspects of being directed or left free
- narrative content
- coherence, clarity
- aesthetic aspects
- social aspects
- duration, and involvement in the design experience
- dramatic aspects, choreographic aspects
- interaction levels, learning curve
- (three levels of testing) - attraction, engagement , experience
- modalities (sensory experiences) related to specific media
- mapping of information
- feedback of the system
- emotional aspects, believability (person, character)
- levels of control, agency in the environment
- how the ecosystem touches you
- art as communication (expression) - has the artist been able to code something that is
beyond the personal relation (code of communication which goes beyond the
- anticipation (anticipatory/reactionary system)
- surprises, disturbance (can the system anticipate the state of observer)
- astonishment (aura & kitsch)
- setting, situatedness (context)
- kinesthetic stimuli for senses
- when (if) does emotional affect happen
Every user was asked to respond to these keywords and articulate their experience, actions and sensations. It became obvious after a while that many of us were telling stories or accounts of our qualitative impressions, emotional responses, and our concerns about the missing sensual or aesthetic aspects. Kemperman and Stock were more concerned with quantitative data: they had monitored how long someone stayed with a particular interface and what the amount of input data was. They were interested in functionality, whether the sensor did what it was supposed to do, and whether the system could observe a particular user, store this information and feed it back into the system as it evolved. Basically, the programmers observed the system they had designed as a communication system, a circuit between communicator (the system they named "Lizzie"), recipient communicator and systemic observer. Angelika Oei then addressed many of her conceptual and artistic queries, elaborating on her research and her interest in neural networks, cognitive and emotional response behavior within self-organizing systems. Perhaps not surprisingly, she tended to anthropomorphize the "Lizzie" system, imaging the Kurort (health spa) to receive a single visitor and engaging him or her in a dialogue (with Lizzie's computer-generated female voice), a process that might reveal layers of whatever it is a person needs for her or his "health" in the imaginative world of the spa. The "personalized" system talks and makes offers, inviting feedback and response. An important conceptual decision was made from the outset: the installation would only be accessible to one visitor at a time, providing for an intimate exchange but also rendering the space more restricted, private.
Oei stated that functionality is of course not where her process ends but where it begins, where all the artistic refinements are set in motion. What is needed in the collaboration with the engineers is a language that facilitates the production process and collects the three levels we distinguished, for heuristic reasons: the level of design (programming), the design of artistic experience, the psychological level or the user's social experience. Kemperman, for example, noted the discrepancy between "crisp" (computer science) and "fuzzy" logic (art) at work in the design process where Oei tended to elaborate on her metaphoric ideas about what Kurort can mean as a "health spa of the mind." To summarize some of the disagreements about these levels, Leydesdorff used the example of the use of objects in interactive installations: How would an apple function in such an environment? There is a distinction between the apple as design object, letting the system learn something or validate the behavior of the user in the system, versus perceiving the apple as a cultural object with many symbolic associations. This led to a longer discussion of the role of objects as interfaces, and how people might handle objects in a system on the psychological level and with aesthetic perceptions. At this stage in the user testing, Stock and Kemperman argued, the system needs more scientific information regarding the technical design of its "learning development." A fascinating discussion ensued about the role of the "trace," i.e. the "footprints" or data input left by previous users which would be encountered by subsequent users of Kurort. For the system, the function of the trace implies observation of user behavior and translating the behavior back into the system. One could also think of the notion of a "trail" - a sign post into participatory design allowing the user to influence the future look of design components in the Kurort.
Nuria Font and Madeline Ritter imagined that "Lizzie" could be perceived as a character whom you visit more than once; the data traces we leave serve to build "personality" into the system and, in exchange, we would be recognized when we return. "Lizzie" could become a mirror of different human personalities and emotions, along the lines of Ruebenstrunk's and de Heer's comments on emotional computing in experimental psychology, which might follow the recognition model of emotion perception/identification/interpretation (simulation of emotion/generation of emotion/expression of emotion). In such a model of emotion parameters, there are numerous indicators (individuals express emotions differently) like facial expression, voice intonation, gestures/movements, posture, pupillary expression, as well as other indicators (respiration, heart rate, temperature, muscle action, blood pressure, physiological data) for which sensors could be used. Ruebenstrunk pointed out that emotions are always person-dependent, thus "Lizzie" would have to develop pattern recognition and also learn moving-objects detection, a technique not yet fully developed in emotional computing which generally uses static sensor devices for measurement (expression glasses, touchphone, seats, voice analysis, mouse, Ekman's FACS, neck sensors, trunk sensors, arms and legs sensors, etc). Ruebenstrunk then explained the OCC simulation model (simulating emotions in the machine) and other models based on this method of determining emotions as valenced reactions to events, agents or objects and how these events, agents, objects can be appraised according to an individual's goals, standards, tastes/attitudes (compound emotions). If the installation system is to communicate emotions, it becomes important to note what the human observer recognizes. For example it has often been the case in robotic installations that observers project emotions on the behavior of little machines: we tend to narrate the world to us in terms of stories and characters.
Ruebenstrunk encouraged the Kurort designers to give "Lizzie" a personality by providing coherence and consistency to objects and "characters" in the installation, providing "characters" with visible cues to support visitors in their attempt to narrate an account of a character's action. He also pointed out that some simulation models (SCREAM, Prendinger & Ishizuka) take into account social context and circumstance to measure narrative intelligence. What is most important in the framework of the psychological model is the user's reaction to machine behavior, whereas Leydesdorff, in his commentary on "Autopoiesis of Interhuman Communication Systems," emphasized a more systemic approach to evolving communication patterns, following Niklas Luhmann (Die Kunst der Gesellschaft ) and posing the interesting question of how social systems can invent new media of communication or how an anticipatory system might contain a model of itself or even entertain several competing models of itself: different kinds of health spas for the mind.
Oei responded that "Lizzie" is a system which also has the character of a therapist, yet someone with a personality which forms itself after the visitor: she is continuously acquiring personalties according to the users' behaviors and the expectations they bring along. Some of us questioned the system's believability; an artistic environment, which one would consider an artificial space, inspires a higher degree of self-consciousness allowing us to play (lie, tamper) with the system, which is not the same as going to a doctor's office or a health parlor. Oei suggested that this is true for "Lizzie" as well, "she" sometimes can be annoyed and treat the user with irony, as it happened when the voice in the installation invited us to taste the fine chocolate, only to tells us moments later that the chocolate was poisoned.
We ended the feedback session by agreeing that the installation prototype, at this stage of development, was ambiguous and unresolved about its internal dynamics, its negotiation between need management or the dynamics between persons (if I have need for food I want someone to take care of me) and artistic/cultural desire (I love music), between denotation and connotation. Furthermore, the users did not notice that "Lizzie" actually learnt something from the input, since the voice which addressed the visitors mostly repeated a small set of phrase loops in different sensor areas of the space, apparently not able yet to access the larger data pool of phrases Oei had written. Visitors were invited to use a drawing program (on screen), the traces of such writing should appear on one of the projection films which can be seen when sitting on a swing. The swing interface did not work properly, nor did the users recognize the purpose of a microphone which was set up in the space to record voices (the microphone volume was turned down). Almost everyone noted that the system hardware, sound mixer, etc., were left in the open, which meant that we could see some of the Max/Msp/Jitter patches on the screens, but it was not at all clear what we were meant to do with the exposed code. I also noted an over-abundance of text (written notes, sentences lying on the table) and text projection (video) cluttering the room, while a meaningful correlation between text, music, and film projection was not apparent. The film I could see when sitting on the swing did not make any particular sense to me, and the closed circuit images (of myself) did not provoke me to ponder the relation between health, memory-traces, and human need.
Kemperman insisted that there is a friction between designing an experience and designing a system that can produce the experience. Oei confirmed this assessment and accepted our critical feedback: even if the functionality of the system was ready to be tested from the point of view of the programmers, the artistic installation was not ready, its content incoherent and perhaps simply uninteresting. Kurort had not found its aesthetic form, it had not begun its formal exploitation of the "medium" of interactive design, it had not determined its "marked space." Luhmann speaks of "Formentscheidung" which constitutes art as a social medium (produced for an observer), and this "Formentscheidung," the artistic direction which creates what Luhmann calls unwahrscheinliche Evidenz, must be aesthetically perceivable: "jede ästhetische Formbildung beginnt mit einem Schritt, einer Entscheidung, die den marked space, den Raum der im Entstehen begriffenen künstlerischen Form, vom unmarked space, dem offenen Raum der Möglichkeiten anderer Formen abgrenzt."(20) Even if we grant that the design process goes through iterations, gradually implementing the engines of content into the program, without aesthetic form superseding the operational, or without aesthetic form guiding and even determining the interface, the installation remains empty. As an unmarked space, it doesn't allow us to be touched on any level of intensity and qualification, through corporeal rhythm, movement, affect, energy, connectibility.
Not having found a systematic approach to the physiological and psychological levels of user testing, it is premature to measure our aesthetic responses, our connections to the interactive design of Kurort. Following Brian Massumi's eloquent plea to look at the different logics of affect and emotion, and to remember that "intensity" for example is "embodied in purely autonomic reactions most directly manifested in the skin - at the surface of the body, at its interface with things," we have to study more carefully the various kinds of connection we make to physical design and material surface, content (via language, image, sound, visual form, sculptural materiality, etc) and "effect." Massumi searches for a way to address the quality of experience: "An emotion is a subjective content, the sociolinguistic fixing of the quality of an experience which is from that point onward defined as personal. Emotion is qualified intensity, the conventional, consensual point of insertion of intensity into semantically and semiotically formed progressions, into narrativizable action-reaction circuits, into function and meaning. It is intensity owned and recognized. It is crucial to theorize the difference between affect and emotion."(21) Especially for performative installations of the kind under examination here, with tactile interfaces, images, sound, and implied physical action, an aesthetic response which registers affective sensation (and the category of "intensity" proposed by Massumi) seems inarguably important.
The testing of the prototype has not provided a conclusive model of user-centered design process. Our net of keywords was cast too wide rather than narrow, but it generated a very useful brainstorming session, affording the design team and the researchers to reflect on method, collaborative praxis, and critical frameworks needed to identify common formal and thematic concerns that allow sustainable interpretation of interactive art as an evolving creative practice, particularly in light of the growing diversity of contemporary intermedia and installation art and the intersections between videogames and art. On the one hand, prototype testing in the DAMPF lab revealed the complex nature of user observation and the challenges that come with systematic scientific approaches to an experiment. While Massumi's philosophical emphasis on bodily sensations is closer to art theories of reception, the application of thinking strategies from the computing and social sciences opens up criteria of measurement that are very valuable for the creative work in development, if we posit that user testing can have systematic features of research in which a hypothesis is tested and all variables are recorded. Research experimentation with interactive systems would strive to discover new knowledge, whereas the modeling of user behavior serves to predict user performance, and the evaluation of user performance is needed to improve usability design. In the latter case the operational and functional side of the prototype needs to be foregrounded, and quantitative results were precisely what the Kurort programmers must have had in mind. DeLahunta pointed out that the system designers had worked hard to complete the software "scaffolding" for Lizzie. They needed to know that the system worked: it knew visitors were in the space, and it started to make calculations based on a small number of parameters that it could interpret.
The qualitative issues of user involvement, on the other hand, highlighted the range of theories which can inform such installations as models of social interaction where the system's "purpose" may not be clearly defined or cannot be limited to aesthetic responses alone. The differences between usability and sociability can be considerable, and artists and designers now need to develop concepts with which to address complex user experience goals involving emotional responses, excitement, trust, discrete levels of participation, etc. There is clearly a socio-anthropological dimension, but the concepts may also revert to dramaturgical notions derived from theatre, dance, and film, especially in regard to narrative (multilinear, nonlinear, hypertextual), performance-writing, choreographic notions of rhythm, dynamics, proprioception, and sensory rather than visual cues, the use of sequence and loop, the construction of movement-images, etc. Extending ethnographic observation of cultural, industrial and scientific performance, we can revisit concepts of "theatricality" developed in aesthetic philosophy, art criticism (cf. Michael Fried's critique of minimalism and intermediality) and performance theory in order to describe how installations theatricalize social encounter and play, and how they exploit the ambiguities or ironies of play in non-competitive contexts (cf. Xavier Le Roy's latest conceptual work, "Project," which examines precisely the stable/unstable conditions of rule behavior).
It is also apparent that interactive users of such installations resemble players of videogames, and that game structures (challenges, levels, control methods, progress/feedback, reward, etc) provide fascinating models (quest, pursuit, play, narrative space or other spatial typologies, multiplayer experience) for a critical and reflective space in which action and performance can be scrutinized.(22) Instead of criticizing interactive installations shown at electronic arts festivals for their lack of content, narrative complexity and psychological depth, we may have to look at how they construct a socio-cultural space for play (or "interaction ritual," as Ernest Goffman described it already in the 1960s), communication and symbolization, physical and mental interaction and interpretation. How they redefine cultural phenomena, in other words, and how they explicitly emphasize the observer's response and active assistance in forming the media text itself. In doing so, they inevitably raise new questions about presence, levels of engagement (competition, collaboration) and responsibility experienced in such art.
This essay is an excerpt from a more extended research project. I'd like to thank Scott deLahunta and Anne Nigten for inviting me to Rotterdam and DAMPF.
(1) Jennifer Preece, Yvonne Rogers, Helen Sharp, Interaction Design: Beyond Human-Computer Interaction (Hoboken, NJ.: John Wiley & Sons, Inc. , 2002), p. 285. In this comprehensive and excellent textbook (website: www.ID-Book.com), the authors lay out the different kinds of, and approaches to, human-centered interaction design, before offering five chapters with very detailed analyses of evaluation frameworks, user and expert testing, design evaluation, even including examples of applying ethnography in design. The characteristics quoted above refer to J.D. Gould/C.H.Lewis's "Designing for Usability" . Within the contemporary context of pervasive computing which increasingly uses wireless technologies (cf. the current work of Blast Theory with the Mixed Reality Lab in Nottingham), it also helps to look at the advances in textile and fashion design, especially the concepts of intelligent clothing integrating software, communications devices, sensors and speech-recognition systems into garments. See, for example, Bradley Quinn, Techno Fashion (Oxford: Berg, 2002), pp. 97-116.
(2) Cf. Roy Ascott, Telematic Embrace: Visionary Theories of Art, Technology, and Consciousness. Ed. by Edward A. Shanken (Berkeley: University of California Press, 2003), pp. 26-27. For a critical synopsis of contemporary modes of interactivity in dance, see my "Dance and Interactivity," Dance Research Journal 35:2/36:1 (2003-04), 88-111, and "Spletna Okolja za Interaktivni Ples/Networked Environments for Interactive Dance," Maska, 18:5-6 (2003), 68-78.
(3) See, for example, Stephen Wilson, Information Arts: A Survey of Art and Research at the Intersection of Art, Science and Technology (Cambridge: MIT Press, 2002); Oliver Grau, Virtual Art: From Illusion to Immersion (Cambridge: MIT Press, 2003); Ollivier Dyens, Metal and Flesh: The Evolution of Man. Technology Takes Over (Cambridge: MIT Press, 2001); Joke Brouwer, Arjen Mulder, Susan Charlton, eds., Information is Alive: Art and Theory on Archiving and Retrieving Data (Rotterdam: V2_ Publishing/NAI Publishers, 2003), and Joke Bouwer, Arjen Mulder, Susan Charlton, eds., Making Art of Databases (Rotterdam: V2_ Publishing/NAI Publishers, 2003), esp. the essay by Sher Doruff on "Collaborative Culture" and the principles of emergence.
(4) Warren Burt, "Installation at Experimenta: Fighting the 'So What' Factor in Electronic Art," in Electronic Arts in Australia, ed. Nicholas Zurbrugg, special issue of Continuum: The Australian Journal of Media and Culture 8:1 (1994), 62.
(5) Kenneth King, Writing in Motion: Body, Language, Technology (Middletown: Weslayan University Press, 2003), pp. 18-19.
(6) "There is no need for visual metaphors anymore," the creators of the 1994 "Handshake" online project argued, implying that net space and data-archives "become one and the same thing: there is no need for one to be translated into the other. The information reaches the recipient as a symbol of a technological space. The space itself is information; any movement is symbolic." Quoted from Gerrit Gohlke, "Restored Revolution: The net.art project 'Handshake': Antecedent to a Critique of Instrumental Aesthetics," in Media Revolution: Ost-West Internet, ed. Stephen Kovats (Frankfurt: Campus Verlag, 1999), p. 70. For Sher Doruff's workshop on collaborative culture, see Making Art of Databases, ed. Joke Bouwer, Arjen Mulder, Susan Charlton (Rotterdam: V2_/NAI Publishers, 2003), pp. 67-81.
(7) Cf. Preece, Rogers, Sharp, Interaction Design: Beyond Human-Computer Interaction, pp. 20-23.
(8) Cf. Adrian Mackenzie, Transductions: Bodies and Machines at Speed (London: Continuum, 2002). See also Marina Grzinic, Stelarc: Politics of the Body? in Stelarc: Political Prosthesis and Knowledge of the Body, ed. Marina Grzinic, Ljubljana: Maska/MKC, 2002, pp. 95-109, and Brian Massumi, Parables for the Virtual: Movement, Affect, Sensation (Durham: Duke University Press, 2002), pp. 89-132. For a good critical overview of the evolution of interactive art within the context of the visual and media arts, see Söke Dinkla, Pioniere Interaktiver Kunst (Ostfildern: Cantz Verlag, 1997). See also Söke Dinkla and Martina Leeker, eds., Dance and Technology/Tanz und Technologie: Moving towards Media Productions - Auf dem Weg zu medialen Inszenierungen (Berlin: Alexander Verlag, 2003); Lev Manovich, The Language of New Media (Cambridge, MA.: MIT Press, 2001); Annette Hünnekens, Der bewegte Betrachter: Theorien der Interaktiven Medienkunst (Köln: Wienand, 1997); Peter Gendolla, Norbert M. Schmitz, Irmela Schneider and Peter M. Spangenberg, eds., Formen interaktiver Medienkunst (Frankfurt: Suhrkamp, 2001).
(9) For an extended discussion of such interfaces in dance, see the chapters "Dancing with Technologies" and "Impossible Architectures" in my book Media and Performance: along the border (Baltimore: Johns Hopkins University Press, 1998), pp. 27-144. For open source concepts applied to dance, see Scott deLahunta, "Open Source Choreography?", in Gerfried Stocker and Christine Schöpf, eds., Code: The Language of our Time. (Linz: ars electronica/Hatje Cantz, 2003), pp. 304-10.
(10) Oliver Grau, Virtual Art: From Illusion to Immersion (Cambridge: MIT Press, 2003), pp. 343-44.
(11) Erving Goffman, Interaction Ritual: Essays on Face-to-Face Behavior (New York: Pantheon, 1967).
(12) For a fascinating discussion of Stelarc's experiments with prosthetic devices and involuntary movement, see Marina Grzinic, ed., Stelarc: Political Prosthesis and Knowledge of the Body (Ljubljana: Maska/MKC, 2002).
(13) References are to my transcript of the discussions at the Rotterdam DAMPF Lab, 17-19 January 2004, held at V2 Institute for the Unstable Media and organized by Scott deLahunta.
(14) How quickly the field of interactive dance, performance and installation art is expanding can be surmised from the more than 140 projects submitted to the Multimedia Section of the 2004 Monaco Dance Forum (www.monacodanceforum.com)
(15) Dawn Stoppiello, with Mark Coniglio, "Fleshmotor," in Women, Art, and Technology, ed. Judy Malloy (Cambridge: MIT Press, 2003), pp. 440-50.
(16) "Nervous environment" is derived from the term sound artist David Rokeby uses for his interactive software "Very Nervous System" (VNS), first created in 1982. VNS uses video cameras, image processors, computers, synthesizers and a sound system to create a space in which the movements of one's body create sound and/or music. In his writing Rokeby has pointed out that VNS is not a "control system" but an interactive system, by which he means that neither partner in the system (installation and moving person) is in control. "Interactive" and "reactive" are not the same thing, according to Rokeby. "The changing states of the installation are a result of the collaboration of these two elements. The work only exists in this state of mutual influence. This relationship is broken when the interactor attempts to take control, and the results are unsatisfying." Quoted from "Lecture for 'Info Art', Kwangju Biennale," 1996 ["http://www.interlog.com/~drokeby/install.html"].
(17) Coniglio, a musician/software programmer known internationally for his work with Troika Ranch, a New York City based company he directs with choreographer Dawn Stoppiello, wrote two interactive programs, Interactor and Isadora, which map data input to control a variety of media outputs, e.g. sonic, video, lighting and robotic. Troika Ranch conducts regular Live Interactive workshops to give participants the opportunity to explore the use of interactive computer technology in performance (http://www.troikatronix.com/)
(18) Cf. Noah Wardrip-Fruin and Nick Montfort, The New Media Reader (Cambridge, MA.: MIT Press, 2003).
(19) Juliane Rebentisch, in her Ästhetik der Installation (Frankfurt: Suhrkamp, 2003), offers a comprehensive philosophical theory of the aesthetics of installation art, gathered under the three main leitmotifs of theatricality, intermediality, and site-specificity. In her chapter on "Raumkunst und Zeitkunst" she refers to Kabakov's definitions of "total installation" which resemble the notion of immersion but also have an explicit association with "abandoned theatre sets": Kabakov's fully designed installations appear like abandoned theatre stages, houses or places where people perhaps just stepped away and could return at any minute. The visitor now is "inside this abandoned scene." See pp. 162-78.
(20) Niklas Luhmann, Die Kunst der Gesellschaft (Frankfurt: Suhrkamp, 1995), pp. 188-89. Cf. Rebentisch, Ästhetik der Installation, p. 90.
(21) Massumi, Parables of the Virtual: Movement, Affect, Sensation. p. 28.
(22) Cf. James Newman, Videogames. London: Routledge, 2004. For new theories on gaming, see also Marina Copier,and Joost Raessens, eds., Level Up: Digital Games Research Conference (Utrecht: Diagra/University of Utrecht, 2003). In late February 2004, Nottingham's Broadway Cinema, in collaboration with the Nottingham Trent University Digital Research Unit, organized the "ScreenPlay" festival on computer games. Aside from its games contests and exhibitions, the festival featured seminars, webcasts and performances exploring interactive digital technology and its impact on culture today. See http://www.broadway.org.uk. While game design is one of the fascinating new areas of experimentation for digital artists, the research collaboration between choreographers and neurobiologists provides another challenging aspect of the contemporary cross-pollination of ideas. For deLahunta's project "Choreography and Cognition", bringing together Wayne McGregor (Random Dance) and experimental psychologists from Cambridge and Birmingham, see http://www.choreocog.net.
Johannes Birringer is a choreographer and artistic director of AlienNation Co., a multimedia ensemble based in Houston (www.aliennationcompany.com). He has created numerous dance-theatre works, digital media installations and site-specific performances in collaboration with artists in Europe, North America, Latin America, and China. He is the author of several books, including Media and Performance: along the border (1998), Performance on the Edge: transformations of culture (2000), and Dance Technologies: Digital Performance in the 21st Century (forthcoming). After creating the dance and technology program at The Ohio State University, he now directs the Interaktionslabor Göttelborn in Germany (http://interaktionslabor.de) and the Telematics DAP Lab at Brunel University.