Transient Systems of Communication - the processual space of interactive installation art
In the course of this paper, I want to outline the main cornerstones of my theoretical approach towards a new understanding of interactive digital installation art. Interactive installation art is a very complex multimedia phenomenon, which refuses traditional analytical methods. In order to be able to grasp the peculiarity of this new art form, we have to construct a new theoretical framework. In the meantime I am quite aware of the fact that a theory often finds its justification and validity in a detailed examination of its usefulness as an analytical tool.
II. MAX DEAN, RAFFAELLO D'ANDREA: THE TABLE - CHILDHOOD (PART 1)
Let's begin with an example. Max Dean and Raffaello D'Andrea's interactive installation entitled The Table: childhood (1984-2001) consists of four, white walls forming a room of about 30 square metres. The walls have two openings opposite each other through which the audience can enter and leave the installation space. The only object in the room is a wooden table. This could be the set up for a piece of art, a piece of installation art. The public could be flaneurs in the space, speculating about the meaning of the piece, the artists' intentions or just letting the lack of sensuous stimuli be the space for imaginations and fantasies. It is my guess, that a piece of installation art of this description would not be considered a supreme example of that art.
But the table is moving. I am not talking about random movements. The table is not bumping into the walls or the visitors - by chance. The table is using movement to build up a kind of non-verbal communication with one visitor. Of course, the table is controlled by a computer program - an algorithm. There is a video camera mounted in the ceiling, tracking both the chosen visitor and the table. The table chooses only one visitor. It is very faithful until the chosen one leaves the room. The algorithm is constantly categorizing this visitor's actions as a method of selecting the next move out of a database of described actions. But it is not only a question of selection; the algorithm can modulate the selected action by analyzing and adjusting to certain parameters of the visitor's movement.
Max Dean, Raffaello D'Andrea, The Table at Biennale di Venezia (2001)
The visitor realizes very quickly, that his movements have an effect. They have a 'meaning'. The movements have gained an enunciative potential. The visitor has become a spect-actor. It was the Brazilian Augusto Boal who coined this term. I am quite fond of the term because it pinpoints an intrinsic functional bifurcation. The participant of interactive art has to fill the role of the spectator and the role of an actor. In plain terms, she as to be active and passive properly at the same time.
III METHODOLOGICAL PROBLEMS
This notion of the bifurcated spect-actor points out at least two methodological issues that are relevant when theorizing about interactive installation art. Firstly, what is the object? And secondly, how are the two functions of the spect-actor related to each other?
Until now, art and artefacts have been comprehended as messages in a one way communicational process. The artist creates expressions and exposes those in a suitable forum. The task of the audience is then to understand the message, to extract information, so to speak. This is a very simplistic but nevertheless correct way of putting it. Much theoretical effort has been invested in the description of the artefact on the bases of different matrixes: e.g. historical, genealogical notions, notions of form including matter or, vice versa, notions of matter, including form. The focal point, however, is still the artist and his or her will to express. This romantic heritage privileges the artist and his art as a medium of a hidden force or truth.
The 'aesthetics of reception' for the first time includes the recipient of art into a theory of art. Researches like Jauß and Iser work with a notion of the implied recipient. Now the artefact includes the reader. In this conceptualization of art the artist is in a constant dialog with the fictive recipient in the process of forming the artefact. It is an enunciation process. Robert Jauß reconsiders Aristotelian notions like poiesis, aisthesis and katharsis as discursive components of art. The elementary change or innovation lies, as I see it, in the connection or interference between these disjunctive notions. Poiesis as a creative act cannot occur without aisthesis, now as expectation or imaginations of the implicit recipient. And on the other hand: aisthesis as the act of reception of art cannot function without the recipient's identification with the artist creating the piece. The audience is so to speak re-living the creational act. The aesthetics of reception dismisses the metaphysical undertones of the notion of katharsis and defines katharsis instead as the communicational aspect of art: an understanding of the communicational aspects of the singular artefact which includes the potential of knowledge.
But the aesthetics of reception still considers the artefact as a communicational element or a medium in a one way communication between the artist and the public. The object of the academic discourse is still a frozen or determined enunciation. But what happens, when communication itself becomes the artefact? When art is communicating by the means of communication? This is precisely what happens in interactive installation art. The installation The Table consists not of the mere settings, but of the action-reaction circuit between one recipient, the spect-actor and the table controlled by the algorithm. Therefore, a theory of the aesthetics of interactive installation art has to conceptualize the communication process itself and not only the spatial, temporal or algorithmic conditions. Communication as artefact opts for a conceptualization of the artefact as communication system. The artist of interactive installations creates communication systems, or more precisely, systems of actions and reactions between the participant and the computer and its various interfaces.
My thesis points to a number methodological problems. How can we theorize about communication as artefact if the object to a great extent depends on each participant? Of course we could make our own experiences the object of our investigation. Or we could turn to sociological methods, collecting data and pointing out common behaviours and tendencies. But my aim is the construction of a theory. Of course, in theorizing we have to rely on our own experiences, either as spect-actor or spectator to other spect-actors or even as an artist.
(I do not argue for the quite common prejudice that artists by definition cannot be good theorists and theorists cannot be good artists. In fact, one consequence of my thesis is that an artist working with interactive art has to have theoretical abilities - which, by the way, should not be confused with academic ability).
But the focus lies on the communication process itself. How does communication occur in interactive art? Why? Because interactive installation art is making communication systems into its artistic devices.
A paradigmatic shift is at stake: a turning away from the what-question towards the how-question. Academic analysis concerning installation art cannot focus any longer on matter bound to a enduring, stable artefact but has to examine the functionality of the work of art. This should not be condemned as an anti-critical or anti-political retreat, because interactive installation art is communication through its own functionality and thus includes critical issues. My approach is a modification of the aesthetics of reception. Very naive, one could say, that the artist is constructing a kind of alter ego, a machine, acting on behalf of himself in order to build a communication system that includes the spect-actor.
But how should we imply the spect-actor in this theory? Would he not become once again a fictive yet implied - and therefore rather patient - placeholder; a kind of variable, which theorists could form after their necessities? The answer must be yes and no. Yes, the spect-actor is a kind of variable, but in my theoretical discourse, he is a functional variable, not a variable for a certain interpretation.
My notion of communication system is heavily influenced by the general theory of social systems by Niklas Luhmann . I do not have the time to unfold the very complex system theory, but I do want to point out the theory of social systems' most basic claim, that is the distinction between a system of consciousness and a communication system. The foundation of this division lies in the simple truth that one can try to communicate thoughts as elements of consciousness, but that thoughts are not elements of communication. In other words, thoughts can not pass from one brain to the other. Thoughts are imprisoned in the sculls of our heads, our bodies. Yet we are social beings, communicating with each other and building social structures and social systems. These social systems are communication systems and the elements of social systems are communication. Following Luhmann, we have to conceptualize these systems as closed, recursive and structure bound. This means consequently, that each system is based on a distinction between itself and its surroundings. The surroundings are the unmarked space. In order to convert the unmarked into a marked and known space, each system has to operate, to perform an act of observation. But each system can only operate with its own means, e.g. a communication system can only produce communications, not consciousness, and vice versa.
But a communication system can only produce communications with the help of the psychic system or system of consciousness and vice versa. Luhmann terms this dependency structural coupling. The two types of systems cannot operate independently, but, and that is very important, each system is part of the other system's unmarked space. Furthermore, a social communication system can only be established between at least two psychic systems.
IV TRANSIENT COMMUNICATION SYSTEM - APPLICATION OF THE SYSTEM THEORY
With system theory at hand, I have found a theoretical foundation that allows me to conceptualize interactive art as communication systems. At this point we have to be very precise. Interactive installation artworks are communication systems, but these works do not construct social systems. Social systems emerge in a process of evolution. They are autopoietic or self generating systems. Interactive pieces of art on the other hand are always artificial, constructed and planned. Interactive installations use the digital machine to establish communication. Machines are never autopoietic. They are allopoietic or trivial machines based on an input output circuit - at least at the present state. Of course, what happens between the input and the output can be far from trivial. Computer algorithms are both very complex and highly differentiated instructions which can appear to be intelligent in a human sense. But they are not intelligent, at least not in a human sense. However, they are flexible and complex enough to simulate a participatory element in a communication system.
The above can be said in another way. Interactive installation artworks are simulations of social communication systems. They are fictive, which does not mean, that the act of communication performed by the participant is a simulation. They are real actions and reactions and consequently they can propose a new kind of transient sociality. So, the object of my theoretical efforts is an artificial communication system. All further categorizations should specify the different kinds of communication systems each interactive artefact is constructing, which includes first of all the medium of communication (e.g. language, text, body movement, nonverbal signs etc.), installation settings and of course matter. Notice, that matter in my concept is solely a functional part of the communication system, not necessarily the main purpose. In order to be communication, communication must be about something, it must transmit information in one way or another. My thesis privileges the idea that interactive installation art first of all is communicating through the construction of a specified communication system, which includes the participant's actions. That means that the communication is its own matter.
V STRUCTURAL COUPLINGS
In the beginning of this article I talked about the bifurcation of the participant into an actor and a spectator, a spect-actor. The spect-actor has to act, which means filling a role (which need not to be a theatrical role; it could be a social role) and at the same time he has to perceive the piece as art. Following Kant, the autonomy of art is partly based upon a detached un-interestedness, which allows the audience to enter into a realm of imagination. I would prefer a more up-to-date expression: art has to reveal and functionalize potentiality. I hope to justify this rephrasing during the second part of my paper.
With the system theory in hand, we can conceptualize the participant's bifurcation. Firstly, the system of consciousness is responsible for the so-called 'detached' reflections. These reflections occur at various levels: from the sensuous perception level to the abstract reflections including associations and imaginations. On the other hand, the participant's actions are now elements in the communication system, which the interactive artefact establishes. Actions are never mental images. Only the communication system can be observed as 'actions'! Nevertheless, mental images are presenting a range of possible actions to the communication system. Maybe it is helpful to underline that actions need not necessarily be communicational elements, but they surely are in the case of interactive art. Digital sensor technology always registers actions (e.g. movement, voice, language, sounds and so forth), and treats these observations as signs.
The most interesting question is now how do these two implicated systems relate to each other? This is a very complicated matter and I only have the time to outline the basic principles.
First of all, interactive installations necessitate the physical presence of the spect-actor. Therefore we can talk about a system of interactions. This sounds banal, but in fact we have to differentiate the notion of 'interaction system' from computer science's notion of interactivity. Interaction systems are communication systems, not merely the algorithmic opening towards external input. Participants in interaction systems extract information from simple observations under the premise that the observation is being observed by the observed. In interactive installations, the spect-actor knows that he is being observed by the digital machine. Therefore all his actions are elements in a communication system.
In interaction systems one cannot act without the body. Movements become acts. But movements are not possible without a body. My theoretical claim would be that the body of the spect-actors intermediates between the two involved systems, the system of consciousness and the communication system. But the body is not a passage between two systems. It does not transform data like the computer. Instead the body constitutes a kind of double sided interface. The body is both a communicational element (e.g. speech actions, bodily actions in space and time and so forth) and at the same time the neurological sine qua non for mental images, thoughts and sensations. It is unnecessary to emphasize that every thought and every feeling roots in body states. (Body state is defined by the neuronal and visceral state of a given moment.) Both systems depend on the body, but the body does not belong to either system, even though both systems are coupled with the bodily system.
Phenomenology makes a conceptual distinction between 'corps vivant' and 'corps objective'. Merleau-Ponty defines 'corps vivant' as the registering body, the body through which we live. Corps objective is the one we look upon. It is the detached body. We need the corps vivant in order to look upon the very same body. I would like to rephrase this phenomenological distinction with system theoretical terms. In this perspective, corps vivant constitutes the external reference of the system of consciousness: external because the body is not an element of consciousness. On the other hand, 'corps objective' constitutes the external reference of the communication system. The body becomes a sign. 'Corps objective's' implicated detachment from the body seems to be a paradox. This detachment, seen from the perspective of the system of consciousness, can only be explained by a re-inscription of the body as sign into the system of consciousness. The system of consciousness can only see itself, when the communication system offers a body sign. The system of consciousness cannot 'think' alone about the body, because consciousness first of all is a registration of body states and not signs. It needs a kind of detour around the communication system: a re-entry.
A similar mechanism occurs for the communication system. This system needs the system of consciousness in order to communicate something; such as an idea, or a feeling, etc. Communication is the unity of the difference between information and utterance. Information constitutes the external reference, whereas utterance constitutes self-reference of the communication system. That does not mean that the information part of communication is a thought or a feeling, not at all. Information is part of the difference (difference) a sign is made off and appears only in the surroundings of the system of consciousness.
VI MAX DEAN, RAFFAELLO D'ANDREA: THE TABLE: CHILDHOOD (PART 2)
Let's go back to my example, Dean and D'Andrea's installation The Table, in order to clarify my theory. The participant is communicating with the table by moving his body. But that is not enough. In order to be an interactive system, the spect-actor has to comprehend her own actions as signs. This she only can do by taking the table's actions into consideration. The first step is therefore necessarily to determine if the table is able to transform the spect-actor's movements into sign-actions. The spect-actor reads the table's movements in space as responses to the spect-actor's own spatial action. Only then can she decide if the table's movements are responses or not. There should be a kind of analogy in the spect-actor's movements and the table's movement in respect to certain parameters. E.g. the computer system has to be able to recognize communicational message units. Further more, the response should demonstrate a specific and selected focal point to the spect-actor. This could be a certain rhythm or a direction in space. In the short Quicktime clips, we can see that the table sometimes mimics the spect-actor's actions. This is a very clear demonstration of the algoritimic system, that it 'understands' communicational units. If the spect-actor can approve the table's actions as communicational units, then the installation 'The Table.' has constituted a artificial, transient communication system. From now on, the table's movements and the spect-actor's movements are sign actions, a language based on a non-verbal, space-bound set of signs. The transient communication system is always, with every action, re-constituting itself as a communication system.
The computer system consists of a large database of pre-described actions. The system is constantly observing the spect-action. Is the spect-actor moving in space? If the answer is positive, where to and how? And so forth. The extracted data allows the system to link the registered actions with the pre-described actions in the database. But before execution, certain parameters have to be modified (rhythm, direction in space and so forth), in order to be a specific reaction to the spect-actors action.
Also the spect-actor is reacting to the computer system's actions. But the spect-actor is made up of two different but coupled operating systems: the spect-actor is part of the transient communicational system and has a system of consciousness. (Of course, the human being consists of many more systems, but they are not relevant for my investigation.) The action signs, produced by the installation The Table, consist partly of information that is used by the system of consciousness. Communicational information (which is not an element of consciousness) is used by the consciousness in order to produce mental images.
The system of consciousness is using the body as an instrument, because it cannot produce mental images without the body creating body states. The human being does not have a database like the computer. Memory is not a collection of data describing the past. Memory is the ability to re-live different aspects of bodily states once experienced. Memory is therefore part of the system of consciousness.
The spect-actor in Dean and D'Andrea's installation is reflecting about his own and the table's actions with the help of his memory. These reflections can be emotions (fear, happiness, etc.) or intellectual consideration concerning the functionality of the interactive installation or even considerations about the intentions of the artists. But my point is that these mental images are now used by the communication system as a 'fundus' of potential actions; e.g. if the spect-actor gets in a happy mood, because he thinks that the table is quite funny, he will perhaps begin to dance, producing certain feelings. But dancing steps as well are elements of the communication system. Dancing steps are action-signs. The Table is reacting to the dance, maybe it will dance too by turning around its own axes. Yet another spect-actor will challenge the algorithmic system, by coming up with very different actions waiting for differentiated responses. Once again, mental images allow the communication system to select messages.
Each interactive digital installation is creating a transient communication system, which functionalizes the actions of both a digital agent and a spect-actor. In order to be able to understand the participant's role, we have to make a conceptual distinction between a system of consciousness and a communication system. This distinction guarantees the simultaneity of communicative actions and aesthetic contemplation. The participant's body constitutes the mediating third between communication and consciences, without being a part of neither the communication system nor the system of consciences. The participant in an interactive installation can act and reflect upon her actions. This bifurcation is a 'sine qua non' for subsequent communicative actions.