Navigating the Ocean of Streams of Story

Grahame Weinbren

Earlier versions of this essay appeared in 1995 in Millennium Film Journal, No. 28 "Interactivities," and in 2003 in the ZKM/MIT Press publication Future Cinema. Since the ZKM publication, The Erl King, one of the works from which I developed the ideas laid out here, has been digitally 'emulated.' It now runs on a single off-the-shelf computer, while the1983-85 version requires several pieces of peripheral equipment, some custom built, communicating with a fairly specialized computer, vintage 1982. A number of interesting issues emerged during the process of developing the emulation, and the last part of this essay addresses some of these issues.

"He looked into the water and saw that it was made up of a thousand thousand thousand and one different currents, each one a different colour, weaving in and out of one another like a liquid tapestry of breathtaking complexity; and Iff explained that these were the Streams of Story, that each coloured strand represented and contained a single tale. Different parts of the Ocean contained different sorts of stories, and as all the stories that had ever been told and many that were still in the process of being invented could be found here, the Ocean of the Streams of Story was in fact the biggest library in the universe. And because the stories were held here in fluid form, they retained the ability to change, to become new versions of themselves, to join up with other stories and so become yet other stories; so that unlike a library of books, the Ocean of the Streams of Story was much more than a storeroom of yarns. It was not dead but alive."

Salman Rushdie, Haroun and the Sea of Stories (London: Granta Books, 1990), p. 71.

It is easy to read the nostalgic tone of Rushdie's 1990 "children's story" as the wish for a return to innocence, to a state of story-telling purity beyond the reaches of politics and intrigue. At the time he wrote Haroun and the Sea of Stories, purportedly for his eleven year old son, Rushdie was, of course, too sophisticated\and too embittered by the outrageous circumstances of his life\to profess any kind of innocence or naivete. The novella abounds with metaphors of cruel suppression and mindless censorship. The nostalgia embedded in the Sea of Stories is better seen as a reminder of the creative process a writer (or any artist) craves in his or her darker moments: an ocean of stories, wide and deep, effortlessly tapped. The power of the image is in its erasure of the line between writer and reader\Haroun's father, a professional storyteller, need only drink from his personal faucet plumbed to the Ocean and report to his audience: this alone constitutes writing.

Can the image be turned around? Can we imagine the Ocean as a source more for readers than writers? Could there be a "story space" (to use Michael Joyce's resonant expressioni) like the Ocean, in which a reader might take a dip, encountering stories and story-segments as he or she flipped and dived? In these waters, turbulences created by the swimmer's own motion might cause an intermingling of the Streams of Story. The Ocean, as I imagine it, is a dynamic narrative region, a Heraclitean river into which one can never step twice, a lake of Heisenbergian uncertainty where the very attempt to examine a particular story-stream transforms it. What a goal to create such an Ocean! And how perfect as an ideal for an interactive fiction!

Fiction, Cinema and Cybernetics

The idea of interactive fiction is not new. But the notion of a time-based medium that is not interactive is no older than mechanical recording. Before the recording of sound, all music was 'interactive,' though in this context the term sounds silly. Musicians are always affected by external factors, from the other performers in the ensemble, to the 'vibe' of the audience, to the acoustics of the hall, etc. The same can be said for theatre. The risk of variability is an essential ingredient of the thrill of live performance. And before recording, obviously, all performance was live. Though it is true that few works were deliberately designed to be influenced by external factors (but consider Tinker Bell in Peter Pan), audiences were always aware of their own potential power. Theatre and music mythology contains many stories of performers either energized or ethrownf by the audience. One of the aspirations of an interactive cinema is to return the medium to this earlier condition, where the potential effect of the viewer on the performance is assumed.

In the process of developing an interactive narrative cinema, I realized early that it will not have the shape of narrative that we have learned to expect since cinema and television\media of the 'moving image'\ have come to dominate our story-structures.. The very idea of user impact opens to question the concepts of end and beginning, of crisis and conflict, of development itself. Traditional notions of cinematic narrative must be rethought.

My own work is in the pull of a pair of forces that defined the late twentieth century\the Cinema and Cybernetics, the Projector and the Computer. In 1995, when this essay was first published, I had made two installations incorporating computers and moving images. In 2002 there were five, and in 2004 the first of these pieces, The Erl King (made in collaboration with Roberta Friedman), has been recreated as a fully digital piece. In all these works the participant's (inter-)actions affect the temporal conglomerate of images and sounds. The computer itself is not considered a medium or a tool, but a device that controls and presents existent media. Thus the questions that arise are about how cinema changes when its apparatus is linked to a computer\just as one can investigate changes in the structure of cinematic communication when recorded sound is added to the moving image.

The first two questions that came up can be posed in quite traditional terms. What kind of story will fit the medium, and what will be the grammar of its telling? Crudely: where is the change\in content or structure? One entry point is to find a narrative for which the sequence of events is not salient, since if the viewer is to wander around and through the story, the order in which the depicted events are accessed will need to be open to variation. And this requirement led me directly to Freud's studies of dream interpretation.

A Branching Structure

"I dreamed that it was night and I was lying in my bed. Suddenly the window opened of its own accord, and I was terrified to see that some white wolves were sitting on the big walnut tree in front of the window. There were six or seven of them. The wolves were quite white, and looked more like foxes or sheep dogs, for they had big tails like foxes and they had their ears pricked like dogs when they pay attention to something. In great terror, evidently of being eaten up by the wolves, I screamed and woke up."

Sigmund Freud: "From the History of an Infantile Neurosis," in The Wolf Man by The Wolf Man (New York: Basic Books, 1971), p.173.

Freud transcribed and published the case history of the Wolf Man in 1914-15, soon after the end of the patient's analysis. It is the apex of Freud's early period, where the central concepts of condensation, displacement, wish fulfillment, the primal scene, etc., reach their full fruition, never to be resolved again in quite the same way.

The analysis revolves around the patientfs dream image of the staring wolves. Freud describes the process of gradually uncovering the components of the dream, linking each element with an event, a character, or an emotion remembered but perhaps suppressed. The dream's significance for the dreamer, manifested in the overwhelming emotional effect it had on him and the fact that it remained in his memory for decades, led Freud to seek further explanation. He finally accounts for this power with the proposal that the dream encapsulates the dreamer's greatest fears and desires, as transformed memories of the events that first produced them. For my purposes the details (and\it goes without saying\the `truth') of the dream-analysis are not important.ii I wish only to appropriate certain aspects of Freud's methodology in my own search for a paradigmatic story structure suitable for an interactive cinema.

Condensation is the key concept. The dream is formed by compressing and combining a set of mental objects. The dream can function in the dreamer's mental framework as the distillation of a set of emotional charges. Its powerful affect comes from the fact that, in an important sense, it embodies a set of memories and the specific emotions linked with them. Repeatedly Freud stresses that there is no universal symbol translation table\every element of the dream image, and every property of every element, is understood by the dreamer in his own individual way. Each element substantiates a combination of particular fears, hopes, desires or beliefs, transformed, by the laws of the unconscious, into a component of the dream image. Seeing the images through the dreamer's eyes\identifying the underlying atomic parts and understanding how they are altered by the dreamer's mental process into the dream image\is understanding the dream. In this understanding is written a page of the biography of the dreamer.

Freud's notion of a dream is a conception of a narrative-type based on a hermeneutic method. Unraveling a dream reveals the narrative of the dreamer's interlocking emotional states. But it is not a narrative that unfolds in time\all its elements are simultaneously present. Freud goes to great trouble to convey this atemporality, but even for him it is a notion that eludes expression. After all, his own mode of communication\writing\is, of necessity, linear, one word following another, forming paragraphs that follow one another, etc., while his conception of the dreamwork is non-linear, unsuited to the writing forms available in the early 20th century.

"This task [of forming a synthesis from fragments that emerge in the analysis of a patient] ... finds a natural limit when it is a question of forcing a structure which is itself in many dimensions onto the two-dimensional descriptive plane. I must therefore content myself with bringing forward fragmentary portions, which the reader can then put together into a living whole."

Sigmund Freud: "From the History of an Infantile Neurosis,h p.173.

Considered as a narrative structure, the underlying elements of the dream can be revealed in any order whatsoever, and the same story will emerge. Thus, it is truly a narrative without specificity of sequence.

The Interpretation of Dreams

A film might try to approximate the structure of Freudian dream analysis in a story structure that step by step unravels the components of an evocative image. However, the linearity of cinema sequence tends to freeze material into narrative hierarchies, one element gaining in significance while another loses, depending on each one's context and their overall order. How better to reproduce the minimal significance of sequence, the irrelevance of order, than through interactivity? For in an interactive work the sequence of events can be determined by the viewer. And by the time the viewer becomes aware that sequence is determined by his or her responses, sequence may already have stopped being a criterion of narrative significance. In normal cinematic circumstances, the weight of an event is given largely by its context: now, with sequence under the control of the viewer, the significance of any given element will be in flux, changing from screening to screening. In these circumstances the viewer's understanding of the events of the narrative can undergo a radical transformation, based entirely on the knowledge that things could have been different.iii Later in this paper I shall make an attempt at describing the "subjunctive" state of mind evoked by the interactive cinema.

It would be a gross oversimplification to suppose that the elements associated with a particular dream-image are, by themselves, sufficient to define the biographical narrative underlying the dream. The ways the elements are transformed and combined into the dream are equally significant, so these methods of aggregation must be considered part of the story. Often this syntax and its application can be expressed only verbally. It is difficult, for example, to imagine an effective visual expression of the transformation of something into its opposite (from "staring" to "being stared at," or from the ornate motions of sexual intercourse to the stillness of the white wolves), or the transfer of a particular quality from one object to another (as the color white is lifted from sheep and flour and attributed to the wolves). Freud's interpretation of the dream is far more than a simple compacting of memory-images into one conglomerate: the grammar of the image-elements' metamorphoses and rearrangements is as significant as the elements themselves.

I am not suggesting that the principles of condensation and displacement cannot form the foundation of a visual narrative, but only that some depiction of the types of transformation will have to be incorporated alongside the results of the transformations. The point is to develop a type of narrative that can retain its identity and make sense independent of the sequence of events. Thus, in Sonata, I found that I needed to make the formative elements of the dream, i.e. the grammar of the transformations, into explicit components of the dream-narrative\without them it became a mere medley of scenes connected only by association.


Cinema, of course, cannot be internally affected by its viewers. Turning one's head, far from affecting the visual experience, removes one from the cinematic world and into the mundane space of the screening room. The straight razors and wolves on screen will yield only to forces that are profilmic, within the diegesis, or (commonly) both.

Furthermore, the impossibility of impacting on the cinematic is one of the sources of our pleasure in it. "Don't go up(/down) the stairs!" we inwardly cry out during Hitchcock's Psycho, first to the private detective Arbogast, and later to the heroine Lila Crane, all the time knowing that, no matter how powerfully felt, our distress will not influence their behavior. The experience of suspense would be fatally distorted by the elimination of the inevitability of the characters' actions. If Lila might turn back because of our pleas, the entire effect of the horror film would dissolve. Much of cinema's power over us is in our lack of power over it. In this sense, suspense is a paradigm of cinematic response. It could be argued that the introduction of viewer impact on the representation is a destructive step for the cinema. The removal of the possibility of suspense is the removal of desire from the cinematic, and, ultimately, the removal of the very fascination of the medium.

So my project became: to find interactive forms in which desire can be sustained. It will require the construction of a new cinematic grammar. If it is to be successful, this search, this construction-process, must foreground the temporal aspect of cinematic communication.iv

Time. Time. Time.

Time always moves relentlessly, tautologically, forward, as long as one is alive. "Real time," clock-measurable duration can always be distinguished from time subjectively felt. The duration of cinema is rigidly defined by the apparatus, fully predetermined by the physical substrate of images projected serially at a regular pace set by an electric motor. A film begins and ends necessarily and predictably. Relative to the beginning, the end is dependent on, and only on, the length of the filmstrip. Whatever its images, however they are organized, a film has a physical beginning, middle, and end. Whether and how this linear temporality structures the image-material in a particular film is a major issue (perhaps the major issue) for a filmmaker. It could even be argued that the stance taken by a filmmaker towards temporal structure, how time is articulated in a particular film, is an index of where in the spectrum of cinematic practice (from Hollywood to experimental) a given work falls.

Duration. This gives us the first sense of cinematic time, duration or running time: the clock-time required for the filmstrip to traverse the projection apparatus.

Story Time . The second sense of time is that of the world depicted in the film. This notion is more or less restricted to the cinema of story-telling, and the time depicted in ediegesisf or depiction. Mieke Bal, in Narratology,v calls this the time of the fabula or fictional world imagined in the story. A cinema narrative may jump forward, eliminating decades (or centuries) in a single cut; or slide back, perhaps using one of the various narrative strategies that fall under the category of flashback; or remain in the present, so that a given passage of film denotes a continuous passage of time. This latter case (the most frequently used) still allows for a broad range of variation: one continuous two minute portion of a story can occupy five minutes of screen time, while the next portion compresses seven years into as many seconds.


Experienced Time. Detective Arbogast's walk up the stairs seems painfully extended, so that his stabbing at the first landing is a dreadful shock. This shock puts us on edge for Lila's later descent into the basement, which stretches time even further as each step seems to last a full minute, since we now (rightly) expect the worst to be waiting for her. On one other end of the scale, a contemporary action film can make us feel as if no time passes during its 100 minutes; while in another discourse entirely, a film like Wavelength (Michael Snow, 1968) insists on equating duration and depicted time, a position transformed into an ideology in the 1970s by such 'structural/materialist' filmmakers as Peter Gidal. The relationship of film grammar, plot, and the experience of time is a fertile area of study, especially since the compression of time is probably one of the major determinants of the longevity of cinema as a cultural phenomenon.

The question is: what happens to cinematic time when viewer input becomes a component of the screen amalgam? To what extent does the incorporation of viewer impact keep time real, canceling out the magnetism of cinema itself\when does it cease to be cinema and become "multimedia" in its drab information-delivery costume, the slick transmission of data in fields of "hot spots," "buttons," and "point-and-click menus?" As I revise this essay (2002), this wasteland has become transmogrified into the paralytic gloom of the World Wide Web, and most people associate interactivity with clicking though endless linked Oracle database fields.

The Kuleshov Effect

The temporal grammar of classical film continuity can be summed up in a single example, which, like many cinema myths, is often described but never seen. The "Kuleshov Effect" scenario consists of a close- up of a Russian actor, intercut with several emotion-laden images (a dead woman in a coffin, a child playing, a bowl of soup). This supposedly produces in the audience an apprehension of the actor's face as saturated with appropriate emotion. But more interesting than this (presumably a commentary on the actor's ambiguous, doleful expression) is the idea, taken as "obvious and certain" by Pudovkin, that the character is seen to be "looking at the soup"\the man and the soup are linked, across the cut, into a single continuous space. Of course it cannot be as simple as this, as the sense of continuous space requires the support of a number of factors such as eye-line, lighting, shadow direction, etc.\but the point is clear. I'll suggest a recasting of this fable later in this paper. Here I introduce it only to restate the homily that in cinema spatial unification is easily maintained through temporal and spatial disruptions,vi given a particular sequence. Sequence determines space. And sequence logically requires time.

The Liberation of the Filmstrip

A standard linear unit of cinema has an A-B-A structure: e.g., the look / cutaway / look of a point-of-view moment, the shot-reverse-shot of a dialogue scene, or the performer-audience-performer of the musical. This atomic structure defines continuity of time and space in the cinema.

The equivalent in my interactive cinema is formed by a sequence in which the middle term is produced by an action of the viewer. If the viewer does not act, the first shot continues. But on action by the spectator the B-shot appears, then, after an appropriate period, the A-shot reappears, perhaps transformed by the interspersed shot, perhaps unchanged. In Sonata this structure is used as a bridge to an alternative point of view (for example of another fictional character, or of the author); as a jump to an earlier (or later) time in the story; as a glance at a different depiction of the narrative situation (e.g., a classical painting of the Judith and Holofernes theme rather than its continuing narration by a story-teller); or as the momentary introduction of a parallel narrative line. Does the sequence still denote a continuity of space and/or time? The interpretive mode the viewer takes toward the new material is associative. Because the new image or scene was produced, i.e. brought on screen, by the viewer, he is forced into connecting to the image it replaces\an act of association, rather than spatio-temporal suturing. In Sonata the mental act is reinforced by two strategies:

1. an automatic return to the previous image, so that it seems that the image produced by the viewer interaction is a temporary interruption of a continuing logic;

2. audio continuity: the sound from the first image continues through the interruption, which reinforces the impression that the viewer's actions are disturbing the natural flow, thus demanding that a sense be made of the new complex.

In the environment created by this structure, duration becomes variable, not fixed. Though the plot of the unfolding narrative is not affected by the viewer's interactions, the screen can now contain multiple diegetic times simultaneously, and the viewer quickly becomes accustomed to navigating between them.

Experienced Time, on the other hand, becomes open and indeterminate. At one extreme the viewer can find him- or herself in the extended time-instant of the computer hacker or videogame player compulsively acting on the screen image. It is this semi-hypnotic state that allows the computer programmer to spend twenty-four hours at a stretch in front of a CRT, "jacked in" as the novelist William Gibson puts it. In his fiction Gibson compares the state to that produced by imagined mind-altering drugs. In this mental condition, the user's impact on the screen output is paramount, while awareness of content and interpretive distance are subordinated to action. Videogame environments are often designed to stimulate this condition\the content is minimal and ancillary to the actions of the user, which are immediate and powerful, either floridly destructive and dependent on hand-eye coordination, or effortlessly navigational, most often a combination of the two. Unlike a videogame, however, changes in interactive cinema are driven by plot and consequence, and consequently compulsion will not be the overriding ingredient of the mental state of the viewer. Here the need for evaluation, interpretation and understanding are in the foreground, though the obsessive need to fully explore the narrative space can serve well as an incentive and accompaniment.

Freed from the predicament whereby the apparatus alone dictates the temporal experience, time can now expand or contract based on the extent of the viewer's involvement or attention, no longer only because of the hills, gullies, and plateaus, the changes in elevation of the plot. One can imagine the user of an interactive cinema alternating between compulsive input, loss of self in the flow of the narrative, and a sense of distance and control of onefs own experience of time, as the tides of the story ebb and flow based on onefs own actions on and in it.

The notion of suspense, for example, can be retained but transformed. If the viewer identifies with a character, seeing him as transfixed with horror at one moment and overcome with relief at the next, there may be some hesitation about accessing the cause of his distress. Now a new emotional affect, begins to emerge. "Don't look behind the door!" we wordlessly warn\but now whether the character opens the cellar door is determined by us, and the vacillation, the holding back, related to a particular experience of suspense, will put the viewer, unexpectedly, in a different grip of the screen. The new pull is a hook of agency\whether we have to face the horror that both terrifies and fascinates is now our decision, and in an effective work we will be equally compelled in both directions.

Then What Can the Interactive Cinema Depict?

A person's life consists of a collection of events, the last of which could also change the meaning of the whole, not because it counts more than the previous ones but because once they are included in a life, events are arranged in an order that is not chronological but, rather, corresponds to an inner architecture.

Italo Calvino, Mr. Palomar, tr. William Weaver (New York: Harcourt Brace Jovanovich, 1983/1985) p. 124.

Our worlds are disorderly and disorganized, unrestricted and loose. Strands of perception and inner experience are interwoven with actions that impact on our immediate environment, causing change in our perceptions and generating new experiences. Time advances relentlessly while our consciousness staggers in and out of it, memories of the past intermingling with hopes for the future as we react to events of the present. Lived experience does not parcel itself into linear, closed structures, though we sometimes represent things that way in order to tell stories about ourselves. But autobiographies, like all narratives based on fact, are always at most distortions and at least abbreviations, omitting many events while inflating others. A complete recounting of the most minor experience (including the mental activity that accompanied it) would last much longer than the experience itself. We compress, excerpt, exclude, and reorganize when we tell stories about ourselves; we must dramatize and deform the facts to fit them into a plotted "story-line" with an ending that provides satisfactory closure. If the interactive cinema is a more faithful rendering of reality, it is precisely because it can bypass some of these criteria of narrative structure. Intermixing and interweaving multiple narrative streams, it can create a meta-narrative sum that is greater than its component parts, if the subject- matter is a match for the potential of the medium. What would be an appropriate model for the subject matter?

We are multi-tasking units. We can whistle and daydream while working, fantasize while having sex, speak the English translation while listening to the German, and so on. And we can switch from one mental activity\one state, one condition\to another, instantly and without effort. It is easy and natural for most people to keep many thoughts and perceptions simultaneously active in their minds, transferring from one to the next at will. One's current inner experience is a conglomeration of perceptions of the present, memories of the past, hopes for and guesses about the future, along with beliefs and fears independent of time markers, dreams, imaginings, pains, etc. Each mental element forms an undercurrent in what is happily called the "stream of consciousness," and navigating these waters is part of what it is to be human. Rushdie's Ocean can be heard in these shells.

How do we move from one mental entity to the next? One thing is certain: it is nothing like making a selection from a list. The "menu" model incorporated in contemporary computer software is aptly named\using it is like negotiating a meal at a fast food restaurant. Switching between streams of mental activity involves responding to hardly perceptible internal and external cues, much as one rides a bicycle around obstacles, keeping balance by slight shifts in position, changing direction by combining such shifts with handlebar adjustments and greater weight adjustments. Except in the least significant cases, we affect things in our lives not by making choices, but by actively responding to situations, with speech acts or in behavior, and equally by silence or inaction. Only in restaurants or department stores are we faced with a closed list of alternatives. The interface of an interactive cinema cannot restrict itself to a model of choice. Response is the operative concept.

Open Issues: What Isn't the Interactive Cinema?

The story so far: the interactive narrative will be in the form of a story space (again the terminology of Michael Joyce and Jay Bolter) laid out for exploration. This story space may consist of a number of related narratives that the viewer forges or discovers links between, or of a single narrative seen from various viewpoints (e.g., of different characters). It may be the breakdown of a particular situation or image or scene into its (non-hierarchical) historical or constitutive elements.

But it will not be a linear story where viewer input determines what-happens-next. Such a structure does not contribute to the notion of interactive form since everything that appears will remain within the limitations of the linear\the fact that a viewer has selected which line the story takes is irrelevant. This particular structure becomes interesting only when the viewer is exposed to different hypothetical situations, so that he can see what would happen if the characters took this turn, that path. Only in this case might the overall experience of the piece retain the quality of a story space of multiple narratives simultaneously present for exploration. Generally, registering response alone will not satisfy the basic requirement that the interactive cinema incorporate (reflect, arouse, respond to) the interest or desire of the spectator.

The bottom line for the interactive cinema is that the viewer has some control over what is on-screen. He or she knows that what is there will change if she or he acts, that it would have been different if he or she had acted differently earlier. Thus the viewer is aware of a fundamental indeterminacy. I have called this epistemological state a subjunctive relationship to the screen\a constant awareness that things could have been otherwise. This state is grounded in the continuing understanding that what is on screen at any moment is a result of the viewerfs interactions\inaction, naturally, counting as decisively as action.

The subjunctive mental state is in direct opposition to the epistemology I identified as essential to a comprehension of the emainstreamf cinema, which depends on a conception of the screen complex as unalterable, the events in the diegesis as inevitable. In an advanced interactive cinema, everything will be in flux, open to the possibility of change\like conversation or competitive sports\and the more sophisticated the system, the more fluid and wide-ranging the possibilities. Awareness of this liquidity has a radical impact on the viewer's relationship to the cinematic material.

Both the structural and the expressive possibilities for the cinema change, expanding in some ways, contracting in others. The details will appear only in the production of new works, not in theory.

A way to examine the issues is to return to the distinction proposed by Maya Deren in the notorious gSymposium on Poetry and the Film,h of 1963 chaired by Amos Vogel and including both Arthur Miller and an obviously inebriated, but nevertheless fairly coherent, Dylan Thomas.vii Deren distinguishes two forms of montage, which she calls ghorizontal and vertical investigations of a situation,h a division that corresponds roughly to that between narrative and lyric forms of poetic structure. Deren argues that narrative film makes use of both of these modes, the vertical when a specific moment in time is investigated and the horizontal when time moves forward along a narrative path. Her presupposition is that the two are separate and distinct, that they cannot occur simultaneously in a film. I will suggest that the interactive cinema allows both modes to occur together.

Sonata is built on this distinction. Story-time is always advancing, both for Lev, the first-person narrator of gThe Kreutzer Sonata,h and for Judith, who recalls her encounter with Holofernes (based on the Apocryphal gBook of Judith.h) However, at any moment, pointing at the screen produces an alternative view of the current juncture of the story, usually in a visual depiction, while the narration continues uninterrupted. The techniques of film montage are evident, in that the sense of continuity is maintained through the interruption elicited by viewer intervention. Vertical investigations include alternative views of the current situation on-screen (for example, an image drawn from the vast database of art-historical images of Judith at various stages in the story), flash-forwards or –back, or commentary from one of the shifting authors (e.g. Tolstoy, on whose novella gThe Kreutzer Sonatah and diaries the piece is based, or the filmmaker, who appears in stained-tooth close-up).

Thus Derenfs vertical is embedded in the horizontal, the lyrical lodged in the narrative, always available at the bidding of the viewer. This story-structure depends on the subjunctivity of the viewerfs perception of the screen. It is necessary that the viewer recognize that behind each element of the screen complex there is a potential set of cinematic data that supports, enriches, subverts, or otherwise amplifies its meaning. Only by viewer action is this meaning realized, but it is always latent. On these grounds, a new relationship of viewer to screen is forged.

Therefore it is incumbent on the interactive cinema artist to find techniques to refresh the viewerfs awareness that his or her actions in relation to a particular image that produces new sounds or pictures. In my judgment, the most immediately available techniques can be found in the language of montage. A deliberate use of recognizable film editing strategies can keep re-convincing the viewer of the specificity of connections between old and new elements, between the elements already there and those produced by viewer action. Once the interactive work has brought the viewer to the idea that actions on the screen complex always contribute to the continuing signification of the work, then the associations can roam more freely than in the city zoo of conventional narrative film. Now the fact that the viewer feels a responsible for the new elements predisposes a comprehension and application of links, associations, and connections that may not have operated in the response to a conventional cinematic work.

Back to the Ocean

"And if you are very, very careful, or very, very highly skilled, you can dip a cup into the Ocean," Iff told Haroun, "like so", and here he produced a little golden cup from another of his waistcoat pockets, "and you can fill it with water from a single, pure Stream of Story, like so", as he did precisely that, "and then you can offer it to a young fellow who's feeling blue, so that the magic of the story can restore his spirits. Go on now; knock it back, have a swig, do yourself a favour," Iff concluded. "Guaranteed to make you feel A-number- one."

Salman Rushdie, op. cit., pp. 71-72.

So now we have two models of potential structure for an interactive cinema: one drawn from a classical text by the father of psychoanalysis, the other from an introspective view of the mind at work. There are many literary precedents for both models. Furthermore, a number of recent works of fiction have plot structures that eminently suit the form of an interactive cinema, either in that they involve the unpacking of a given image or scene into its underlying components\ Graham Swift's Waterland provides several excellent instances\or that their narrative consists of the meeting point of a number of interrelated themes\John Barth's Tidewater Tales and his masterful The Last Voyage of Somebody the Sailor both follow this pattern. More recent examples are Graham Swiftfs Last Orders, a novel in which the person narrator changes from chapter to chapter, John Barth's On With the Story and Coming Soon, and both of which require of the reader an attention split among many chapters simultaneously in order to make sense of the story.

Tying Up & Moving On

There are still, there will always be, loose ends. Given that narrative is imposition of order on chaos, intrusion of form on the formless, and that the order, the form, the logic narrative imposes is of time and sequence, sequence in time, we must now ask again whether we can retain narrative when we abandon endings, when we are entangled in an endless middle.viii A catchphrase often used by theorists to describe narrative is "the illusion of sequence," but in Freud's conception of dream interpretation we can see how sequence can be abandoned without losing the narrative thread. Freud's understanding of dream-structure is an alternative to the Aristotelian model, not only because the components can appear in any order, but also because the story is never over, the analysis is always incomplete, there are always more biographical details to uncover. In an Interactive Cinema, where the desire for closure can also be more or less overcome, the viewer continues to explore the narrative space until he considers it exhausted. There is no totality, there is only withdrawal.

And yet. And yet.



Real Time cannot be trashed with the need for closure like potato peels or an old jalopy. There is, there always will be, a Beginning, an End to a viewer's exposure to an Interactive Cinema work, and a Time Between. She walks up to the device, she interacts with it, she walks away. He walks up, sits down, stays a while, gets up. Do we place a viewer with an interactive work until it starts to repeat on him like rote learning or yesterday's overspiced entree? The Interactive Cinema will succeed only if, in retrospect, the experience seems substantial.

All and any loose narrative ends will never be knotted; this is one of the features (i.e. not bugs) of interactive cinema. If a viewer navigates through a mass of material, some of it will be seen and some won't, and surely some of what isn't seen earlier will raise issues that remain unresolved in what is seen later. But a system can be sensitized to repetition, either so as to avoid it, or so that as soon as repetition starts the viewer is offered the opportunity to enter a structurally different region, a territory of culmination or summary. In general terms, a map of ground covered can be kept by the system, and once a certain area has been explored, closure possibilities can be introduced.

In The Erl King (1983-86), after certain segments have been repeated, a box with the work "END?" appears on the screen. If this box is touched, it produces a mildly interactive segment that starts with images of a few key production crew members touching the inside of the video screen from within the monitor, followed by a rapid series of production stills. A viewer can switch on or off two cardinal text overlays\passages from Wittgenstein and Baudrillard that describe something of the theoretical underpinnings of the work--by touching different areas of the screen.

Sonata (1991/93) reserves two narrative segments that are acknowledged and indicated throughout the piece. If the viewer perseveres, following a story through to one of its climactic moments, the reward will be one of the two culminating murder scenes, one decorated with the blooming image of a blood-fountain, the other with the voluptuous sounds of a blade severing flesh and splintering bone. The possibility of viewing these scenes emerges when the viewer has covered a certain amount of the narrative ground of the piece. And after the murder the work ends or, more precisely, returns to the beginning.

Frames (1999) has internal endings, in the form of erewardsf for successful completion of a task. A viewer attempts to transform a contemporary actor into a 19th century mental asylum inmate, a Mad Woman, based on the first photographs taken in a mental institution (by Hugh Diamond in the 1850s when photography was in its teens). When the viewer succeeds in fully transforming his or her actor into character (there are some obstacles), she will move, in character, to the center screen, and enact a small drama. If two viewers succeed together, their characters will meet on the third screen. After the drama is over, the piece returns to its opening screens.

All this is to say that despite its need for an open narrative, closure cannot be banished from the interactive cinema. Remove the imminence of closure and we begin to drain cinema of desire. Closure must be recast in a more radical mold.

The most fruitful possibility for me, based on my commitment to multiplying and intermingling narratives, is that several story lines continue until one, some, or all of them end. Here the idea is that numerous diegetic times are constantly flowing forward, many narratives operating in time simultaneously independently of which one the viewer encounters. Narrative time in this model keeps moving on. This provides another form for the interactive fiction cinema, a picture of multiple narrative streams not interconnected by a central image, theme or scene. The viewer navigates from one current to an adjacent one in a constantly flowing river, crossing between streams of story at moments of similarity or juncture. Or, to descend one level more, we might rather think of the narrative streams as potential, elements themselves unformed or chaotic, but taking form as they intersect, gaining meaning in relation to one another.

In Sonata I attempt this by juxtaposing the stories of Podsnyeshev (the anti-hero wife-killer in Tolstoy's Kreutzer Sonata) and Judith (the Apocryphal heroine who decapitated the enemy general Holofernes). Both narratives progress, but it is at their connections, where the viewer can cross from one to the other, that they come into focus and take on meaning. A viewer will access an episode of one or the other narrative but not both, and their forms are similar enough that their plot movement can be seen as concurrent. Each of the two killers\Podsnyeshev and Judith\is reflected in the light of the other, since each emerges out of the story context of the other. And thus an act of interpretation is forced on the viewer: the morality of each character comes into question when they are placed in parallel, especially because the former is presented as Evil and the latter as Good.

Without the act of interpretation, the stories are raw and problematic, but when clashed together at the points of interaction, the viewer must take on the role of a judge. As Eisenstein recognized explicitly, Griffith at least implicitly, and Kuleshov claimed as his own, the force of a cinematic element is determined by its context\in the multi-linear narrative interactive cinema, context is in constant flux, the elements appearing always different as their surroundings shift.

As the viewer is drawn in by the act of interpretation, now the magnetic attraction of the interactive cinema can be felt, and the question of the expansion and contraction of experienced time finally addressed; for it is here that the hacker mindset takes over\as we jack into William Gibson's cyberspace. Umberto Eco describes the state somewhat more suggestively than Gibson, though Eco is talking about the travels of a steel ball around an electric pinball surface, not a sprite in a graphic representation of a data environment.

"You don't play pinball just with your hands, you play it with your groin too. The pinball problem is not to stop the ball before it's swallowed by the mouth at the bottom, or to kick it back to midfield like a halfback. The problem is to make it stay up where the lighted targets are more numerous and have it bounce from one to another, wandering, confused, delirious, but still a free agent. And you achieve this not by jolting the ball but by transmitting vibrations to the case, the frame, but gently, so the machine won't catch on and say Tilt. You can do it only with the groin, or with a play of the hips that makes the groin not so much bump, as slither, keeping you on this side of an orgasm."

Umberto Eco: Foucault's Pendulum (Great Britain: Martin Secker and Warburg, 1989), p. 222.

More appropriate to the times than Eco's pinball machine is a game like Tetris, in which the player arranges falling shapes into an unbroken plane, a theatre of geometry and spatial anticipation often played on long airplane flights\as the time sense is held in abeyance, the magnified time of the cramped Atlantic crossing is compressed into a single moment of hypnotic focus. Tetris's hook of involvement is the desire for closure, for the completion of the pattern, an end that is always attainable but just out of reach, like Eco's "brink of orgasm." It is in this space that the machine absorbs time, providing in its place the never-quite-fulfilled promise of consummation.ix

Random Access and the Digital

The interactive cinema structures I am proposing require a specific quality of the moving image streams that constitute them. The system must be able to access any frame of any cinematic stream and display the stream, from that frame forward, as a flow of moving pictures. The cinematic apparatus, the concatenation of hardware with its software armature, must allow leaps from anywhere to anywhere in the moving picture database, ideally in no time. Random access to the atomic cinematic data, i.e. the individual frames, is, in other words, the primary requirement. It must be noted that digitization is neither a sufficient nor a necessary condition of random access. The laser videodisc, the technology on which all my early works are built, is an analog device: video is encoded using infinitely variable values, not a two-valued edigitalf system. However, the achievement of laser disc technology is that its video frames can be accessed and played back in any sequence. It may take up to a second to get from one frame to another on the disc, but all the frames in between are not traversed en route, as is required with videotape or film media. In this way laser video disc is a random access, analog medium\encoding and storage are distinct processes and techniques.

In 2003, John Hanhardt and Jon Ippolito, curators at the Guggenheim Museum, proposed employing The Erl King as a test case for the concept of preservation for media art that the museum had developed. The key term in their gVariable Media Programhx is emulation, the idea being that physical devices are emulated in the form of software structures independent of specific manufacturers or technologies, which will become obsolete because of both advances in technology and the requirements of the market-driven economy. However, eliminating all products is far from straightforward and may not be possible, since software development packages require operating systems (which are usually proprietary), and tend to exploit the attributes of specific chips and other hardware. But emulation of hardware on open source software is an ideal that the Variable Media Initiative is committed to pursuing, despite (or possibly because of) the pursuitfs utopian underpinnings. And naturally it was a program I was thrilled to participate in, since the hardware that The Erl King depended on was twenty years old and had been off the market for almost as long.

The project was to reconstruct The Erl King as software on a single computer\replacing physical video players, switchers, and other hardware devices with simulations that could access and sequence data elements and present them as motion images on screen. The programmer Isaac Dimitrovsky developed a scheme in which the original Erl King glimosineh source code was installed on a contemporary computer and interpreted line-by-line by a program written in the Java programming language. Video frames are encoded as images, which are presented on screen, 24 per second, in a simulation of cinema. Contemporary CPUs and data busses are at last fast enough to handle video in a way parallel to laser video disc technology. Since each frame is individually encoded, access to it is random, but now the frames are digital data, each frame a single image file, rather than a set of variable physical pits engraved in a groove of a twelve inch disc. And since access to files on a hard drive is almost instantaneous, the worst-case one second delay of the video disc player is eliminated.

However, this apparent advantage of minimal access time turned out to be somewhat of a liability. In developing The Erl King, we were very much aware of the inherent lags and hesitations of the system, the friction and slippages caused by the physics of real world gears and bearings. Like the prized delay between pressing a key and hearing the note in church pipe organs, these infelicities became something we exploited as the piece developed. The timings and rhythms of the video flow depend on the egrainf of the system. The physical limitations of the components of the early 1980s were embedded in the program of The Erl King to such an extent that they became determinants of the way it produced meaning.

But the very first version of the emulated Erl King had no delays. When I touched the screen, the upcoming video appeared and began playing immediately. It was so fast that one could not believe that onefs action had had an effect on the system, and the power and complexity of the piece dissolved into an arbitrary porridge with no distinction between viewer-caused changes and those built in. Isaac Dimitrovsky painstakingly worked on replicating, in computer code, exactly the right balance of delays and pauses, distinguishing between those caused by disc search time (which depend on the distance the focusing head has to travel to find the requested frame), and the time required for the computer to communicate with the laser disc players and the touch screen.

It is an irony that, in order for the emulation to retain a meaning that had a relationship to the original, the very limitations of the original had to be reproduced by software, effectively making the 2004 system appear less efficient, and undercutting some of the advantages and potential of the update.

There is a sense in which this point can be generalized. Timothy Binkley argues strenuously that the computer is not a medium or tool,xi by which he means that it does not have inherent characteristics that can be explored and exposed in kind of modernist gesture. The computer is a chameleon, the universal tool whose qualities can be defined and redefined from project to project and from use to use. The granularities that we depended on so heavily for The Erl King are material properties of film, video, and the laser disc player. When everything becomes software, qualities are indeterminate, undefined until fixed by code. This is simultaneously the liberation and the burden of computer-based art.


Another way of describing the double-edge, the shackles and open spaces simultaneously generated in the conversion to digital, is in the understanding that the notion of the final or autograph text is an anachronism. This is the third published version of this essay. Ifd like to add that it is the last, but of course I also believed that about the first two. Digital technologies facilitate change in places and at stages of production where previously it was highly problematic and consequently exorbitantly expensive. Even after a text has been printed, editorial changes are easily managed, movable type being a long outdated technology, replaced by digital plates output at the touch of a key. Zizek has pointed out that there is no longer such a thing as a emasterf text.xii So I may be disappointed in my desire for this to be the master version of this essay.

My interest in interactivity is tied to the notion that flux is among the few defining characteristics of the digital. I am seeking a cinema that encompasses variability while retaining cinematic power, a cinema based first on random access to its atomic elements, and now on their digitization. The ironic aspect of the edigital emulationf of The Erl King is that its digitization undercuts its aura (in the Benjaminian sense of signs of the artistfs labor emarkingf the medium). Qualities that were inherent to the base media can be peeled away when the base media are no more than simulations, so that these qualities have to be replicated piece by piece, rather than worked with and against as currents or resistances in a physical world. There are no qualities inherent to the digital: the interplay between artistfs desire and friction-encumbered media vanishes when every friction and abrasion is deliberately determined, placed there by the artist.

One component of The Erl King was digital from the beginning. This is the routing dataset that defines the possible paths through the audio-video streams. Each time the piece was exhibited, we would revise and add to the navigation database, using the authoring component of the software. Thus, not only did no two viewers ever see exactly the same sequence of images, each exhibition (considered as the sum total of all possible paths through the audio-visual database) was a different version of the piece. However, the cinematic material, imprinted on the set of laser discs, remained the same throughout the multiple iterations.

Now that the cinematic database is digital, it can also be effortlessly added to, or subtracted from. It is as simple to add an image or a sequence to the database as to change a pathway through the database, using exactly the same data management techniques. Why would an artist not improve a piece in any way he could if it was about to be exhibited again? Digital emulation has removed the foundations. The piece can evolve like a city, until an archaeological study would be required to reconstruct the original, through traces unearthed; except that, unlike the physical world, changes of digital data leave few remains. Had The Erl King been produced as a 100% digital work (impossible in 1983), the idea of the an original would be meaningless. Only the physical videodisc media, encoded with images and sounds, anchor the work. The Erl King, Sonata, and Frames may be the last interactive cinema works for which the notion of an original is tenable. From now on, there will be only versions.

I plan to move on. But at the same time I constantly look back. In life we can look back in anger or regret or satisfaction, we can misremember or lie to ourselves based on our desires or our fears, but we cannot alter the past. However, when we enter the realm of the digital, change will always be an option, eliminated only when data is burned, encoded, imprinted, etched, painted, chipped, engraved, cut, carved, or bitten into a non-reversible medium.

i Michael Joyce and Jay Bolter developed the software application gStoryspaceh which is essentially a hypertext word processor. Bolter theorizes the software and the general notion of hypertext writing in his book Writing Space: The Computer, Hypertext, and the History of Writing[Hillsdale, New Jersey, Hove and London: Lawrence Erlbaum Associates, 1991]

ii In his illuminating and expansive book Reading for the Plot: Design and Intention in Narrative [Cambridge, Massachusetts and London, England: Harvard University Press, 1984. 2nd paperback edition 2002], Peter Brooks refers to Freudfs acknowledgement that the Wolf Manfs vision of the primal scene might have been either phantasy or recollection\its historical truth-status is irrelevant to the effect of the mental image on the subject. This of course opens the possibility of the excision of origins and a collapse into endless deferrals, an organizational paradigm developed in detail by Derrida (e.g. as gthe chain of supplementsh). This conception is useful for interactive works that depend on links, since each link can lead to another, eventually giving the impression of boundlessness, though in practice, of course, repetition is ultimately inevitable: I rely on the structure of one link leading to another in the Wolf Man section of Sonata.

Brooksf book was recommended to me by the editor of this volume, Marsha Kinder, on the occasion of my revising the present essay for this context, and I have found it extraordinarily useful. In some cases it confirms and fixes my groping ideas, in others it has forced me to rethink the more naïve presuppositions I was making about the place of plot and structure in narrative. I will refer to the book in footnotes throughout; here Ifd like to acknowledge my gratitude to Kinder for her gentle suggestion that I might find Brooksf work helpful.

iii Brooksf focus on the relationship between readerfs desire and pleasure in the text leads him to propose a dynamic analysis of narrative, an analysis which foregrounds plot and its multiple temporalities. Thus the beginning of the classical narrative text engages the reader in the text by breaching the question of how the situation(s) set up in opening scenes will be resolved, i.e. eliciting a desire for an ending. As Brooks puts it: g... the end writes the beginning and shapes the middle ...h [p. 22]. Interestingly, Brooks sees the structures of the classical narrative reprised in the mainstream cinema of the 20th century, while it is challenged in serious 20th century fiction. Since the subject of the present essay is an interactivity of cinema, Brooks e analyses are particularly useful, though I will propose that the interactive cinema has much to learn from the contemporary novel. The classical narrativefs insistent desire for resolution is immediately apparent , for example, as the basis of the interactivity of computer games, the most recognized and financially successful dynamic interactive works so far, and this same wish to move forward and settle will be the lure for the viewer of the interactive cinema.

iv Brooksf argument for the essential temporality of narrative structure folds into my proposal for a subjunctivity of the elements in the interactive work. If a narrative is endowed with meaning retrospectively, it is the desire for the ending that maintains the readerfs interest in the text. In this model, the middle of a narrative becomes a mobile interplay of variation: grepetition as binding works toward the generation of significance, toward recognition and the retrospective illumination that will allow us to grasp the text as total metaphor, but not therefore to discount the metonymies that have led to it ch Reading for the Plot, p. 108. Here we can imagine the viewer replacing one story-element by a structural equivalent, each replacement painting the current story-situation in a different light. Brooks describes the middle as gthe place of repetitions, oscillating between blindness and recognition, between origin and ending.h [p. 108] For the reader to have the power to see a story-moment in a different way, from an alternate angle, or through another characterfs eyes, can recharge this oscillation and extend its authority, an authority that, in the best circumstances, will extend to the experience of the whole work, invigorating the desire for the end and at the same time transforming it.

v Mieke Bal, Narratology [Toronto: University of Toronto Press, 1997]

vi See film editor Walter Murchfs illuminating discussion of the significance and heuristics of this disruptiveness in In the Blink of an Eye: A Perspective on Film Editing, 2nd edition [Los Angeles: Silman-James Press, 1995 & 2001]; also Sean Cubittfs extensive analysis of the cut as a determinant of cinematic meaning in The Cinema Effect [Cambridge, Massachusetts and London, England: MIT Press, 2004].

viiPoetry and the Film: A Symposium with Maya Deren, Arthur Miller, Dylan Thomas, Parker Tyler, Chairman, Willard Maas. Organized by Amos Vogel.,h in P. Adams Sidney (ed.), Film Culture Reader [New York and Washington: Praeger Publishers, 1970]

viii Brooksf insight about the death of the ending is that though the modernist novel eschews the exhaustive ending as a falsehood, the (now frequently frustrated) desire for the ending is still the tie that binds the reader to the plot. gSince ends have ceased to be simple–indeed, on inspection we realize they never were simple–the end that narrative seeks, in its anticipation of retrospection, may disappoint and baffle. Yet this may only make it the more necessary to construct meaning from that end, moving back to recover markings from the past, reconstructing the outposts of meaning along the way. g Reading for the Plot, p. 323.

ix This was written before the emergence of the computer game culture and its conquest of Hollywood. Now (2002) airplanes have games built into screens in the seat backs. But the point about the erasure of the perception of time's passage remains unchanged. I have written about the computer game and its false promises in "Mastery: Sonic Cfest Moi," New Screen Media: Cinema/Art/Narrative, ed. Martin Reiser and Andrea Zapp [London: British Film Institute, 2002]

x See Alain Depocas, Jon Ippolito, and Caitlin Jones (eds.), The Variable Media Approach: Permanence Through Change [New York: Guggenheim Museum Publications, and Quebec: The Daniel Langlois Foundation for Art. Science, and Technology, 2003]. For a detailed description of the process of emulating The Erl King, see Jeff Rothenberg, "Renewing The Erl King" Millennium Film Journal, "Hybrids", Nos. 45/46, Fall 2006.

xi Timothy Binkley. gCamera Fantasia: Computed Visions of Virtual Realities.h Millennium Film Journal Nos. 20/21, New Technology, Fall/Winter 1988-89. I elaborate this argument in gThe PC is a Penguin,h in Bild Medium Kunst ed. Yvonne Spielmann and Gundolf Winter [Muenchen: Fink, 1999] and in gOcean, Database, Recut,h in Database Aesthetics :Art in the Age of Information Overflow, ed. Victoria Vesna {Minneapolis: University of Minnesota Press, 2007].

xii Slavoj Zizek, gCyberspace, Or, The Unbearable Closure of Being,h in The Plague of Fantasies [London, New York: Verso, 1997] p.150 and passim. I discuss this in gMastery: Sonic Cfest Moih op. cit.