Proceedings of the SIGGRAPH 2003 conference on Sketches & applications in conjunction with the 30th annual conference on Computer graphics and interactive techniques - GRAPH '03
Office Voodoo is an interactive film installation using exclusively live action footage and running on a real-time, shot-based editing engine that fluidly assembles the film as it is being watched, while respecting the conventions of continuity editing. Each character in the film is represented by a physical voodoo doll. As viewers manipulate these dolls, they affect the emotions of the people on screen. They can also call the people in the film using their phones. Real-time editing engine In
... editing engine In the tradition of lens-based, non-synthetic cinema, new forms of film for computational media are becoming possible as footage is freed from its inherently linear celluloid substrate : they are nondeterministic and algorithmic films [Manovich 2001] . But trying to achieve generativity and extended interactivity in the purely filmic genre is especially difficult, because the previously shot material is finite ; solving this problem comes about defining techniques by which footage can be recombined in different ways in real time. The approach taken in Office Voodoo is to use time-based multiplexing on a shot-based granularity level, i.e. continuity editing as it has been practiced since Griffith. It relies extensively on the creative use of automated editing [Agamanolis 2001], where the task of the interactive film editor is no longer to define univocally where cuts happen, but i nstead to script the editing rules and define the constraints and freedom that the viewer has in the exploration of the movie. The real-time editing engine works by assembling streams in parallel for each of the different characters (for example shot and reverse shot in our case) and cutting back and forth on a semirandom basis between these streams. It does so while avoiding jump cuts and insuring sound continuity, accomplishing L-cuts by simulating look-aheads in the streams. Shots that make up the streams are selected from an annotated clip database in real-time so that they match the desired turn of the story. This process makes use of loops, repetition, and footage repurposing : a single shot is re-used in multiple contexts, in a successful application of the Kulashov effect. Emotion-driven narrative Office Voodoo is an interactive sitcom featuring Frank and Nancy, two bored 30-something officemates, condemned to spend their lives in an office, in the vein of a Sartrian huis clos. The piece explores the influence of emotions as initial conditions in any social interaction; set as a satirical simulation of office life, it works as a social laboratory where it is like injecting drugs into the protagonists and seeing what would happen... if Frank is cranky and Nancy is depressed ? If Frank is lethargic and Nancy is flirtatious ? The underlying narrative structure of this infinite film is emotionand character-driven : each character has an emotional state (euphoric, depressed, agitated, mellow..) that is represented as x-y coordinates in a two-dimensional valence/arousal emotional space as used by Shlosberg (1952). As a viewer shakes and squeezes a doll, he's changing the corresponding character's emotional state, thereby navigating this space. Therefore e ach particular experience of the film can be described as a trajectory across these emotional spaces. Finally, for certain emotional combinations of the two characters, pre-edited stories are triggered, starting a short non-interactive sequence. The semantics used to annotate the shots are essentially based on emotional state, type of shot and character. Interface design To control the film, custom tangible interfaces were designed : voodoo dolls representing the characters in the film [Johnson et al. 1999 ]. Force-sensitive resistors and accelerometers placed in the dolls measure activity, while vibrating motors and flashing LEDs provide visual and haptic feedback. Microcontrollers perform simple signal processing on the inputs and then relay the data to the real-time editing engine on the main CPU. The other interface is the phone : viewers have the office numbers of Frank and Nancy and can call them at any time. A two-line voice server picks up the calls and signals the editing engine. Phones and voodoo dolls were conceived as very credible, intuitive interfaces bridging the real world and the fictional world. As the installation was shown in festivals, we were pleased to notice that they actually reinforce the suspension of disbelief and accentuate the illusion of life of the characters in the film. By their nature, they also allow continuous intervention of the viewers, at any time during the film. Office Voodoo demonstrates that it is possible to craft compelling interactive narratives that only rely on live action footage, thus benefitting of the aesthetics of cinematography and unsynthesizable subtlety of real actors.