Canvas + Triggers programs - Armadillo Dance Project

Canvas + Triggers: Programs

Canvas is a movement-interactive video installation run off a laptop computer. We call the underlying computer program Painter because it creates video frames similar to abstract expressionist paintings. First used publicly at the February 2009 Phoenix Experimental Arts Festival in our piece Red Bites for a Rainy Day, Painter is a Max 5 software program that uses a video camera to target either (a) a previously set color or (b) any movement. In each frame of video the target pixels are painted a new color, which changes gradually from frame to frame in a random walk through the video color cube, and the paint accumulates on the screen. A separate parameter controls how quickly the paint fades (to either black or white); if the target color is not present, or if there is no movement, the canvas will gradually go blank.

In performance, the target (either color or movement), the range of color used for painting, and the duration of fade time change on cues. In installation, Painter targets movement only, while color range and fade time reset periodically, thus varying and refreshing Canvas's response to motion.

Canvas, being visual, remains connected to its observers even in an environment rich with sound--for instance in an echoey lobby filled with noisy theatre goers. If the installation space is quieter we can set up Canvas + Triggers, which combines Painter with a few of our audio-producing programs that we've used in dances.

The earliest of these programs comes from the dances Playing with Liquid Mercury and Playing with Liquid Mercury II, and we may as well call it Double Keyboard because it essentially turns each half of the stage into a monophonic MIDI keyboard. Double Keyboard splits video input from a camera into fifty vertical strips which are associated with pitches; the left twenty-five increase in pitch from left to right like a piano, but the right twenty-five decrease in pitch from left to right. Double Keyboard finds the strip on each half of every video frame that shows the most change from the previous video frame, and plays the notes associated with those strips at a volume proportional to how much change has occurred--i.e. the two halves of the stage are independent musical instruments that play louder when there's more movement and softer when there's less.

Like Painter, Double Keyboard is a purely reactive program: when the computer senses movement it immediately and directly reacts, so that the dancer(s) can potentially control the computer output in at least some of its particulars--though not all. In performance Double Keyboard varies the type of sound it produces (piano, percussion, strings) when cued by the computer operator; in installation it produces swishes only, as heard in the video to the right. The swishy sounds work with movement improvisation better than musical pitches do, probably because the "swishes" merge together more than discrete musical notes do, and so feel more like the continuous motion of a moving body.

Canvas + Triggers includes two more programs, from the dance Broken Chase, which have a less direct relationship with the movement they track. Both divide the video frame into fifty vertical strips like Double Keyboard, but the strips are not used to generate pitches. The simpler of the two programs is Footsteps which outputs ... guess what! Footsteps is anti-reactive in the sense that it does the opposite of what Painter and Double Keyboard do; when it senses movement, Footsteps does nothing, but when it senses stillness, Footsteps produces the sound of footsteps. The speed and pitch of the steps is directly related to how much movement the program Footsteps detects just before the stillness begins. The sound of the footsteps pans back and forth across the stereo audio space, the speed of the panning determined by how quickly the filmed movement source traveled across the fifty vertical strips of video before stopping and triggering the sound.

While Painter, Double Keyboard, and Footsteps essentially do nothing in the absence of motion (or color if Painter is targeting color), the last program has a certain life of its own. Agent is a program that plays a somewhat wistful sounding constantly evolving melody; when linked to a piano sound it comes off a bit as an intensely contemplative jazz pianist. In Canvas (as in Broken Chase), Agent responds to movement and stillness, but while Footsteps simply reacts to these triggers Agent acts in ways that cannot be controlled by the dancer(s).

The model of interactivity used in designing the Agent program's responses is that of human conversation. Agent detects when the dancer is speaking (moving) and when the dancer is silent (not moving). As seen in the excerpt from Broken Chase to the right (in which Agent is playing over a set percussion track), generally Agent will reply immediately upon noticing the dancer is silent, but sometimes it will wait a few seconds, thinking about what it is going to say. If the dancer chooses to interrupt Agent by beginning to move before it finishes playing, Agent will probably allow the interruption and stop playing; however, occasionally Agent will doggedly continue on regardless of the dancer's beginning to move. Likewise, the Agent may choose to interrupt the dancer's talking, but usually it will respectfully listen to all the dancer has to say before replying. Thus the dancer must always be ready to respond to the Agent program's behavior in ways that are not necessary with the other programs operating in Canvas + Triggers. Agent is not controllable, so the relationship between dancer and Agent is interactive and not simply reactive.

In Canvas + Triggers the video program Painter is always active, while the audio programs Double Keyboard, Footsteps, and Agent regularly loop through a sequence in which sometimes only one accompanies the video while at other times a combination of two or three may do so. Thus the audiovisual environment transforms from one in which audio and video (and movement) occur together to another where they do not. Each amalgamation has its own character and inspires distinct movement strategies in its human interlocutors, whose immersive experience is repeatedly renewed by the cycle of variations.