II

Parameters of Perception: Vision, Audition, and Twentieth-Century Music and Dance

November 2-5, 2006, Tempe, Arizona

Recent experimental psychological research on visual perception, auditory perception, and cross-modal perception has shed light on how these processes differ, and how the relations between visual and auditory stimuli shade our understanding of the events perceived. This work offers a possible way into considering the question of how music and dance “go together” or not, and particularly may shed light on the unusual twentieth-century human behavior of NOT having music and dance “go together.” Our paper presents relevant research in perception, examines factors contributing to the separation of perceptual modalities that has often appeared in twentieth-century dance, and discusses the separation in terms of the specific behaviors of dancing and looking at dance.

We come to this paper with a practical interest in what makes music and dance go together, or not, in that one of us choreographs and one of us composes and we share a desire to make computer-interactive pieces. In such pieces the connection between movement and sound must be examined explicitly, and this has led us to explore the literature from experimental psychology relating sound to movement and attempt to connect it to the making and viewing of dances.

We start by trying to distinguish between cases where dance and music are seen to “go together” and those where they aren’t. The latter appear to be unusual in human history. Indeed one standard explanation for the evolution of music and dance hypothesizes that they contribute to group cohesion (McNeill 1995, Cross 1999, Brown 2001, Hagen and Bryant 2003), and perhaps the traditional cohesion of music and dance not only reflects but is constitutive of that function.[1] Many languages do not distinguish between dance and music but have one word denoting both together (Besson and Schön 2001).[2] All peoples have some dance with music, and usually dance and music occur in tandem and intertwine (Hanna 1982, 60-61; Brown 1991, 140).[3]

But what is it about dance and music that makes them go together? Or more precisely, from the observer’s standpoint, in what ways does a dance and its accompaniment seem to fit each other? In this paper we consider three sound-motion phenomena that contribute to answers to these questions.[4] First we look briefly at simple congruence, where similarities or alignments are perceived between the two media (Krumhansl and Schenck 1997, Mitchell and Gallaher 2001). Then we turn to various types of “capture,” where similarities and alignments are not seen until the media are presented together (Repp and Penel 2002; Sekuler, Sekuler and Lau 1997; Watanabe and Shimojo 2005). And finally we consider virtual motion, the movement which is felt or imagined when sound is heard.[5]

Most immediately relevant to music and dance is the idea of congruence. In a 1997 study, Carol Krumhansl and Diana Schenck used Balanchine’s choreography set to Mozart’s Divertimento No. 15 to specifically address whether dance could reflect the structural and expressive qualities of music. Their participants were asked to 1) watch the choreography in silence, 2) listen to the music without seeing the dance, or 3) watch the dance and listen to the music together. All three groups were then asked to rate the tension and emotion throughout the piece as well as indicate the beginnings and endings of phrases. Results showed that all three groups concurred in their judgments and that the response to the “both music and dance” condition could be predicted as a combination of the responses to the “music only” and “dance only” conditions. Krumhansl and Schenck, following the ideas of Paul Hodgins (1992, 25-30), who proposed a detailed paradigm for choreomusical analysis, attribute the parallel responses to parallel elements in the dance and music, including structural organizations, rhythms, dynamics, qualities of motion, and tempi.[6]

Robert Mitchell and Matthew Gallaher’s 2001 study examined the ability to detect a “match” between a piece of music and choreography intended to go with it. The musical pieces were by John Cage, Peter Gabriel, and David Lanz, and the dances were by Mitchell and Gallaher’s colleague Marianne McAdams.[7] Participants were presented with a piece of choreography in silence followed by three selections of music. They were then asked to choose the musical selection that best matched the choreography. Participants were also presented with a piece of choreography paired with a piece of music, either intended or not intended to go with it. In both the sequential and simultaneous test format Mitchell and Gallaher’s results show consistently high ratings for perceiving a “match” between a piece of music and choreography intended to go with it. Participants’ answers to a questionnaire indicated that their matches were influenced by similar characteristics perceived in the media, “including emotion, fluidity, an African or Middle Eastern quality, and temporal characteristics such as rhythm and pace.” Again, congruence was found to be a result of similarities perceived in the movement and the sound.

Combining a categorical scheme of Mitchell and Gallaher with the analytical framework of Hodgins yields a short list[8] of parallels between music and dance that may contribute to a sense of congruency:

(1) matching or intertwining pulse[9] or rhythm.

(2) alignment of structural temporal aspects other than pulse that coincide.

(3) analogous cross-modal qualities.

(4) complementary referents.[10]

These results are certainly quite intuitive and predictable, but we are equally interested in examining instances where music and dance are NOT intended to go together. We earlier noted a human tendency towards cohesion in dance and music, yet there has been considerable experimentation in the 20th-century in producing works in which sound and movement seem to be more complementary or even in competition rather than congruent. We would like to briefly consider how that came about before turning to the phenomenon of auditory and visual “capture.”

Claims that music was being used as only a sonic backdrop, not as rhythmic material to which dancers coordinated their steps, can be traced at least back to some of the works of the Ballets Russes and the Parisian avant-garde.[11] In Nijinsky’s L’Après-midi d’un faune (1912) “the score is reduced to background music” (Buckle 1971, 164).[12] In Cocteau’s Le Boeuf sur le Toit (1920), the characters perform “in ‘slow motion,’ against the music” (1955, 16).[13] Similarly silent film could be presented with music that seemed unconnected.[14] As described by Wilfred Mellers (1942, 224), Satie’s minimalist music for the intermission film in Relâche (1924) “provides a background: it nowhere tries to give musical expression to the images of the screen.”[15]

It is in film where a developed theorization of the asynchrony of sound and image first appears in the late 1920s.[16] But in films like Battleship Potemkin (1925) by the best known of film theorists, the Russian Sergei Eisenstein, music and visuals are still closely related, although perhaps in complex ways.[17] In contrast, the theory of the independence of music and movement that has been present in American modern dance since the 1950s, traceable to John Cage and Merce Cunningham, rejects imposing a unity. Cage and Cunningham’s ideas can be seen as descending from the practical and theoretical concerns of the composer Henry Cowell, who through the 1930s was developing an elastic form of accompaniment in response to American modern dance choreographers’ chafing at dance being slave to music.[18] The process of de-linking dance from music was finally brought to its logical endpoint with Cage composing indeterminate scores that shared only a common duration with the dance, so that dancers, as Cunningham relates, “could not count on the sounds as cues,” and “had to rely on [their] own dance timing to guard the length of any phrase” (1992, 142-143).[19] The Cage/Cunningham approach to dance and music has had continuing influence, directly affecting the choreographers in the Judson Dance Theater and even distantly marking multimedia performance such as that directed by Robert Wilson.[20]

This path to cross-modal incongruence has been influenced by many cultural and historical factors peculiar to western Europe and the United States that might encourage a conceptualization of music and dance as parallel lines of objectively experienced materials. There is the ideology of autonomous listening and viewing associated with the concert hall tradition. In tandem with this is the structural and syntactic emphasis in attending to concert music most ably described and theorized by Leonard Meyer (1956; 1994), which encourages discerning contrapuntal activity and hierarchical simultaneities. Audio recordings and silent films allowed the experiencing of sound and movement as separate events. Finally we note that Cage’s (1961, 109-145) explanatory framework borrowed from Zen Buddhism an acceptance of simultaneity—in particular that of any sound with any movement.[21]

But in practice multimodal incongruence seems difficult to achieve. While sound and movement may be produced independently in a Cunningham dance, their reception is hard to keep separate. In our experience of watching his work, not only does each medium have “hooks” that “grab” elements of the other medium to create coincidental events, but qualities of the music and the dance often seem complementary, something occasionally also true of their referential meanings. Each instance of coincident sound and movement is perhaps experienced as a perceptual unity akin to the McGurk Effect, a classic form of what is called “capture,” to which we now turn.

The McGurk Effect (McGurk and MacDonald 1976) is what happens when a visual image of a person saying “ba” is paired with an auditory track of someone saying “ga,” resulting in a composite perception of seeing and hearing a person say “da.” The significance of the McGurk Effect is that it shows how simultaneous presentation of auditory and visual stimuli capitalizes on a model of optimal combination. The combination is optimal because it combines the information from each medium into a composite interpretation that makes the most sense.

This optimal combination model may extend beyond speech into more abstract representations, as may be seen in the bounce-inducing effect, which can be produced as follows. Viewers are shown two identical discs moving toward each other in a straight line. They meet and then move apart, continuing along the line. Normally they are seen to pass through each other. But if a sound is heard at the time the discs meet, they are instead seen to bounce off each other and return in the direction from which they came. While the bounce-inducing effect can be interpreted as an instance of optimal combination, it is undoubtedly one in which audition dominates. This is called “auditory capture,” for the visual stimulus is perceived differently when combined with the auditory stimulus—the auditory stimulus “captures” the visual. Likewise there may be “visual capture” of auditory stimuli.

Extending the idea of capture to the more complex stimuli that interest us, Scott Lipscomb and Roger Kendall (1994) paired an abstract film excerpt with a variety of different musical accompaniments and found that viewers perceived several musical choices as a “good fit.” Similarly the study by Mitchell and Gallaher (2001) mentioned earlier showed not only that dances intended to match their music were indeed seen to do so, but also that there was a significant perception of matching between dances and music not intended to go together.[22] This is to be expected, for whether or not relationships between music and dance are intended by their creators, viewers will inevitably attempt to construct connections between experiences that are coincident in time according to the Gestalt principles of perception (Sloboda 1985). It has been shown that even when the intention of congruence between the auditory and the visual is removed, congruence can still be perceived and in fact very likely will be perceived (Bolivar, Cohen and Fentress 1994; Iwamiya 1994; Lipscomb and Kendall 1994).[23] As Mitchell and Gallaher conclude, “simultaneous presentation of visual and auditory stimuli (e.g., dance and music) may enhance the experience of a match between them from only a few similarities” (67). This is because viewers actively look for cross-modal perceptual congruences formed by “capture.”

While capture can help explain how some music and dance judged incongruent separately may seem congruent when judged together, there still are combinations of dance and music in which the media remain stubbornly incongruent no matter what. For these cases perhaps a process somewhat opposite to capture may kick in. Roger Copeland argues that the independence of music and dance in Cunningham’s work contributes to a championing of “perceptual freedom”,[24] where multimodal incongruence helps disable our tendency to fuse simultaneous events and helps make us aware of the individual characteristics of sounds and movements, spurring us to attempt to follow them with selective or divided attention so far as we are able.[25] The sort of skills this requires are cultivated for example by listening to polyphonic music, for which the ability to hear multiple lines of music simultaneously pays dividends.[26]^,[27] The analogous problem for dance would be following multiple dancers or multiple groups of dancers.[28]

It is not clear how easily humans may follow independent musical and dancing objects simultaneously, but certainly a byproduct of the attempt to see the parts instead of the whole is to notice details—of individual dancers, of their group dynamics, and of their relationship to the music—without them joining into a unitary experience. The writer Italo Calvino speaks of the contemporary novel as “a network of connections between the events, the people, and the things of the world,” a “multiplication of possibilities” (1988, 105 and 124), and such description may perhaps be applied to dance that moves without regard for its music. When the relationships between modalities become indirect and complex, the dance will seem most familiar to an audience entrained by the multiplicities of our urban, global, networked, multi-tasking world.

But this audience of multi-tasking viewers experiences the dance differently from the dancers. While musicians and their listeners hear approximately the same mix of sound, dancers and their viewers do not see the same field of movement: the dancers are within a changing array while the observer is without. Yet dancing and looking at dance cannot be entirely separate experiences. Ivar Hagendoorn (2004) has recently proposed that the experience of watching dance creates pleasure for us through the fulfilling and undermining of our expectations. The prediction mechanism that drives the expectations is our kinesthetic response to movement[29]—an embodied response gaining experimental credence due to accumulating evidence that the brain pathways for action and perception are the same. While the dancer dances on stage, the viewers to some extent sense the dancer’s movement as if they were doing it themselves.

Hagendoorn’s proposal of a neurobiological system for a kinesthetic response to dance aligns with the musicologist Charles Keil’s notion of “engendered feeling.” Keil associates “engendered feeling” especially with the motor drive prominent in many Western and non-Western popular musics—the “groove” or “swing” which is linked strongly to dance (1966).[30] While Keil’s “groove” is felt as repetitive movement in music (and dance), and Hagendoorn’s kinesthetic response is felt as actions by the body of the dancer (in the body of the spectator), they are linked by the perception of feeling movement. Music psychologist Eric Clarke (2005, 62-90) argues further that human beings have a tendency to hear movement not just in “groove” music but in all music, movement which may be felt as self-motion or, especially in highly polyphonic music, as the multiple motions of external agents.[31] Given all the felt movement induced by hearing music and watching dance, perhaps the congruence or incongruence of music and dance is sensed by the audience as the congruence or incongruence of the virtual motion of the music and the virtual motion of their kinesthetic response to the dancing.

The virtual motion evoked by music, however, has been found to be asymmetrical, sometimes counterintuitive, and sometimes untrue to the physical properties of sound in the real world. In a recent study Zohar Eitan and Roni Granot (2006) asked people to visualize the movement of an animated human character in response to various musical selections. Participants’ responses clearly show that the fact that a musical stimulus seems to suggest a particular kinetic quality does not imply that an opposing musical stimulus suggests the opposite kinetic quality. For many people, for example, when pitch ascended their characters moved faster, but when pitch descended their characters did not move slower; however, when pitch descended, their characters moved to the left, but when it ascended, the characters did not move right. Thus the overall picture emerging from this study shows that the perceptual relationship between sound and motion can be far more nuanced than entrenched analogies and direct one to one mappings suggest.

One other interesting characteristic of music’s virtual motion is that it may be still. Clarke, in arguing that all music implies motion, includes the limiting possibility that the motion specified by sound could be none—that is, there may be stasis.[32] His example (210 n. 7) is LaMonte Young’s Composition 1960 no. 7, which consists of a perfect fifth on two strings played, as directed by the composer, “for a very long time.” Much of the music of Cage, his colleagues, and other collaborators of Cunningham, falls close to this example, allowing us to propose that what Cage and Cunningham discovered was not so much the independence of music and dance, but rather the complete congruence of the real motion of dance with the virtual stillness of random sounds. Cage created music as environment for dance, not as partner in movement. This would imply that, for dances set to Cageian soundscapes, the audience is NOT following independent musical and dancing objects simultaneously, but following a dancing figure or figures against a sonic ground.

While such music implies stillness, it still requires movement for its production—at least it did until the twentieth century. Music requires the movement of musicians, but the invention of sound recording broke that intimate connection. Recorded sound limits the interaction of music and dance, for while dancers may still respond to music, without musicians the music cannot respond to dance. With the introduction of computer-based interactivity between dancers and music, the connection between movement and sound must once again be made immediate, and furthermore it must now be made explicit, for what might have been intuitive or the result of long experience must now be programmed. What we hope to have shown by examining congruency, capture, and virtual motion is that the relationship between sound and movement is both a close one and a complex one. The connections are not direct; the mapping is not one-to-one. In fact, the way in which an interactive system maps sound onto movement can actually change a viewer’s perception of that movement or focus their attention to a particular quality of that movement. These findings, while relevant to us for their application to computer-interactive dance, are equally of use in the analysis of more traditional dance, where dancers and musicians together create moving performances, capturing the attention of their audiences both artistically and perceptually.

Bibliography

Altman, Rick. “Introduction.” Yale French Studies 60 (1980, Cinema/Sound): 3-15.

Arnheim, Rudolf. Film As Art. Berkeley: U of California P, 1957.

Arom, Simha. African Polyphony and Polyrhythm: Musical Structure and Methodology. Cambridge: Cambridge U P, 1991.

---. Liner notes to the LP Central African Republic: Banda Polyphony. Philips, 1976. Rpt. as CD. Auvidis-UNESCO, 1999.

---. “Prolegomena to a biomusicology.” Ch. 2 in The Origins of Music. Ed. Nils L. Wallin, Björn Merker and Steven Brown. Cambridge: MIT P, 2000. 27-29.

Balkwill, Laura-Lee and William Forde Thompson. “A Cross-Cultural Investigation of the Perception of Emotion in Music: Psychophysical and Cultural Cues.” Music Perception 17.1 (1999): 43-64.

Balkwill, Laure-Lee, William Forde Thompson and Rie Matsunaga. “Recognition of Emotion in Japanese, Western, and Hindustani Music by Japanese Listeners.” Japanese Psychological Research 46.4 (2004): 337-349.

Banes, Sally. “Dancing [with/to/before/on/in/over/after/against/away from/without] the Music: Vicissitudes of Collaboration in American Postmodern Choreography.” Choreography and Dance 1.4 (1992): 3-22. Rpt. in Writing Dancing in the Age of Postmodernism. Hanover: Wesleyan U P, 1994. 310-326.

---. Democracy’s Body: Judson Dance Theater, 1962-1964. Ann Arbor: UMI Research P, 1983.

Batson, Charles R. Dance, Desire, and Anxiety in Early Twentieth-Century French Theater: Playing Identities. Burlington: Ashgate, 2005.

Besson, Mireille and Daniele Schön. “Comparison between Language and Music.” Annals of the New York Academy of Sciences 930 (2001): 232-258.

Bigand, Emmanuel, Stephen McAdams and S. Forêt. “Divided Attention in Music.” International Journal of Psychology 35.6 (2000): 270-278.

Blacking, John. “The Concept of Identity and Folk Concepts of Self: A Venda Case Study.” Identity: Personal and Socio-Cultural. Ed. Anita Jacobson-Widding. Stockholm: Almqvist and Wiksell, 1983. 47-65.

Bolivar, Valerie J., Annabel J. Cohen and John C. Fentress. “Semantic and Formal Congruency in Music and Motion Pictures: Effects on the Interpretation of Visual Action.” Psychomusicology 13 (1994): 28-59.

Bregman, Albert S. Auditory Scene Analysis: The Perceptual Organization of Sound. Cambridge: MIT P, 1990.

Brown, Donald E. Human Universals. New York: McGraw-Hill, 1991.

Brown, Steven. “Evolutionary Models of Music: From Sexual Selection to Group Selection.” Perspectives in Ethology 13 (2001): 231-281.

Buckle, Richard. Nijinsky. New York: Simon and Schuster, 1971.

Cage, John. Silence. Middletown: Wesleyan U P, 1961.

Calvino, Italo. Six Memos for the Next Millennium. Cambridge: Harvard U P, 1988.

Chion, Michel. Audio-Vision: Sound on Screen. Ed. and trans. Claudia Gorbman. New York: Columbia U P, 1994.

Clair, Renè. Cinema, Yesterday and Today. Trans. Stanley Appelbaum; ed. R. C. Dale. New York: Dover, 1972.

---. Reflections on the Cinema. London: William Kimber, 1953.

Clarke, Eric F. “Rhythm and Timing in Music.” The Psychology of Music. 2nd ed., ed. Diana Deutsch. San Diego: Academic P, 1999. 473-500.

---. Ways of Listening: An Ecological Approach to the Perception of Musical Meaning. New York: Oxford U P, 2005.

Cocteau, Jean. A Call to Order. Trans. Rollo H. Meyers. London: Faber and Gwyer, 1926.

---. Théâtre de poche. Monaco: Editions du Rocher, 1955.

Cohen, Annabel J. “Music Cognition and the Cognitive Psychology of Film Structure.” Canadian Psychology 43.4 (2002): 215-232.

Collins, Nick and Fredrik Olofsson. “klipp av: Live Algorithmic Splicing and Audiovisual Event Capture.” Computer Music Journal 30.2 (2006): 8-18.

Cook, Nicholas. Analysing Musical Multimedia. New York: Oxford U P, 1998.

Copeland, Roger. “Beyond Expressionism: Merce Cunningham’s Critique of ‘the Natural’.” Dance History: An Introduction. Ed. Janet Adshead-Lansdale and June Layson. London: Routledge, 1983. 182-197.

---. “Fatal Abstraction.” Dance Theatre Journal 14.1 (1998): 38-42.

---. Merce Cunningham: The Modernizing of Modern Dance. New York: Routledge, 2004.

Cowell, Henry. “How Relate Music and Dance?” The Dance Observer 1.5 (1934), 52-53.

Cross, Ian. “Music, Cognition, Culture, and Evolution.” Annals of the New York Academy of Sciences 930 (1999): 28-42.

Cunningham, Merce. “A Collaborative Process between Music and Dance.” Tri-Quarterly 54 (1982): 173-186. Rpt. in Merce Cunningham: Dancing in Space and Time. Ed. Richard Kostelanetz. Chicago: Chicago Review P, 1992. 138-150.

Dissanayake, Ellen. Art and Intimacy: How the Arts Began. Seattle: U of Washington P, 2000.

Drake-Boyt, Elizabeth M. Dance As a Project of the Early Modern Avant-Garde. PhD dissertation, Florida State U, 2005.

Eitan, Zohar and Roni Y. Granot. “How Music Moves: Musical Parameters and Listeners’ Images of Motion.” Music Perception 23.3 (2006): 221-247.

Flinn, Caryl. Strains of Utopia: Gemder, Nostalgia, and Hollywood Film Music. Princeton: Princeton U P, 1992.

Fogelsanger, Allen. “On the Edges of Music: Trisha Brown’s Set and Reset and Twelve Ton Rose.” Proceedings of Sound Moves: An International Conference on Music and Dance, Roehampton University, London. 2005. 39-46. <http://www.roehampton.ac.uk/soundmoves/SoundMovesConference2005_Proceedings.pdf>.

Friberg, Anders and Johan Sundberg. “Does Music Performance Allude to Locomotion?: A Model of Final Ritardandi Derived from Measurements of Stopping Runners.” Journal of the Acoustical Society of America 105 (1999): 1469-84.

Gabrielsson, Alf. “Adjective Ratings and Dimension Analyses of Auditory Rhythm Patterns.” Scandinavian Journal of Psychology 14.4 (1973): 244-60.

Garafola, Lynn. “Dance, Film, and the Ballets Russes.” Dance Research 16.1 (1998): 3-25.

---. Diaghilev’s Ballets Russes. New York: Oxford U P, 1989.

---. “Diaghilev’s Musical Legacy.” Legacies of Twentieth-Century Dance. Middletown: Wesleyan U P, 2005. 45-53.

Gibson, James Jerome. The Ecological Approach to Visual Perception. Hillsdale: Lawrence Erlbaum, 1979.

---. The Senses Considered as Perceptual Systems. Boston: Houghton Mifflin, 1966.

Gourlay, Kenneth A. “The Non-Universality of Music and the Universality of Non-Music.” The World of Music 26.2 (1984): 25-37.

Hagen, Edward H. and Gregory A. Bryant. “Music and Dance as a Coalition Signaling System.” Human Nature 14 (2003): 21-51.

Hagendoorn, Ivar. “Some Speculative Hypotheses about the Nature and Perception of Dance and Choreography.” Journal of Consciousness Studies 11.3-4 (2004): 79-110.

Hanna, Judith Lynne. “Is Dance Music?: Resemblances and Relationships.” World of Music 24.1 (1982): 57-69.

---. To Dance Is Human: A Theory of Nonverbal Communication. Chicago: U of Chicago P, 1979.

Hodgins, Paul. Relationships between Score and Choreography in Twentieth-Century Dance: Music, Movement, and Metaphor. Lewiston: E. Mellon Press, 1992.

Humphrey, Doris. The Art of Making Dances. Ed. Barbara Pollack. New York: Rinehart, 1959.

Iwamiya, Shin-ichiro. “Interactions between Auditory and Visual Processing when Listening to Music in an Audio Visual Context: 1. Matching, 2. Audio Quality.” Psychomusicology 13 (1994): 133-154.

Jordan, Stephanie. “Music Puts a Time Corset on the Dance.” Dance Chronicle 16.3 (1993): 295-321.

Keil, Charles. “Motion and Feeling through Music.” Journal of Aesthetics and Art Criticism 24 (1966): 337-349.

Kirby, Michael. Futurist Performance. New York: E. P. Dutton, 1971.

Kramer, Jonathan D. The Time of Music: New Meanings, New Temporalities, New Listening Strategies. New York: Schirmer, 1988.

Krumhansl, Carol L. and Diana Lynn Schenck. “Can Dance Reflect the Structural and Expressive Qualities of Music? A Perceptual Experiment on Balanchine’s Choreography of Mozart’s Divertimento No. 15.” Musicae Scientiae 1.1 (1997): 63-85.

Kylian, Jiri. Black and White Ballets. Image Entertainment, 1995.

Leman, Marc and Antonio Camurri. “Understanding Musical Expressiveness using Interactive Multimeda Platforms.” Musicae Scientiae, special issue on Interdisiplinary Musicology (2005/2006): 209-233.

Lipscomb, Scott D. and Roger A. Kendall. “Perceptual Judgement of the Relationship between Musical and Visual Components in Film.” Psychomusicology 13 (1994): 60-98.

Martin, John Joseph. The Modern Dance. New York: A. S. Barnes, 1933.

McGurk, Harry and John MacDonald. “Hearing Lips and Seeing Voices.” Nature 264 (1976): 746–748.

McNeill, William Hardy. Keeping Together in Time: Dance and Drill in Human History. Cambridge: Harvard U P, 1995.

Mellers, Wilfrid H. “Erik Satie and the ‘Problem’ of Contemporary Music.” Music & Letters 23.3 (1942): 210-227.

Merker, Björn. “Synchronous Chorusing and Human Origins.” Ch. 18 in The Origins of Music. Ed. Nils L. Wallin, Björn Merker and Steven Brown. Cambridge: MIT P, 2000. 315-327.

Meyer, Leonard B. Emotion and Meaning in Music. Chicago: U of Chicago P, 1956.

---. Music, The Arts, and Ideas: Patterns and Predictions in Twentieth-Century Culture, with a New Postlude. Chicago: U of Chicago P, 1994. Orig. pub. 1967 without the postlude.

---. “A Pride of Prejudices; or, Delight in Diversity.” Ch. 7 in The Spheres of Music: A Gathering of Essays, Chicago: U of Chicago P, 2000a. 262-278.

---. “A Universe of Universals.” Ch. 8 in The Spheres of Music: A Gathering of Essays, Chicago: U of Chicago P, 2000b. 281-303.

Milhaud, Darius. Le Train Bleu: Opérette dansée de Jean Cocteau, Choréographie de La Nijinska. Paris: Au Ménestrel, 1924.

Miller, Geoffrey F. The Mating Mind: How Sexual Choice Shaped the Evolution of Human Nature. New York: Doubleday, 2000.

Miller, Leta E. “Cage’s Collaborations.” The Cambridge Companion to John Cage. Ed. David Nicholls. Cambridge: Cambridge U P, 2002a. 151-168.

---. “Henry Cowell and Modern Dance: The Genesis of Elastic Form.” American Music 20.1 (2000b): 1-24.

Mitchell, Robert W. & Matthew C. Gallaher. “Embodying Music: Matching Music and Dance in Memory.” Music Perception 19.1 (2001): 65-85.

Neumeyer, David with Caryl Flinn and James Buhler. “Introduction.” Music and Cinema. Ed. James Buhler, Caryl Flinn and David Neumeyer. Hannover: Wesleyan U P, 2000. 1-29.

Nicholls, David. “Getting Rid of the Glue: The Music of the New York School.” The New York Schools of Music and Visual Arts: John Cage, Morton Feldman, Edgard Varèse, Willem de Kooning, Jasper Johns, Robert Rauschenberg. Ed. Steven Johnson. New York: Routledge, 2002. 17-56.

Patterson, David. “Cage and Asia: History and Sources.” The Cambridge Companion to John Cage. Ed. David Nicholls, New York: Cambridge U P, 2002. 41-59.

Paulin, Scott D. “Richard Wagner and the Fantasy of Cinematic Unity: The Idea of the Gesamtkunstwerk in the History and Theory of Film Music.” Music and Cinema. Ed. James Buhler, Caryl Flinn and David Neumeyer. Hannover: Wesleyan U P, 2000. 58-84.

Pudovkin, Vsevolod Illarionovich. “Asynchronism as a Principle of Sound Film.” Trans. Marie Seton and Ivor Montagu. Film Technique and Film Acting. Ed. Ivor Montagu. London: Mayflower, 1958. 183-193.

Pylyshyn, Zenon W. and Ron W. Storm. “Tracking Multiple Independent Targets: Evidence for a Parallel Tracking Mechanism.” Spatial Vision 3.3 (1988): 179-197.

Ramachandran, Vilayanur S. and William Hirstein. “The Science of Art: A Neurological Theory of Aesthetic Experience.” Journal of Consciousness Studies 6.6-7 (1999): 15-51.

Repp, Bruno H. and Amandine Penel. “Auditory Dominance in Temporal Processing: New Evidence from Synchronization with Simultaneous Visual and Auditory Sequences.” Journal of Experimental Psychology 28.5 (2002): 1085-1099.

Ries, Frank W. D. The Dance Theatre of Jean Cocteau. Ann Arbor: UMI Research P, 1986.

Royce, Anya Peterson. The Anthropology of Dance. Bloomington: Indiana U P, 1977.

Sager, Rebecca. “Creating a Musical Space for Experiencing the Other-Self within.” The Musical Human: Rethinking John Blacking’s Ethnomusicology in the Twenty-First Century. Ed. Suzel Ana Reily. Burlington: Ashgate, 2006. 143-169.

Scholl, Brian J. “Objects and Attention: the State of the Art.” Cognition 80.1-2 (2001): 1-46.

Sekuler, Robert, Allison B. Sekuler and Renee Lau. “Sound Alters Visual Motion Perception.” Nature 385 (1997): 308.

Sloboda, John A. The Musical Mind: The Cognitive Psychology of Music. Oxford: Oxford U P, 1985.

Spackman, Stephen. “Wallingford Riegger and the Modern Dance.” The Musical Quarterly 71.4 (1985): 437-467.

Spelke, Elizabeth, William Hirst and Ulric Neisser. “Skills of Divided Attention.” Cognition 4 (1976): 215-230.

Watanabe, Katsumi and Shinsuke Shimojo. “Crossmodal Attention in Event Perception.” Neurobiology of Attention. Ed. Laurent Itti, Geraint Rees and John K. Tsotsos. Amsterdam: Elsevier Academic P, 2005. 538-543.

Watkins, Glenn. Pyramids at the Louvre: Music, Culture, and Collage from Stravinsky to the Postmodernists. Cambridge: Harvard U P, 1994.

Yantis, Steven. “Multielement Visual Tracking: Attention and Perceptual Organization.” Cognitive Psychology 24 (1992): 295-340.

Zbikowski, Lawrence M. “Modelling the Groove: Conceptual Structure and Popular Music.” Journal of the Royal Musical Association 129.2 (2004): 272-297.

Allen Fogelsanger is a composer and the Director of Music for the Cornell University Dance Program. He teaches courses that focus on the relationships between music and dance, ranging from the theoretical through the practical, concentrating on the twentieth-century and also on computer-interactive dance with music.

Kathleya Afanador is a choreographer and an NSF IGERT Fellow in the Arts, Media, and Engineering Program at Arizona State University pursuing a Ph.D. in Cognitive Psychology. The intersection between perception/cognition and interactive arts is her primary research area.

[1] Other explanations posit sexual selection (Miller 2000) and socialization (Dissanayake 2000).

[2] Cross (1999) offers specifically, “The Igbo term nkwa [...] seems to capture the interlinking of sound and action that characterizes music for most cultures” (39). For further examples and discussion of the myriad ways in which dance and music are categorized see Royce (1977); Hanna (1979, 18-19); and Cross (1999, 30). Hanna (1982, 58 and 60) includes examples of performance forms in which the dance and music are inseparable. There have also been experiments exploring how closely related are analogous parameters in music and dance, e.g. by Friberg and Sundberg (1999), culminating in proposals that humans dance “to music” to the extent that mirror neurons are engaged by music and result in action (Leman & Camurri 2006).

[3] Gourlay (1984, 36) and Besson and Schön (2001, 234) argue that music and dance are but parts of a more universal form of expression that involves them together with all aspects of performance. A similar assertion from the perspective of American modern dance was offered by Martin, “[A]ll the arts are linked together so closely that it is difficult to pry them apart” (1933, 89).

[4] Another question is how to interpret the degree of congruency. Cook (1998, 98-106) develops a scheme in which any two media exist together in one of three models: conformance (temporal congruence as in [1] and [2] below), complementation (broader congruence as in [3] and [4] below), or contest. The relationship between media contributes to the meaning of multimedia; the whole point of contest is that the broader multimedia meaning is a result of perceived disunity of component media. Collins and Olofsson (2006, 13) argue further that while complementation and contest are more interesting, they depend for their effectiveness on their contrast with conformance, a reliance on the play of expectations that recalls Meyer (1956; 1994); they are determining effectiveness in the context of VJ performance. On the other hand Chion (1994, 37-39) argues that audiovisual dissonance occurs only when the meaning of the visual element and the meaning of the auditory element are contradictory.

[5] For movement implied by sound see Gabrielsson (1973) and Eitan and Granot (2006).

[6] Krumhansl and Schenk also find their results to be consistent with the conclusions of Jordan (1993).

[7] The works respectively were “Solo for Prepared Piano,” part 4 of Amores (1989); “Sandstorm” (1989); and Spiral Dream (1988). “Each musical selection was interpreted [...] as a modern dance” (70).

[8] In this list we roughly follow Mitchell and Gallaher (66-67), but choose to separate pulse as a temporal aspect on its own. Merker (2000, 315), following Arom (1991, 2000), has remarked that music seems to roughly divide into two types, “Either it is measured, that is, avails itself of a regularly paced timegiver, or it gives up reliance on time marking altogether, and is unmeasured.” Arom (2000, 27-28) notes that “[i]n Africa, nonmeasured music--music one cannot dance to--is not usually considered music at all, but is classified as a lamentation ("tears") or a type of signaling device.”

[9] We note that there is evidence that the perception of downbeat is culturally mediated (Cross 1999, 31). Clarke (1999, 494-496) summarizes the connection between music, especially with “a periodic rhythmic structure”, and movement.

[10] Hodgins categorized choreomusical parallels as either intrinsic or extrinsic; (1), (2), and (3) correspond to the former while (4) corresponds to the latter.

[11] Dances without music exist both in western art dance (Banes 1994, 312) and in folk dance (Hanna 1982, 61).

[12] Drake-Boyt (2005, 196), from which we found Buckle’s quote, argues that the dance and music are “remote” from one another, and only “coincidentally inhabit the same time/space frame.”

[13] Cited in Garafola (1989, 102-103, n. 10).

[14] Cocteau’s reference to “slow motion” is evidence of film’s impact on non-film performance.

[15] Gillmor (1988, 255) notes that Satie “composed a kind of neutral rhythmic counterpoint to the visual action. Divorced from the film the music is practically meaningless, refusing to take on a musical identity of its own, but as film music it fulfills its function admirably.” Watkins (1994, 333) adds that “Satie’s music-hall ensemble [...] crossed the line from banality to boredom by mechanically projecting a series of four- or eight-measure ostinati totally lacking in continuity, and arbitrarily recalling musical materials without narrative consequence.” Yet Clair (1972, 10) relates that Satie “timed every sequence with meticulous care, thus preparing the first musical composition written in perfect synchronization with a film.”

[16] Although primarily about a counterpoint between sound and image, it also had applications to counterpoint between sound and movement. The influence of film on multimedia performance can be seen throughout the 1920s. Nijinska’s Le Train Bleu (1924) included both slow motion and freeze framing (Garafola 1998, 16), though the freeze framing occured “[b]etween each ending and the orchestra starting to play again” (Cocteau in Milhaud 1924 cited by Batson 2005). Cocteau notes the “silence of thunderous events in silent films” in 1926 referring to Parade (cited in Ries 1986, 40 n. 36, from Cocteau 1926, viii). Hindemith’s Hin und Zurück (1927) included the retrograde of both music and visual elements (Watkins 1994, 333-334). But although the idea of simultaneity had been present since at least the Italian Futurists, an explicit theorization and advocation of asynchrony between sound and visual images was not produced until the late 1920s, by Eisenstein with Pudovkin and Alexandrov, and independently by Clair and by Balázs; see Altman (1980, 11-12) and Neumeyer, Flinn & Buhler (2000, 15-16); also Arnheim (1957). What Pudovkin (1958, 191-193) describes is a counterpoint of the emotional referent of the music with the images telling the story, with the purpose of deepening the understood meaning. Clair (1953, 94-95) describes a counterpoint of the referring sound with the image in the service of the narrative, e.g. a car door slamming shut needs only be heard but not seen. Neumeyer et al note that Flinn (1992, 35) catches that “discussents in all these texts tend to conflate imagetrack with narrative; it is the latter, and not the imagetrack per se, that is normally meant to parallel or to ‘counterpoint’ music” (16). Interest in this aspect of film seems to wane with the arrival of “talkies.” Cohen (2002) comments that “Perhaps attention to speech robs processing resources from visual structure, but this is not necessarily the case” (218).

[17] The film theory and films were still operating under the ideal of the Gesamtkunstwerk. Garafola (2005, 51-52) makes a similar point about the Ballets Russes. See also Paulin (2000). An interest in arbitrary relationships in film doesn’t seem to appear until Kracauer’s anti-parallelism in 1960 (Paulin, 69-71).

[18] Miller (2002a, 2002b) links Cage’s complete independence of music and dance to Cowell’s development of elastic form, which allows internal contraction or expansion of sections in order to match the needs of choreography. Cowell was attempting to solve the problem that “dance was a parasite on the musical form” (1934, 52). That this was indeed seen as a problem is affirmed by Humphrey (1959, 142) and Martin (1933, 115). Spackman (1985), in his study of composer Wallingford Riegger’s involvement with modern dance (who wrote the music for Humphrey’s New Dance [1935] and With My Red Fires [1936]), concludes that the “simplicity and directness of form” of Riegger’s music was “reinforced by Reigger’s experience with dancers,” as “any sort of complex musical accompaniment would either distract attention from the dance or simply be lost on the ear” (465), thus offering more evidence of the period’s sensitivity towards the potential for interference between music and dance.

[19] Cunningham recalls that an intermediary stage was “where meter was completely abandoned” (142). It should also be noted that although the dance and music in Cunningham’s later work share only a common simultaneous duration, they still often seem “to go together.” This is not surprising if, as Copeland (1983, 183) argues, “the relationship of sound, movement and decor in a Cunningham piece is not entirely arbitrary.” The most comprehensive look at the relationship between music and dance in Cunningham and Cage’s work is Copeland (2004, 145-164). For an example of a younger choreographer, Trisha Brown, influenced by Cunningham and Cage but determining her own varied relationships between dance and music see Fogelsanger (2005).

[20] Banes (1983) thoroughly describes the impact of Cunningham and Cage on the next generation of modern dance choreographers in the Judson Dance Theater; Copeland (2004, 7-8) and Nicholls (2002, 52-53) argue that their impact reaches far beyond dance.

[21] Patterson (2002) discusses the Asian sources of Cage’s philosophy. Copland (2004, 165-182) adds more on the acceptance of simultaneity by Cunningham. Cage’s appreciation of simultaneity appears to be one of enjoying the coincidences of events one sees if one looks. This seems fundamentally different from the Italian Futurists, who according to Kirby (1971) were stimulated by motion pictures “eliminating distance while compressing time” (47).

[22] Lipscomb and Kendall suggest that with narrative film, “music exercises a strong and consistent influence over the subject responses to an audiovisual composite, regardless of visual stimulus” (60). Mitchell and Gallaher further note that, “ambiguous visual images such as dance may be more influenced by auditory capture than are visual images with a strong narrative organization” (67).

[23] As cited in Mitchell and Gallaher (67). A striking example from contemporary dance is expressed by Earle Brown speaking of Springweather and People, a 1955 Merce Cunningham piece. He says, “An extraordinary moment in [the piece] was Merce coming out of the wings in a leap at a point in the music when there was only one note being played on a violin. Merce flew on the one held note. A dancer other than Merce—a composer other than myself—would have supported that leap with a dramatic musical flourish…Merce’s leap was so stunning dramatically the way it joined that one violin note, much more astonishing than it would have been to anything else I could have written in order to support it” (Copeland 2004, 162-163). An example within a pulsed setting where congruence is intended is Jiri Kylian’s Falling Angels (1989) choreographed to Steve Reich’s Drumming (1970/1971) Part I. (See Kylian [1995].) The music’s beat is divided into six pulses. Kylian alternately choreographs movement dividing the beat into two equal parts or three equal parts, and the resulting visual stimulus influences how easily the accompanying music may be heard as organized in either duple or triple time.

[24] “Many of Cunningham’s innovations [...] serve the ultimate goal [...] of providing us with opportunities to choose when and where to focus our visual and auditory attention” (2004, 17). See also Copeland (1998).

[25] Copeland (2004, 129-130) connects this “separation of the elements” to Brecht’s “intentional disunity.”

[26] This is distinct from the syntactic and structural listening that Meyer describes, though it certainly contributes to more fully practicing such listening, at least when applied to, say, fugal forms. Yet “polyphonic” listening can increase the understanding and appreciation of any polyphonic music, from the jazz quartet to the Banda ongo ensemble. See Arom (1976).

[27] Bregman (1990) argues that there are both innate and learned processes for separating sound into separate streams of sequential aural material (206-209), that attention can allow us to “select a part of a sequence against the dictates of the primitive process” (450-453), and that we are capable of following multiple perceptual streams (464-465). Bigand et al (2000) examine divided attention in music listening and propose that listeners employ a strategy of integrating multiple voices into a single, acoustically complex, stream, but this may not apply to watching dance and listening to music. They also point out that beginning drivers have difficulty speaking while driving, but that practice eventually makes this easier. Likewise Spelke, Hirst & Neisser (1976) found that students could be trained to take dictation while simultaneously reading an unrelated text.

[28] Pylyshyn and Storm (1988) showed that people can track up to four or five moving objects simultaneously. Scholl (2001) provides a recent survey of work in this area, and notes that results indicate that attention is split among the target objects (10). Additionally multiple object tracking can be enhanced by employing a grouping strategy, for instance imagining five targets as a pentagon and tracking the pentagon (Yantis 1992). This is somewhat analogous to Begand et al (2000) suggesting that musicians use learning harmony as a strategy for following multi-voiced music, though this is a strategy for catching mistakes in the music on repetition, not for tracking the voices the first time one hears it, which is more like what Yantis’s strategy is for.

[29] While Hagendoorn’s mechanism driving expectations is our kinesthetic response to movement, Meyer’s (1956, 1994) is the gestalt of pattern and good form. Hagendoorn’s framework, however, does involve a second system that invokes visual gestalt principles, as updated by Ramachandran and Hirstein (1999), for explaining our interest in group relationships. It must be noted that while Meyer’s theory of perception is grounded in a universalist Gestalt psychology, he argues that expectations are thoroughly conditioned by experience and thus culture (2000a, 2000b).

[30] Keil’s “groove” differs from, for example, Zbikowski (2004), where “groove” is defined as “a large scale, multi-layered pattern that involves both pitch and rhythmic materials, and whose repetitions form the basis for either a portion or all of a particular tune” (275). Sager (2006, 147-148) relates Keil’s “groove” to the “bodily resonance” of Blacking (1983, 57), defined as “the sensation and awareness of synchronicity with the physical movements of others around one in a music situation” (Sager, 147). This differs from more general kinesthetic response in its specific inclusion of synchronicity (usually created by pulse) as an integrating factor and in its reliance on the bodily motion of all participants, not simply their sympathetic experience of virtual motion.

Keil was writing in opposition to Meyer’s ideas. A second criticism of Meyer’s ideas is that his work uses Western classical music, with its associated structural and syntactic listening style, as its starting point, and empirical research has been almost uniformly of Western listeners attending to Western music. Balkwill & Thompson (1999) provide a short list of exceptions. In studies meant to begin to address this, they and Balkwill, Thompson and Matsunaga (2004) found that Western and Japanese listeners could accurately recognize basic emotions expressed in Western and Hindustani music (the Japanese listeners in Balkwill et al also listened to traditional Japanese music), and that these emotions were consistently associated with common acoustic cues, such as volume, tempo, and complexity, across cultural boundaries. These are not so much structural and syntactic as they are psychophysical elements, of which volume and tempo, at least, may be felt as well as heard.

[31] Sager makes a similarly proposes that music can create a virtual space for transcendent experience (163-167).

[32] Kramer (1988) proposes a variety of types of stasis in music, dealing largely with twentieth-century Western music in which listeners’ experience of time is shaped by non-processual or minimalist organization. Meyer’s (1994) information-theoretic framework includes such possibilities; see chapters 5, 10, 11 and the “Postlude.”