Edited and expanded by Dustin Ragland
Academy of Contemporary Music at The University of Central Oklahoma
“Light appears to be an ideal mediator between music as a temporal art and painting as a spatial art…
its movement allows the structure of flowing time to become visible…”
– Anne Hoorman, Lichtspiele
Visualization of abstract ideas is woven into our typical experiences to the degree that we think and speak under the guidance of “visualism” without giving much mind to it: “Have you seen…?” “Look into it…” “Let me show you…” are all typical ways of encouraging thought, study, and creative activity – in our provenance we might expect assorted sonic results from these visualist encouragements. Many of us learn complex mathematical processes (even in the early ages of software for such a thing) by means of visual help. Whether eigenvectors or digital signal processing, visualization of complex implements leads to better understanding for anyone looking to employ them in their own creative work. Not only do these visual elements promote understanding of complex phenomena – they are aesthetic gestures as well: they reveal beauty (in the widest possible understanding of the word) in technological structures. When understood aesthetically, they show potential for not only creative description, but for creative use. Visual representations of algorithms and signals become part and parcel of the raw imagination of a composer: patterns, disjoints, harmonies, signals. Thinking about sound visually is rarely optional, and the resulting musical artifacts are more often than not accompanied by some visual medium. This visual medium can vary in formality from presenting musical work on social media with a low-resolution video demonstration, to laser-focused (or laser-based) audio-visual interaction.
The primacy of the eye in contemporary creative cultures is not merely something for those working in the arts of sound to discard in lament, or reify without thought to how images work epistemologically. In electroacoustic musical communities, there is a strong thread at work that imagines interactions between musical sound and visual phenomena that extends beyond the familiarity of film scoring into all manner of fixed and un-fixed media. I wanted to provoke a small discussion on these extensions by hearkening back to a small piece of aesthetics from Theodor Adorno and Hans Eisler, Composing for the Films. I return to this work not because I find the arguments convincing, as several responses to the prompt bore out, but because the work of Adorno and Eisler can help to reveal among young composers, and among many experienced composers, how we might continue to privilege image or sound in our musical-visual works that seek some kind of epistemological unity between the two. Peter-Paul Verbeek’s reflections on Don Idhe’s “hermeneutic of instruments,” applied to our musical context, might guide us here: these visual interactions with sound (including algorithmic and computational elements of sound) are not neutral to the sonic acts we undergo. Musicians have leveraged the visual guidance of scores into sonic gestures for ages, we know that visual sonic illustrations “do not simply depict reality, but co-determine how reality can be present for and interpreted…”
Adorno and Eisler caution the independence of musical elements in relation to the images that are at work in a film, where “poetic-lyric” forces at work in the musical elements give too much conceptual primacy to the melody, against what they see as the music’s role in service to the film’s onscreen activity. The famously gnomic “sparkle and glisten” as the prescription for music for films treats music as a refractive glare – perhaps appearing when the primacy of the eye hits the screen visuals just so. K. Paul Boyev’s response on the SEAMUS Facebook group is exactly what I find unconvincing (but instructive!) in Adorno and Eisler’s framing:
Music and image are treated here as if they are on parallel tracks, when I would rather we treat them as a synergistic whole borne of collaboration between composer and visual artist (unless of course an artist chooses to tackle both forms.) I look at this as One Thing.
Sound accompanying film has typically been the beginning point for discussions around the interactions of music and image, and it does form a massive cultural artifact – economically and imaginatively. Reflections ranging from Eisenstein’s “texture-timbre matching” with Prokofiev, to Sofia Coppola’s use of shoegaze and pop remain vital and important to contemporary sound-image epistemologies, but seem to still operate under the assumption that sound and image are on “parallel tracks.” Michel Chion’s seminal Audio-Vision, and Sandra Naumann’s The Expanded Image are two examples of works that begin to explicate Boyev’s One Thing. Chion’s work describes the phenomenon of “synchresis,” where the modalities of image and sound collapse into one another – forming a synthetic epistemology that moves beyond correspondences of concept or action into a realm where aesthetic experience casts a new alloy. Naumann’s work captures innovations in sound-image understandings, from the advent of the improvisational light show in the 1960’s (often accompanied by hallucinogenic assistance), to the rise of the VJ – where live, reactive, usually digitally-derived video accompaniment to music has made moves “from club culture to traditionally high-culture contexts.”
In another Facebook comment, Dave Gedosh outlines his purposeful disputation of Adorno and Eisler’s prescriptions for creative and pedagogical effect:
In several different courses I’ve taught, I’ve had sections on composing for media. One of the things I like to do in class is to take a video, film, or commercial and add different music to it and then discuss its impact. The music I use is not related to the media, its “formed independently of the action onscreen. This places a “poetic-lyric” activity into the melody that does not serve the “utility” of the motion picture, according to Adorno and Eisler.” The results can be amazing – and seem to contradict Adorno’s critque (in which case I’m usually happy with that). Had Adorno had modern tech to play with he might have not had the same criticism.
This is an excellent example of what I hoped for in my initial class assignment as well: for young composers to confront ideas that are likely not fully convincing to their own musical contexts – but ones that most of them readily admitted to informing how they understood sound’s relation to moving images onscreen. When written out before them, they were quick to decry the elitism that emerges from handwringing over contemporary musical forms. Yet they were at a loss as to how articulate their own formal principles – even when they would create brilliant new interactions of their own. Gedosh’s project is a wonderful idea on just how readily subversive sound-image combinations can become, and how to break the crust of expectations that forms on a composer’s formal approaches from time to time.
Gedosh’s last point was echoed by a former student and gifted composer, Santiago Ramones, who saw the prompt and came into my office, observing that much of the modernist framing, and the technological milieu surrounding Adorno and Eisler’s work would barely have the benefit of “electronic music” as a compositional and generic category, much less the ubiquity of compositional tools that employ some form of sound-image relation, typically via a screen. Ramones’ main emphasis was on how Max/MSP/Jitter might explode the framing mentioned in the prompt. Jitter’s ability to reflect, respond to, and even generate audio in a visual format that is highly personalized, and instrument-like, provides ever-new approaches to composition and performance that practically employ sound-image as One Thing. The term “media” is used in myriad ways, but its concise plurality serves this technological vanguard well, and reflects the accessibility of affordable tools like Max/MSP/Jitter, and the newer VIZZIE/BEAP sub-formats.
Our discussion will close with Kyong Mee Choi’s email response to the prompt, which is worth quoting at length:
In regards to the subject of “independence” of the musical ideas, and their “interdependence” with the visual element in the electro-acoustic music: I think keeping a balance between independence and interdependence is the key. If there are too much synchrony between the two medium it could be too predictable, yet, if there is too much independence then it would lose focus of the piece. Cohesiveness needs to evolve from contrast and diverse activities, allowing room for audience to digest and reflect connections. Adding more medium does not necessarily result in enhanced experience. The ideal result is that “the whole is greater than the sum of its parts.” Also, each medium (music or visual part) should be convincing to begin with, instead of hoping to gain strength through adding other medium.
Choi’s response is a thoughtful balance between some of the responses and provocations we began with, and the act of receiving audio-visual stimuli itself. Concepts from design and technology interfaces work their way ever further into aesthetic imaginations, and the ideas of UX are reflected in Choi’s helpful rejoinder to the “media” I had previously emphasized: “adding more medium does not necessarily result in enhanced experience.” As composers understand, adding more notes, complex harmonies, allusions and formal elements – all of the “more” can suffocate what sonic bits of light are formed by the musical work itself. When it comes to providing actual bits of light (or whatever visual form you might imagine: corporeal, color-driven, architectural, etc.) that interlace with the sound elements – even the most multi-media trained audience can experience the media as clutter. Electroacoustic composers and performers have unique vantage points at the borderlands of music and technology – the spirit of sonic provocation: interlacing acoustic, acousmatic, and electronic technologies leads many electroacoustic musicians into audio-visual works that can provoke listeners into what Edwin van der Heide calls their “own spatial languages of light and sound.”