Mladen Milicevic

Experimental/Abstract Film and
Synaesthetic Phenomena 1725 - 1970

This is a short history of synaesthesia between the music and visual phenomena which started when Isaac Newton in the 18th century compared the spectral colors he had produced by splitting up the suns light into the spectrum and the musical notes of tempered scale.

Following Newton, the Jesuit, Bernard Castel constructed his clavcien oculair (ocular harpsichord) around 1725, which consisted of two colored discs (in 12 parts corresponding to the 12 semi-notes of the tempered octave) connected to a harpsichord. He believed that a base note, (which we call C) rendered a firm, tonic color serving as a foundation for all colors, and that that was blue.

The next interesting way of connecting the music and light dates to the beginning of the 19th century with Italian Futurists. Painter and composer Luigi Russolo was interested in color-sound correspondence in his painting Music of 1911. The Futurist Film Manifesto (1916) written and published by Filippo Tommaso Marinetti, Bruno Cora, Emilio Settimelli, Arnaldo Ginna, Giacomo Balla and Remo Chiti had a series of 14 points. Point 4, entitled Cinematic Musical Research, focused on dissonances, harmonies, symphonies of gestures, events, colors, etc. One of the co-authors of The Futurist Film Manifesto, Bruno Corra, had already published an article Abstract Film - Chromatic Music in 1912 in which he described experiments carried out with his brother Arnolda Ginna, and their work in film to that date. Unfortunately these films have probably been lost and are not available for research. But in his article Corra argues that the chromatic scale consists of only one octave and managed in correspondence seven colors in four octaves. They used a series of twenty-eight colored electric light bulbs (four reds, four greens, four violets etc.) corresponding to twenty-eight keys, to construct their color-organ. Afterwards, Corra and Ginna composed a few color sonatas for their new instrument.

Russian composer Alexander Scriabin was also interested in the problem of the colored light visualization of music, having a vision of a performance of his Prometheus with colored light. He invented a Fiestra per Luce, or keyboard of light, and Prometheus was performed in Moscow in 1911 with color projection apparatus which unfortunately failed to function. His next performance was in New Yorks Carnegie Hall in 1915 but Scriabin was not satisfied with optical effects which produced color-piano projecting color light onto the ceiling.

Wassily Kandinsky, one of the co-founders of non-objective painting, attempted to connect music and visual images through his ideas on abstract theater, published in 1912 in an essay entitled On Stage Compositions in the journal The Blue Rider. Kandinskys artistic sensitivity and his attitude to the theater are intimately bound up with his own peculiar, synaesthethic ability to experience certain color tones in relation to musical notes. His work for the theater was his staging of Mussorgskys composition, Pictures at an Exhibition which Bauhaus master Kandinsky produced in Dessau in 1928. The musical piece of Mussorgsky consists of sixteen images which reflected Mussorgskys impressions of an exhibit of pictures. Kandinsky produced the whole stage image as abstract, with the exception of two pictures Samuel Goldenberg and Schmuyle and The Market Place at Limoges in which he had included two dancers. He also used different shapes which were remotely objective, but he did not proceed programmatically. Kandinsky said; The main materials were: 1. the shapes themselves, 2. the colors on the shapes to which were added; 3. the colors of lights as painting in depth, 4. the independent effect of colored light and 5. the composition of each image as it related to the music, necessarily, its decomposition. The action of shapes were movements of colored, graduated light which is woven into the abstract stage event through spot-lights, lamps and, in one case through a kaleidoscope projected onto the back wall. Kandinskys theater stage visualization of music was based on his own imagination and was not so strongly correlated in transposition with the acoustical phenomena as with musical structure.

Alexander Laszlo, trained as a composer and a pianist, worked on the realization of the differentiated projection machines he had sketched in 1924. He built his color-light piano, sonchromatoscope, with four large projectors and four small footlight machines operated from a switchboard. He also designed a new form of notation called sonchromatography. On the sonchromatoscope Laszlo played romantic music in the style of the 19th century. He performed this music with colored shapes of light, moving kaleidoscopically on the screen.

Russian film maker Dziga Vertov, who had started music studies in 1912, in the Bialystok Conservatory of Music, moved in 1915 to Moscow in order to escape the German onslaught in Poland. A year later he went to Petrograd where he became a follower of the Russian avant-garde. During this period he was deeply interested in music and sound, building his own rudimentary Laboratory of Sound in 1916. Working with a Pathephone was disc recorder and collecting sounds, Vertov was interested in sound-montage through his work as editor of the weekly newsreel Kino Nedelia (Movieweek). At that time he attempted to create concrete symphonies but was not able technologically to realize them. Later, during his film work he transposed his sound-montage ideas onto the film media especially in the Man with a Movie Camera, where he used superimpositions and split-screen forms. In 1930 he shot his first sound film Symphony of Donbas, or Enthusiasm in which he used a variety of different sound manipulation techniques. Eighteen years later broadcast engineer Pierre Schaeffer introduced those same techniques in Paris as his own invention and called this music musique concrete.

The Bauhaus film makers of the twenties, like Viking Eggeling, Hans Richter and Walther Ruttmann devoted to the idea of optical music liked using concepts from musical creation: orchestra, orchestration, symphony, instrument, fugue, counterpoint, and especially too the term score - with the distinguishing feature the exact notation of time and the flow of movement. They were not able to produce music colored films. Soon they discovered that the medium adequate to their dreams was the hand-drawn animated film. Thus Werner Graeff, in his Filmscore 1/22, painted shapes onto the picture frames in color, getting the film equivalent of the play of light linking it with music.

Viking Eggeling also believed that music, like an abstract medium, could be similarly transposed into visual film medium. In 1924 he finished his two films Horizontal-Vertical Orchestra and Diagonal Symphony, based on the musical analogy.

Hans Richter, who collaborated closely with Eggeling, was interested in musical organization of film time in his films Rhythm 21 and Rhythm 23.

Walter Ruttmann, like Eggeling and Richter, worked with idea of optical music through his film serial Opus 1, Opus 2, Opus 3, etc. In 1928 Ruttmann gained access to the best sound recording system at that time in Germany. He made a film Weekend with no pictures, just a sound montage, recording the noises of a working day and Sunday in the country-side. This can be also considered as pre-music concrete.

Oscar Fischinger, the youngest of the German Bauhaus abstract film makers, was extremely impressed by premiere of Ruttmanns Opus 1, which stimulated him to begin practical work as a film maker. Fischinger made a lot of music illustrated films and devoted his career to visualized music, but he himself only reluctantly accepted the burden of musical accompaniment. An analysis of Fischingers films, even when viewed silent reveals a clear musical structure.

Hungarian Laszlo Moholy-Nagy, who was also involved in the Bauhaus, continued Fischingers experiments with direct (hand drawn) manipulation of the optical sound track. In 1933 he made his film ABC of Sound which he described as a lighthearted experiment. In that film, the soundtrack was rephotographed for simultaneous projection onto the screen.

Moholy-Nagys ABC of Sound was shown in London in 1934 at the Film Society where Norman McLaren was able to see it. Later, after McLaren had moved to the USA in 1939, he made a film called Allegro, one of his first experiments in drawing both the image and sound track on film.

In 1939 a Swiss painter, Blanc-Gatti, made a five-minute film called Chromophonie based on the principle that each tone of music ought to have a single, consistent corresponding color in the spectrum. In that film he continued the concept of correlation between auditory and visual tones which had occurred to many artists.

In 1940 James and John Whitney developed a new approach for combining sound with film. John was a composer and his brother was a painter. John had been introduced to Schoengbergs twelve-tone row composing technique and was acquainted with expressionist music from Europe. The Whitney brothers used a series of manually activated pendulums whose movement controlled the amount of light passing through an aperture to which a pendulums were connected. An optical sound track was guided past the opening so that changing light patterns were registered on the film. Since the rate of pendulums movement was slow, the optical sound track had to be played back at a faster speed in order to avoid sub-audible sounds, frequencies below 16 Hz. Because they had been forced to work at such slow speeds, the Whitneys were able to precisely synchronize the temporal relationship between sounds and their visual graphic imagery, with a parallel to the transpositions, inversions and retrogressions of the twelve-tone row technique. In addition, they became aware of a music timbral change when they increased the playback speed. Whitneys soundtracks sounded like early electronic music, and their technique of speeding-up was employed by German composer Karheinz Stockhausen twenty years later in his mixed-electronic composition, Kontakte.

Viennese film maker, Peter Kubelka, was strongly influenced by the music of Anton Webern. Just as Webern reduced music to single tones and silent intervals between them, so Kubelka reduced film to the film-frame and the interval between two frames. In his film Adebar (1957) Kubelka constructed a sound-track consisting of Pygmy music in four phrases, each 26 frames long. That film was a rhythmic harmony for the eyes, and clearly demonstrated his interest in alteration of rest and motion, like the sound and pause in music. In his next film Schwechater (1958) Kubelka worked with black & white and red frames and pure electronic sounds composed in the order 1 red frame, 1 image frame, 2 red frames, 2 image frames, 4 red frames, 4 image frames and so on. Whenever a red frame appears the two sine-tones (pips), are heard, higher introducing lower. Extreme reduction was achieved by Kubelka with the film Arnuf Reiner (1960). The film was made without a camera, consisting of black and white frames which correspond to sound in one-to-one relationship. Black color represents no-light, translated in the music that is a silence, white color is simply the light, translated into music that is a white noise, consisting of the whole range of audio frequencies. Based on that principles Kubelka made Arnuf Reiner. In the Unsere Afrikareise (1960-1966) Kubelka recorded about ten hours of sound, talk, animals, noises, bits of radio music, either separately or combined, and a few hours of images shot in Africa. In the six years between shooting and finishing Unsere Afrikareise Kubelka focused his work on the speed of perception on the instant-to-instant relationship between sound and image. Soundtrack in this film is one kind of music concrete continuing the same path Vertov had begun in his Enthusiasm.

In Great Britain, Anthony Scott takes chance, in the John Cages sense, in making his film The Longest Meaningless Movie, which consists of adverts, complete and incomplete sequences from feature films, out-takes, sound-only film, home-movie material, and so on. Sometimes whole lengths of film appear upside down and running backwards, and in the 35 mm version it flips left to right to reveal the soundtrack.

Structural film maker, Michael Snow, in his Wavelength linked zoom objective on the camera with pure sine wave. The whole film is only one shot which takes forty-five minutes to reach its maximum extension, the soundtrack parallels this movement with an ascending sine wave, starting at 50 Hz and ending around 12 kHz.

All these experiments have to do with the history of experimental film and to the history of exparimantal music, linking together these parallel paths.

TO LEARN MORE ABOUT SYNAESTESIA CLICK HERE