William Larson

In the 70s and 80s artist William Larson took advantage of the duality of the fax machine — transmitting images as sound — by transmitting both images and sound in the first electronically mediated collages.

The Pantelegraph

Precursor to the fax machine, the pantelegraph was developed by Giovanni Caselli in the 1860s. A needle attached to the end of a pendulum "scanned" a sheet of paper, with the ink and paper producing different amounts of resistance, sent over telegram to another location with a closely synchronized pendulum.

Sound Modulated Light

Sound Modulated Light by Edwin van der Heide transcodes sound (relative air pressure over time) as light (relative brightness over time). A "light receiver" is provided to visitors, allowing them to listen to the light.

Playlist Creation as a TSP

A playlist (or mixtape) can be considered an ordering over a set of songs. Defining this ordering can be a fairly difficult process, and if we frame it as TSP solving, it's clear why.

Every song has a number of features (tempo, key, genre, theme, etc.), and the difference between songs is some distance metric over these feature vectors. Each song is a node, and there is an edge between every node the length of the difference. As with a traditional TSP, the solution is the shortest path that visits every node.

This alone is a difficult problem, but it's compounded by the possibility of a context dependent metric, or even a total time constraint less than the sum of all the song lengths (with the ability to ignore expensive songs).

Visible Speech

The IPA presents a visual, written representation of spoken sounds. The orthography is derived mostly from standard alphabets. Alexander Melville Bell (father of Alexander Graham Bell) developed Visible Speech towards a similar end, but he attempts to preserve physiological characteristics of the sound in the orthography.

Sonic Visualiser

Sonic Visualiser transcodes audio visually. Meant for "studying a musical recording", it presents a number of "layers" of the audio — a spectral representation, signal representation, note onsets, note values, etc.

4 Performer/Instrument Interaction Models

Joel Chabade has some thoughts on performer/instrument interaction models in the latest SEAMUS newsletter. His classification system goes:

1. Simple: there is a "simple, predictable response" to input (he critiques these models in a NIME 2002 paper)
2. Fly-by-wire: the performer has an abstracted, high-level control of the system
3. Interactive: the instrument acts autonomously, responding to the performer
4. Interacting with life: the performer shares control of the sound with the system, as a sailor shares control of the boat with the waves

The difference between the third and fourth models seems a little unclear, but besides that there are useful distinctions here. "Simple" vaguely corresponds to what I'll call as transcoding, where you interpret one type of data directly as another (gestures as sound parameters). "Fly-by-wire" is a few-to-many transcoding. "Interactive" and "Interacting with life" can be modeled by including a noise source in the transcoding or by adding memory.