3D capture in performance

Capturing 3D data from moving scenes is a hard problem in itself, but the harder problem is how to integrate it with a performance in real time. First you have to hide the 3D capture process from the audience, and then you need to present something related to the 3D capture in an engaging way.

Re-projecting coarse effects can be approximated with a multi-camera system, and fine details are dangerous due to system latency.

The real question is: what can you do with 3D data that you can't do with 2D images? The most interesting idea I have: you can cast shadows on a 3D form in a way you can't on a 2D form.

Removing Image Noise

Gray-level images for foreground/background are often afflicted by fine noise when they are thresholded.

I've used a few different approaches to dealing with this:

• blurring before thresholding
• median filtering before thresholding
• morphological filtering after thresholding
• distance transform after thresholding, followed by thresholding again as a more continuous morphological filtering algorithm

Unfortunately, these all have the same problem: inner corners are rounded. These algorithms can't tell the difference between a black pixel in an inner corner and a black pixel that is just noise.

Perhaps what we should really be using is something like bilateral filtering (in Photoshop, "Surface Blur"). Bilateral filtering preserves sharp edges, while blurring large undefined regions. Unfortunately, bilateral filtering creates a sort of "glow" that still has issues with corners.

Maybe we need something more like "smart blur"? It seems to not propagate across edges...

Face the Beat

I See Beats, but with face tracking. This would solve dark room restriction, but create a host of other issues.

Full Inbox

As of this moment, Gmail offers 7365 MB of storage. To me, 7365 MB looks like an open desert, or a desolate sea. I want to fill that up, or at least craft it. What would email land art look like? What is the architecture of an empty inbox? And how can it be shaped toward an aesthetic or conceptual goal?

Email To Real Mail Converter

I like this idea for an iPhone app, "Shoot It". You take a picture on your iPhone, and they print it out and send it as a postcard for you.

My favorite part is that this is an "algorithm" that involves both automated and non-automated actions. There are real people involved in pushing a postcard around. I'd like to start a service that send letters for you. You would send an email to a specific address, include a mailing address, and we would print out and mail the letter for you.

This service actually exists in reverse for older folks that don't want to deal with the internet. They can have a service print out emails for them, and send out their handwritten letters as emails.

Every Every Icon

John F. Simon Jr.'s "Every Icon" iterates through all the 32x32 black and white icons at approximately 100 icons per second. Sintron's "God's Eye" iterates through all the 800x600 color images at 97,000 images per second. Jim Campbell's "The End" iterates through grayscale images using custom electronics. Leonardo Solaas' "Magic Mirror" iterates through every 720x576 color image at 25 images per second.

I propose a meta-Every Icon, "Every Every Icon". This work will include all of the above works, as well as any other variations that may be dreamed up in the future. It will accomplish this by iterating through every possible resolution, at every possible framerate, for every possible bit depth, in every possible order.

Crowdsourcing Glitches

I'm interested in the possibility of destroying data through natural processes. I've already explored this a bit with Future Fragments, where quotes from friends were carried in their pockets and destroyed over the period of a summer.

Another interesting system that naturally destroys information over time is the human mind (memory). Consider the possibility of crowdsourcing distortion: for example, with Mechanical Turk, create a task that plays a subset of a song that the worker is then asked to repeat. These responses are then averaged to create an approximation of the original tune.

This task takes advantage of our ability to hear melodies. We can also treat the mind as a more generic digital signal processor: ask each worker to recall the midi pitches of each note. Or better: portions of the mp3 encoded audio in hex. Besides the auditory system, we can also take advantage of the visual system. Ask each worker to recall and draw the audio signal (at a sufficient scale).

These tasks would be especially interesting in the case of people like Ben Pridmore, who is able to quickly memorize large amounts of data (e.g.: 364 playing cards in 10 minutes). I imagine his memory slowly degrades. It'd be great to see what a compressed image looks like that is memorized slightly incorrectly, and watch it degrade over time.