An ordinary whiteboard become an interactive play space
This was my very first collaboration with Brent Bushnell.
ArtFall is a collaboration between Brent Bushnell and Eric Gradman, in association with Mindshare Labs. ArtFall is an interactive installation that allows visitors to construct a dynamic physical simulation by drawing on a whiteboard. For centuries, people have played games that require nothing more than a pen and something to write on. ArtFall opens up a whole new space of possibilities.
When you draw on this whiteboard, your art is captured by a camera. A custom computer vision algorithm analyzes your art and inserts its contours into a physics engine. In this physics engine we simulate a two dimensional world wherein simple objects drop from the top of the whiteboard and bounce off your artwork. A projector overlays the results of this simulation back onto the whiteboard.
It then appears as though circles and squares “fall down” the whiteboard, collide with your artwork, with one another, and with the edges of the whiteboard. The shapes slide down ramps, collect in concavities, balance precariously on one another, and eventually come to rest at the bottom of the whiteboard.
We added some extra features, like controlling the direction of gravity using an iPhone accelerometer. And when installed in a loud environment, a bandpass filter turns musical “thumps” into physical impulses that agitate the virtual objects in the ArtFall world.
Building and testing ArtFall with Brent was incredibly fun. We were able to develop the system largely in two independent pieces. I mostly built the computer vision and reverse-projection component using OpenCV in C++, while Brent used the Chipmunk Physics library in Python to do the physics. Somewhat late in the game, we joined the two pieces together using OSC for interprocess communication.
I make that sound simple, but in fact there were quite a few very late nights spent hacking this code together. But the most memorable occasion was one morning at 3am when we finally got a proof of concept working, and we could see shapes bouncing off our artwork.
We were completely exhausted, but we still spent at least two excited hours drawing on the board, erasing one another’s ideas, and inventing games to play on ArtFall. The shaky iPhone video we took of the event is hilarious to watch. Even our half-working debugging test was so much fun to play with, we couldn’t stop playing! That night, we reinvented skeeball, mazes, pachinko, and other games that have probably never existed before.
And when we finally finished ArtFall and watched how other people played with it, I realized that the limits of this sort of interactive technology are beyond my imagination. Every new drawing suggests a new way of having fun with interactive whiteboards.