Integrate Processing on the web

I worked with Dan Shiffman this past semester as an intern for Processing. My goal was to create documentation about various ways to integrate Processing on the web. I go over how to control a sketch from a webpage, how to control a webpage from a sketch, how to use audio and video with Processing.js, and how to port java libraries to javascript:


Firewall is an interactive media installation created with Mike Allison. A stretched sheet of spandex acts as a membrane interface sensitive to depth that people can push into and create fire-like visuals as well as expressively play music.

The original concept stems from a performance piece I’m currently developing as Purring Tiger (with Kiori Kawai) titled Mizalu, which will premiere in June 2013. During one scene in the performance dancers will press into the spandex with the audience facing the opposite side. Mizalu is about death and experience of reality, so this membrane represents a plane that you can experience but never get through. As hard as you try to understand what’s in between life and death, you can never fully know.

The piece was made using Processing, Max/MSP, Arduino and a Kinect. The Kinect measures the average depth of the spandex from the frame it is mounted on. If the spandex is not being pressed into nothing happens. When someone presses into it the visuals react around where the person presses, and the music is triggered. An algorithm created with Max allows the music to speed up and slow down and get louder and softer, based on the depth. This provides a very expressive musical playing experience, even for people who have never played music before. A switch is built into the frame which toggles between two modes. The second mode is a little more aggressive than the first.

User Testing:


An image of a Campbell’s Chicken Noodle Soup can sliced up into the Campbell’s Chicken Noodle Soup bar code:

I’ve been reading Charles Petzold’s excellent book “Code: The Hidden Language of Computer Hardware and Software.” This book breaks down how code (and computers) work in a very simple and easy to understand way. Each chapter builds on what has come before, and a very lucid and detailed map of our modern information age slowly unfolds. In one chapter Petzold explains how UPC bar codes work. I was totally fascinated to learn that bar codes are binary. In the book the author used the UPC code from the Campbell’s Chicken Noodle Soup can as an example:

Take a look at that bar code, hopefully you’ll notice that the bars and the spaces are all different widths. When you see a larger bar, that is just smaller bars stacked beside each other. The largest bar is actually four small bars put together, and likewise, the largest space is four spaces put together. Each single width bar is equal to a 1 and each single width space is equal to a 0. The largest bar (fours bar wide) translates to four 1′s, and the largest space translates to four 0′s.

When reading about the bar codes I started thinking about what kind of art I could make out of this binary information. I thought that it might be nice to translate the image of a product into it’s own bar code.

I started with the image of a Campbell’s Chicken Noodle Soup can and turned it on it’s side, so it would be a better width for a bar code.

I then downsampled, and afterwords upsampled, the image (thanks Mark Kleeb) to smear the image into lines. This algorithm compresses an image down 100 times it’s original size vertically, then converts that new image back to  the size of the original. The effect is quite nice.

This is already starting to look like a bar code, but I really wanted to represent the actual bar code with the image. So I sliced up this picture vertically into hundreds of smaller pictures. Each new picture was the same height as the original, but just 1-pixel wide. I wanted to sort the pictures based on brightness so I could then use darker slices for bars and lighter slices for spaces. I made a new PicSlice object (variable name ‘bars’ in the code) which stored both the picture slices and their brightness. I’m getting the brightness at 20 pixels in height, this is arbitrary. The slices change color moving from top to bottom so different results happen when taking brightness at different heights. The PicSlice object uses a really nice Java interface for sorting: Comparable. Here is the code for the picture slicing:


I next had to encode the UPC algorithm, this was pretty time consuming. I created a class that takes an array of integers and translates that into the binary code that the UPC standard uses. You give it the numbers printed out on a bar code (the same type of bar code standard that Campbell uses) and it will give you back the binary for it, but not just straight binary. Each number in a bar code is represented by a certain binary combination, so this class gives you the proper binary combination. I included the code for this class at the bottom of the post.

I then arrange the picture slices based on brightness according to the 1′s and 0′s the UPC class gives me. If it’s a 1, a darker slice, if it’s a 0, a lighter slice. The final result is not an exact representation, I’m using different (and multiple) colors for different single width bars beside each other, and each strip changes color a little from top to bottom. I don’t think it will scan, but it definitely looks like the bar code.

Here are some variations:

UPC class:


My name in three sans serif fonts and three serif fonts. See them all together here.


The Glockentar is an instrument I created where a robotic glockenspiel is triggered by plucking guitar strings and beams of light are projection mapped on the strings. For the project’s video I created a title where the L and the K become glockenspiel mallets and the T has guitar tuning pegs on it. In the video the title is animated.


The next two were made with Processing code taken from the Generative Gestaltung book. All the (amazing) code in that book is available here:

This is a picture of my wife and I with the words from my marriage vow as pixels. The letters are colored with the same colors from the original picture. The sizes of the letters are determined by brightness (bright is small, dark is large).