info@aaron-sherwood.com
bandcamp
vimeo
objkt
flickr

Using Processing on the Web

Introduction

One strength Processing has always had is being easily deployable to the web. As the web keeps evolving so do the tools we use to get Processing running in a browser. Currently, the main tool available to us is Processing.js, a port of the Processing language to javascript. Great introductions for Processing.js have already been written on the Processing website as well as the Processing.js website, so I'm not going to spend too much time on the basics here. Rather, I'm going to explore how to integrate Processing more cohesively into the browser, getting it to interact with other elements on the webpage, using it with HTML5 audio & video, loading a library in with Processing.js, and how to port a library from Processing to Processing.js. All the examples will be geared towards someone who has little or no experience with HTML5 or Javascript.


Basics

Ok, so I know I said I wasn't going to go over the basics, but each example is going to build on the last so lets review just a little. Processing 2.0 allows you to export a sketch in javascript mode directly that gives you a premade index.html with all the things you need to run your sketch online included, but it's really helpful to know how to insert a Processing sketch into your webpage by hand. Lets say you have a very basic sketch that just draws a circle that follows the mouse: To embed this sketch in your webpage first you need to put in a script that loads Processing.js and then have a canvas element that tells the browser where your sketch.pde is: A webpage with this in it would look like: Here's the sketch:


Interaction between Processing and the webpage

Let's get into something a little more interesting. Now that you have Processing running as javascript in the browser you have access to everything else on your webpage via your sketch, and vice versa. Let's say we wanted to change the color of something in our sketch by clicking on HTML text outside of the sketch. First thing we have to do is make the sketch and the web page see each other. This is accomplished by calling a function in javascript that "gets" your sketch and then places it inside a javascript variable: An in depth exploration of how javascript works is beyond the scope of this tutorial, but understanding certain key points will be helpful. Taking a look at the line above we see on the right we're calling the function Processing.getInstanceById and then passing in "circleSketch". What is "circleSketch" you may be asking? In order for javascript to access our instance of Processing we need to give our canvas element an HTML ID that javascript can reference, this time we'll name it circleSketch: To change the color of a circle in our sketch we need to have a function inside our sketch that lets us do so: Back on our webpage it's time to put these two together. We make a script where we get Processing, assign it to a variable, then call our setColor() function in our sketch from that variable: Above our canvas element in the body of our webpage we have three Spans with an "onclick" event attribute that calls the changeColor() javascript function we just made. This is where the ID of our canvas and the number of the color we are looking for are actually passed in.

(full code)

These words are HTML text, try clicking on them:

Red Green Blue

Here's our sketch:

What if we wanted to go the other way and click on our circle and have that change some text on our webpage? Well, javascript is so nice to us, it allows us to put a javascript function right into our processing sketch. As long as the element we want to change has an ID we can change the HTML text inside that element. The word "document" below is how javascript refers to the webpage. We are saying get the element with this ID in our HTML document and change its inner HTML text. All you need in the HTML is to set the ID of the element that you want to change the text in: (full code)

Try it out, click on the circle.

I'm some big ole HTML text

One thing to note is that once you start putting non-processing fuctions in your processing sketch, the sketch won't run in the Processing IDE anymore. It will only run in a browser.


Audio

Processing.js doesn't support audio so if we want to play audio in our sketch we have to use the HTML5 audio element built into most browsers. For simplicity, let's say we wanted to trigger a beat everytime we hold down the mouse in our circle. Once again we'll put the code for this directly into our Processing sketch. The first thing we want to do is make an HTML5 audio object. Also, different browsers take different file types, so we'll make a string that we can append to the file name with the proper extension: We'll fade the volume of the audio as well as the color of the circle out whenever we let go of the mouse so we need variables for those. We set the volume at 0 so there will be no sound until we click inside the circle: Next we'll check the browser to see what kind of audio it can play, then assign the corresponding extension to our string: To load the source file we can set the "src" attribute of the audio with the name of the file and append the extension. We start playing the audio by simply calling .play() : We want to loop the audio so we add an event listener that calls a function we created named "repeat" whenever the audio file ends playing. "Repeat" simply starts playing the audio file again: The HTML5 audio element takes normalized volume levels, so we constrain the volume between 0 and 1. Whenever fadeOut is set to true our volume and color amounts will decrease by 10% each frame: Whenever the mouse is pressed within the circle we stop the fade out, set the volume all the way up (to 1), and set the color of the circle to red. Whenever the mouse is released we turn the fade back on: (full code)

Try it out, click and hold inside the circle.


Video

Video doesn't work natively in Processing.js either so lets take a brief look at combining HTML5 video with Processing. Video can be played very easily using the HTML5 Video element, and it can even be drawn into the Canvas element, which is where Processing.js is actually drawn. However, it takes a lot more work by the CPU to draw the video in Canvas so unless pixel operations are called for it's better to use the Video element itself without drawing into Canvas. For example's sake we are again going to use our circle as a button to reveal the video. It'll have a slight twist though, when the button is pushed an image that looks like a still image will be revealed, but when the mouse is dragged left to right the image will scrub in time (the image in fact being a video).

We've already learned how to make a button fade in and out so let's focus on the video. Since we want to use the Video element to display the video and the Canvas element to display our Processing sketch we need to put both of these on screen at the same location, one on top of the other. We can accomplish this with a little simple CSS. We can put both the Video and Canvas elements together in one div, both with the exact same 'absolute position'. This will give them the same position relative within the div. We also have to set the div height the same as our video/sketch, since the absolute position will make the div look like it has a height of 0. Just like we did with audio, we use two different types of video format to accommodate different browsers. To make this work though we need to make the background in our Processing sketch transparent: In our sketch we also need to get a hold of the Video element in the HTML doc so we say: The only thing left is to have Processing set the video location based on the mouse position: (full code)

Click on the circle, drag the mouse left & right.

One thing to note: if you want to scrub a video backwards in the browser you're going to need to make every frame a keyframe. If you don't do this you'll be able to scrub forward just fine, but backwards will be real jumpy because the browser can't interpolate between keyframes moving backwards.


Libraries

One downside to Processing.js is that libraries, as a whole, are not implemented at present. A few have started being ported, Toxiclibs being a notable one, but most of the added functionality we have come to enjoy in Processing proper does not translate over to Processing.js. What if we want to implement one of the libraries that has been ported? What if we have written a Processing library and we want to port it to Processing.js? Let's take a very simplified look. We'll start by porting an ultra simple library to javascript, and then see how to link that in with our sketch in the browser.

Going with our circle theme let's say we wrote a library that draws a bunch of random circles to the screen, having a variety of sizes. The original java library looks like this. This library is very simple and essentially has one function, Circles.draw(), that uses Processing's ellipse function to draw the circles. To be able to use Processing.js's functions in our own library we need to pass it into our javascript library, much the same as you do with a java library. To do this we need to make a variable for our library named the same thing as in the java library and assign a function to it that Processing.js is passed into. The magic happens like this: Then, whenever we want to call one of Processing's core functions within our own javascript ported library all we need to do is say this.app. then the function. This is illustrated with Circles' main draw() function. Prototype simply means the function has the ability to inherit functions (because we want to inherit from Processing.js). What if we have a function in our library that is not a core function in Processing? Let's say our library, for whatever reason, has a function called .setVariable that will take two numbers, add them together, and then store that result in a variable. In our javascript port we can say Circles.prototype.setVariable (prototype again in case we want to pass in a processing function like mouseX) and then assign a javascript function to it that does what we want: In Processing the business ends look like: How do we add this library into our HTML document alongside Processing.js? After we include the processing.js script we simply include a script for our javascript library: Putting it all together (with our .setVariable function doubling the mouseX value) we have:

mouseX: doubled:

(full code)


Resources

In these tutorials we've barely scratched the surface of how Processing can be integrated into a web page. Hopefully with these tools and your imagination you can really make some interesting things happen in the browser. If you want to dig deeper and learn more about some of the things we went over, here are some helpful resources: