Multimedia - Fall 2005

Room 1221, 719 Broadway
7-9pm Mondays

Each of you has a space for your work on-line.

There is also a Mailing List for this course.


This will be a projects course. Over the course of the semester you will do several projects that make use of various modes of input (gesture, touch, speech, vision, etc.) as well as various modes of output (2D graphics, 3D graphics, sound, projection onto walls and tables, etc.). You will have the opportunity (and challenge) to program "out of the box", to create games, simulations, stories, and other interesting sorts of content. The goal is to become familiar with thinking beyond keyboard and mouse, to learn how to use the computer to create experiences that can more fully immerse the user.

Mere you can see an Mpeg of last semester's Krakatoa game.

An example of our first little test environment is here.

We will also look at some landmark papers in the field of multimedia, as compiled in The New Media Reader, and students will have the opportunity to present such a paper to the class. This week I asked you to read Ted Nelson's Computer Lib/Dream Machines.


Here's a more interesting example than the one I showed last week.

Note that there is an event on Friday September 23 concerning Work Opportunities for Women in Computer Science. All those interested in the alarming (and growing) gender imbalance in computer science are encouraged to attend.

It would be fun to show our work in public at the New York Hall of Science

Which, by the way, inspired me to make these experiments in "virtual optics"

What do we want from science museum exhibits?

What are the kinds of variation of interactive experience?

ASSIGNMENT for next week:

Write a java applet that uses the same base class as the applet you wrote for this week (ie: extend But this time, think in terms of creating something you would have liked to seen as an interactive science museum when you were a kid.

Try to follow the principles in the section above entitled "What do we want from science museum exhibits?" Pick some interesting science or math topic to explore. Don't forget to use cool sounds where appropriate. Some possible topics (among thousands) are:

Post your result to the web.


For next week we are forming six teams of four. Each team is responsible for creating a game or demo for the projection table. The subject this time is narrowed down to the theme of nanotechnology.

I want you to make sure the following principles are adhered to:

There is lots of room for creativity in this topic. One possibility is conveying the idea of intermolecular forces. For example, you could explore the topic of packing, in which components have shapes, attractive/repulsing force, etc and you could play with how to pack shapes tightly and then see what kind of patterns form.

Or you could show what happens to particles as more thermal energy is introduced, and transitions happen between solid, liquid and gaseous states.

Or you could show how magnetically charged particles interact, and how they align in the presence of magnetic fields.

Or you could do something with molecular robots, such as self-replicating machines, or even machines that gradually build large things from tiny pieces, through the process of convergent assembly.


Dino Lee has put up a jar file that contains all the files you need to use the actual projection table:

As of today, it look as though the AudioClip interface that we have been using is not going to work with this new software base. However, it looks as though the java sound API should work ok.

Your group can sign up to use the black box theatre and the projection table by going to the following link:


We had the wonderful visit and critique from our friends at the New York Hall of Science You can email Eric Siegel at NYHS at the following address:

esiegel AT nyscience DOT org

It looks like we're on for Saturday November 12 starting set-up at around 10am - and then opening to the public around noon. To get there from Manhattan, take the 7 train out to 111 street in Queens and then walk two blocks (directions are on the NYHS web site). It might take up to an hour, so plan accordingly. You will all get free admission to the museum. :-)

Here is Eric's preliminary plan for that visit, in his own words:

It looks like Saturday November 12 will work for this in the Connections Discovery Lab. Probably the students could come in at 10 AM or so, and the space would open to the public at around noon. We'd like to encourage kids to come in, and to have one Explainer available as usual.

Eric Siegel has pointed out this web site with an interesting selection of java apps relating to nanoscience:

The Molecular Workbench Project

Here is a link to:

Jeff Han's framework for Procedural Graphics in OpenGL ES


Happy Halloween!

Here are some intriguing links:

Great links to work with Tangible and Graspable Interfaces

James Patten's interactive tables

Collaborative Multitouch Fingerpainting

Trends: all that matters is the display!

Trends in Augmented Reality

Wooden disks

   int stringWidth(String s, Graphics g) {
      return g.getFontMetrics(g.getFont()).stringWidth(s);


Guest lecture by Eric Petajan, founder of face2face, inc.

Really funky looking chocolate chip cookie

Some student thoughts about which links from the previous class are the most interesting.


We had a short guest lecture from Maria Mencia who is interested in students who will want to work on her upcoming London project.

Some photos/videos taken November 12 at the NY Hall of Science

Student descriptions of what they learned at the NYHS


Stephen Lewis' amazing home page

Links to student/group project ideas


Interesting link to the PlayAnywhere system by Andrew Wilson of Microsoft Research.


Here are videos of the final presentation, thanks to Chris Natali: