Room 1221, 719 Broadway
7-9pm Mondays

Each of you has a space for your work on-line.

There is also a Mailing List for this course.

SEPTEMBER 12

This will be a projects course. Over the course of the semester you will do several projects that make use of various modes of input (gesture, touch, speech, vision, etc.) as well as various modes of output (2D graphics, 3D graphics, sound, projection onto walls and tables, etc.). You will have the opportunity (and challenge) to program "out of the box", to create games, simulations, stories, and other interesting sorts of content. The goal is to become familiar with thinking beyond keyboard and mouse, to learn how to use the computer to create experiences that can more fully immerse the user.

Mere you can see an Mpeg of last semester's Krakatoa game.

An example of our first little test environment is here.

We will also look at some landmark papers in the field of multimedia, as compiled in The New Media Reader, and students will have the opportunity to present such a paper to the class. This week I asked you to read Ted Nelson's Computer Lib/Dream Machines.

SEPTEMBER 19

Here's a more interesting example than the one I showed last week.

Note that there is an event on Friday September 23 concerning Work Opportunities for Women in Computer Science. All those interested in the alarming (and growing) gender imbalance in computer science are encouraged to attend.

It would be fun to show our work in public at the New York Hall of Science

Which, by the way, inspired me to make these experiments in "virtual optics"

What do we want from science museum exhibits?

• They should be social: multiple people can enjoy them together
• They should be physical: hands-on objects, not just mouse/keyboard
• They should be unbreakable: 3000 kids can go through a science museum in one day.
• They should be instantly learnable: the experience should be self-explanatory
• They should be fun: or else what's the point?
• They should be inspirational: not "here's the math" but rather "I want to learn more!"

What are the kinds of variation of interactive experience?

• Competitive game
• Destruction toy
  • "How does this work?"
  • "What are the components?"
  • "If I take this piece out, does it break?"
• Construction toy
  • Anything that lets people create things together
• Simulation
• Teaching tool
• Puzzle to solve
• Co-located versus networked
• Exploring space, or safari, or generated environments
• God-game: alter a virtual world
• ...

ASSIGNMENT for next week:

Write a java applet that uses the same base class as the applet you wrote for this week (ie: extend table.java). But this time, think in terms of creating something you would have liked to seen as an interactive science museum when you were a kid.

Try to follow the principles in the section above entitled "What do we want from science museum exhibits?" Pick some interesting science or math topic to explore. Don't forget to use cool sounds where appropriate. Some possible topics (among thousands) are:

• bacteria growing in a Petri dish,
• elastic materials,
• electrical conduction,
• flocking of birds,
• growth of a cell,
• how things break or shatter,
• lenses and magnifying mirrors,
• magnetism,
• making harmonics in vibrating strings,
• planetary orbits,
• plants growing toward light,
• prime numbers and factoring,
• pulleys and gears,
• radiation and critical mass,
• sending light through prisms,
• sorting numbers,

Post your result to the web.

SEPTEMBER 26

For next week we are forming six teams of four. Each team is responsible for creating a game or demo for the projection table. The subject this time is narrowed down to the theme of nanotechnology.

I want you to make sure the following principles are adhered to:

• Keep it simple and self-evident

• Engage kids actively - they should be using the pucks and when they do, they should get immediate feedback as cool interactive things happen;

• Make good use of sound.

There is lots of room for creativity in this topic. One possibility is conveying the idea of intermolecular forces. For example, you could explore the topic of packing, in which components have shapes, attractive/repulsing force, etc and you could play with how to pack shapes tightly and then see what kind of patterns form.

Or you could show what happens to particles as more thermal energy is introduced, and transitions happen between solid, liquid and gaseous states.

Or you could show how magnetically charged particles interact, and how they align in the presence of magnetic fields.

Or you could do something with molecular robots, such as self-replicating machines, or even machines that gradually build large things from tiny pieces, through the process of convergent assembly.

OCTOBER 3

Dino Lee has put up a jar file that contains all the files you need to use the actual projection table:

http://homepages.nyu.edu/~dcl255/multimedia/table.html

As of today, it look as though the AudioClip interface that we have been using is not going to work with this new software base. However, it looks as though the java sound API should work ok.

Your group can sign up to use the black box theatre and the projection table by going to the following link:

http://homepages.nyu.edu/~dcl255/multimedia/signup.html

OCTOBER 24

We had the wonderful visit and critique from our friends at the New York Hall of Science You can email Eric Siegel at NYHS at the following address:

esiegel AT nyscience DOT org

It looks like we're on for Saturday November 12 starting set-up at around 10am - and then opening to the public around noon. To get there from Manhattan, take the 7 train out to 111 street in Queens and then walk two blocks (directions are on the NYHS web site). It might take up to an hour, so plan accordingly. You will all get free admission to the museum. :-)

Here is Eric's preliminary plan for that visit, in his own words:

It looks like Saturday November 12 will work for this in the Connections Discovery Lab. Probably the students could come in at 10 AM or so, and the space would open to the public at around noon. We'd like to encourage kids to come in, and to have one Explainer available as usual.

Eric Siegel has pointed out this web site with an interesting selection of java apps relating to nanoscience:

The Molecular Workbench Project

Here is a link to:

Jeff Han's framework for Procedural Graphics in OpenGL ES

OCTOBER 31

Happy Halloween!

Here are some intriguing links:

Great links to work with Tangible and Graspable Interfaces

James Patten's interactive tables

Collaborative Multitouch Fingerpainting

Trends: all that matters is the display!

Trends in Augmented Reality

Wooden disks

   int stringWidth(String s, Graphics g) {
      return g.getFontMetrics(g.getFont()).stringWidth(s);
   }

NOVEMBER 7

Guest lecture by Eric Petajan, founder of face2face, inc.

Really funky looking chocolate chip cookie

Some student thoughts about which links from the previous class are the most interesting.

NOVEMBER 14

We had a short guest lecture from Maria Mencia who is interested in students who will want to work on her upcoming London project.

Some photos/videos taken November 12 at the NY Hall of Science

Student descriptions of what they learned at the NYHS

NOVEMBER 21

Stephen Lewis' amazing home page

Links to student/group project ideas

NOVEMBER 21

Interesting link to the PlayAnywhere system by Andrew Wilson of Microsoft Research.

FINAL PROJECT PRESENTATIONS

Here are videos of the final presentation, thanks to Chris Natali:

http://cs.nyu.edu/~cjn212/multimedia/FinalVids/