CSCI-UA-0480-006
Undergraduate Computer Graphics

60 Fifth Ave 150
Tuesdays and Thursdays, 11am-12:15pm

There are many courses that can teach you how to use commercial computer graphics packages and APIs. This course, in contrast, will teach you how to build 3D computer graphics from the ground up. This will include 3D modeling, animation, and rendering. At the end of the semester you will have built your own complete working real-time 3D computer graphics systems that runs in web browsers.

What you should already know:

If you are already familiar with JavaScript, that's great. If you are already familiar with Java, C++ or any similar high level language, you should not have any trouble picking up enough JavaScript to do this course.

Since this is a special topics course, I will assume that you are already an experienced programmer. If you are not, then I do not suggest you take this course, as there will be weekly programming assignments, and you would not be able to keep up.

Computer graphics uses a lot of matrix and vector math and some differential calculus. During the semester we will go over all of the math that you will need.

Text:

Class notes (so make sure you come to class!), will be posted on-line after each lecture.

A generally useful (but not required) reference: Computer Graphics: Principles and Practice (3rd Edition)

Office hours (at 60 Fifth Ave, room 344):

Connor DeFanti (grader): Mondays 3pm-4pm (room 342)
Ken Perlin (prof): Tuesdays 4pm-5pm
Daniel Zhang (grader): Wednesdays 3pm-4pm
Wenbo Lan (grader): Fridays 11am-noon
Mailing list:

http://www.cs.nyu.edu/mailman/listinfo/csci_ua_0480_006_sp17

Rough outline of topics we will cover (this may change):


Tuesday January 24: Introductory lecture

You will need to set up a web site for this class. As we mentioned in the first class, you can feel free to use any website that you like.

But in case you need one, Here are instructions for setting up a Courant web site.

Remember to email Connor to tell him the URL you will use for posting your assignments. Make sure to get that done before this week's Thursday class.

If you are interested in why I relate learning computer graphics to poetry, and in ideas about education in general, you might want to read this very interesting essay by Seymour Papert.

Here is a link to that bubble example we played with in class.

Here is a link to the World Builder short film that we watched at the end of class.


Thursday January 26: Introduction to WebGL shaders

Here is a zip file of the code we created in class.

Your assignment, due next Thursday February 2 before the start of class, is to create your own original fragment shader example. Don't just copy what I did with minor variations. Be original. Create cool patterns, and things that animated over time. Use the sin() and cos() functions to make crazy stripes. Have fun with it.


Tuesday January 31: Introduction to ray tracing

Here is the link to the bubble breakdown that I showed in class.

Here is a screenshot of the ray tracing to a sphere derivation we did in class. I will be expanding these to step by step notes soon.

Here is a link to the Augmented Hyperreality (making tea) video that we watched at the end of class.


Thursday February 2: Ray tracing, part 2

Tutorial on fundamentals of WebGL

WebGL Quick Reference Card

In class we started developing the code for a raytracer. Here is a code breakdown for what we developed in class.

To get started, download the zip file for the raytracer breakdown, and start to familiarize yourself with the contents of index.html in that folder.

For your assignment due before class starts next Thursday Feb 9 I'd like you to try adding to that simple raytracer in several ways. The point here is for you to experiment, and learn your way around the code.

Here are some things you can try -- feel free to add features that are not on this list. The more things you add the better:

  • Add multiple light sources.
  • See if you can figure out how to make different colored lights.
  • Create a second sphere
          Hint: The visible sphere is the one with the lowest positive value of t.
  • Animate the two spheres so that they intersect each other
  • Add textures to your sphere(s) using functions such as sin().
  • Add a cool background gradient.


Tuesday February 7: Ray tracing, part 3: Reflection, Phong Shading, Refraction, Shadows

Here is a link to Chris Landreth's Academy Award winning animated film Ryan that we saw at the end of Tuesday's class.

I was planning to post combined notes for this week's Tuesday and Thursday lectures, together with a homework assignment based on both lectures. Alas, the unexpected storm has forced me to rethink that plan.

After talking with the graders, I see that a number of people are still struggling with the second assignment. That, combined with the unexpected snow storm, has caused me to revise my thinking.

I am going to give people until this coming Tuesday, Feb 14, to finish that assignment (with extra credit given to people who already got it done by yesterday). That will give everyone a chance to catch up.

IMPORTANT: If you need help finishing the second assignment by this coming Tuesday, it is your responsibility to reach out to the graders to alert them, and you will get help.


Tuesday February 14: Implementation of Phong Shading and Reflection

In class we went over an actual implementation of some ray tracing features, including a simple version of Phong shading, reflections, and a non-uniform background. The code for that is here.

At the end of the class we watched The Centrifuge Brain Project.

Download this zip file, look through the code, and make sure you understand it.


Thursday February 16: Introduction to procedural textures

Here, in one place, are the notes for our various lectures to date.

We started to go over new material -- procedural noise-based textures. I went through the example that you can see on-line that implements a procedural bubble, and we began to discuss the implementation of the turbulence() function, which we will finish discussing in next Tuesday's class.

Videos I showed in class:

HOMEWORK due before class starts on Tuesday February 21: Implement Shadows. This should be very simple. As we said in class (and as you can see in the online notes), you just need to modify the code within the part of the Phong algorithm that loops over light sources. for each light source shoot a ray from the surface in the direction of that light source. If the ray hits another sphere, then don't add in the diffuse and specular reflection from that light.

EXTRA CREDIT due before class starts on Thursday February 23: Implement Refraction, as we discussed in class, and as you can see in the online notes. This simulates a glass sphere. You can assume an index of refraction of 1.5 if you'd like, but feel free to choose a different one. In addition to a reflection ray, also compute a refraction ray that goes into the sphere, bending according to Snell's law. When the ray hits the back of the sphere (the second quadratic root, rather than the first one), then apply Snell's law again -- this time going from a higher index of refraction to a lower one, to compute the direction into which the ray re-emerges from the sphere.


Tuesday February 21: Data structures in shaders

In class we went over several iterations of the ray tracing code. First we showed how to incorporate shadows, in raytrace3_breakdown.

Then we showed how the same thing is done more cleanly and readably by using higher level data structures: Specifically, arrays of structs -- in raytrace4_breakdown.

Finally, we showed how to create many light sources, and we put in the hooks for incorporating procedural noise, in raytrace5_breakdown.

At the end of class we watched the short film Carlitopolis.


Thursday February 23: Introduction to homogeneous coordinates

We went over how to define a plane equation, and how this can be used to define a half-space volume.

Then we showed how the intersection of half-spaces along a ray can be used as a way to ray trace to a cube.

I will be posting the notes for that material shortly.

At the end of class we watched the kitchen scene from Jurassic Park.

Your homework assignment for this week, due before the start of class next Thursday, March 2, is to creatively apply noise() and turbulence() based textures. Have fun with this. See whether you can create fun and aesthetic patterns. Use sin() and cos() functions together with the noise() and turbulence() functions to create interesting effects. Apply the textures to different parts of the rendering process to get striking or unusual effects. Use color.

This assignment is relatively easy, and it will give those of you who are still catching up a chance to get all caught up with your earlier assignments.

For extra credit: See if you can create flying saucer shapes, by doing boolean intersection of two spheres. Remember, the portion of the ray that is inside the flying saucer is the intersection of the two spherical volumes. Which means that the ray enters at the maximum of the first (entering) roots of the spheres, and exits at the minimum o the second (exiting) roots of the spheres. If that first value of t is greater than that second value of t, then the ray has missed the flying saucer.


Tuesday February 28: Matrices

We went over matrix math, including the primitive operations of identity, translate, rotate, and scale. We showed that matrix math is associative but not commutative, and how applying matrices to points (x,y,z,1) and direction vectors (x,y,z,0) can produce different results.

This material is described in depth in the on-line course notes.

At the end of class we watched the computer animated short film Over Time.


Thursday March 2: Working with a 2D canvas

Today we talked a bit about ray tracing and matrices, and we went through the section of the on-line course notes on matrices.

Then we interactively wrote some code using HTML5 CanvasRenderingContext2D. A zip file of the code we wrote is here

I good reference for the Properties and Methods of a HTML5 CanvasRenderingContext2D is here.

At the end of class we watched Fiat Lux.

Your Homework assignment, due before the start of class next Thursday, March 9, is to create something cool and fun using the tools of the CanvasRenderingContext2D -- the same environment that we played with in class today.

There are very few rules about this assignment other than to have fun with it and do something really awesome and creative. Make something that inspires you, perhaps a mini-game, a work of art, a short animation, a cool character or creature, or an interactive toy. Pretty much anything. The important thing is that you learn the tools while also using your imagination.


Tuesday March 7: Higher level discussion

In this class we briefly showed how to prototype an architectural space in computer graphics, and then how such a space looks inside the Unity Game Engine.

Then we had a higher level discussion around the ethics and implications of ubuiquitous computer graphics and its relationship to on-line data.

We talked about Jaron Lanier's influential book Who Owns the Future.


Thursday March 9: Using matrices to create a human figure

We showed how to create first a cube and then a human figure using hierarchical matrix transformations.

At the end of class we showed Pixar's classic computer animated short Luxo Jr.

Your homework, due before the start of class on Thursday March 23, is to complete the implementation of the matrix class M.js. Start with canvas_example1.zip, which contains the tumbling cube and the dancing human figure that we implemented in class.

In the version posted on-line, I have left some of the methods in M.js unimplemented. Your assignent is to implement all the methods in M.js. When you do this, the tumbling cube and the dancing figure will work the way they worked in class.

Look in M.js to see more detailed instructions about what is needed.

Note that in the function M.scale(m, v) in the homework, the second argument can either be a number or an [x,y,z] array. You should check to see which it is. One way to do that is:

   var x,y,z;
   if (v instanceof Array) {
      x = v[0];
      y = v[1];
      z = v[2];
   }
   else
      x = y = z = v;

As usual, if you have any questions, feel free to contact the graders for advice. They are there to help you.

Once you have gotten that done, then for extra credit try creating your own original animated scene in a canvas, animated by using M.js.


Tuesday March 21: Introduction to parametric surfaces

First we discussed how to implement the save() and restore() methods for matrices. The code I wrote in class is here:

M_stack = [];

M.save = function(m) {
   var i, _m = [];
   for (i = 0 ; i < m.length ; i++)
      _m.push(m[i]);                 // MAKE A COPY OF MATRIX
   M._stack.push(_m);                // PUSH IT ONTO THE STACK
}

M.restore = function(m) {
   var i, _m = M._stack.pop();       // POP THE COPY OFF THE STACK
   for (i = 0 ; i < m.length ; i++)  // COPY ITS VALUES INTO MATRIX
      m[i] = _m[i];
}

Then we started to go through the mathematics of creating parametric surfaces, such as cylindrical tubes, spheres and tori.

You can see the parametric functions we built in allnotes.html.

At the end of class we saw:

The very first shaded computer graphic film ever made


Thursday March 23: More about parametric surfaces

We wrote some more code in class, to create some examples of parametric surfaces, and we also used the same framework to create a regular octahedron, one of the five Platonic solids.

The code that we wrote in class is in canvas_example3.zip, minus the M.js source file. You need to provide the version of M.js that you wrote to make things work.

We also had a conversation about dual geometry, in which planar equations and points are swapped, and about how one shape can be the dual of another. For example, a cube and a regular octahedron are duals of each other, as are a regular doecahedron and a regular icosahedron. The dual of a regular tetrahedron is a regular tetrahedron.

We also briefly discussed four dimensional polychorons (the 4D equivalent of polyhedra), and looked at the 24-cell, a regular polychoron which is the close-packing shape in four dimensions.

Your homework, due before class this coming Thursday, March 30, is rather simple. I want you to add sphere and torus functions to S.js, and incorporate them into a working example. The only thing I ask is that you be creative: Create a creature, an animated character, a snowman, or something else original. Whatever you do, don't copy the work of anybody else in the class.

For extra credit, there are several other things you can do:

  • Implement a function that creates a cube mesh (hint: your function can create six square-shaped parametric meshes, one for each face of the cube).

  • Add extra arguments to your sphere, torus or tube functions so that you can create partial shapes, for example, a hemisphere.

  • Add end caps to your tube to create a function that makes a cylinder shape.

  • Implement some other sort of shape as well. Note that if you are already adding end-caps to your tube to make a cylinder, then you've already done the work to create a regular polygon function.

  • I've already implemented a regular octahedron in the example code. See if you can create the other four regular polyhedra: tetrahedron, cube, dodecahedron and icosahedron.

Tuesday March 28 and Thursday March 30: Introduction to splines

On Tuesday we introduced cubic splines, and talked about Hermite splines. Then on Thursday we introduced Bezier splines.

Course notes describing that material can be found, as usual, at the end of allnotes.html.

During class on Thursday we also implemented a way of drawing a multi-spline curve, where each segment of the multi-spline curve is an Hermite spline. You can download that code as canvas_example4.zip.

In that folder, note that in addition to the final result index.html I also saved two intermediate versions during class, as indexA.html and indexB.html.

At the end of class on Tuesday we watched Karl Sims' 1990 short computer animated film

Panspermia

Then at the end of class on Thursday we watched Karl Sims' 1994 short computer animated film

Evolved Virtual Creatures

Your homework, due before the start of class on Thursday April 6, is to extend what I implemented in class into a way of drawing multiple multi-spline curves (my example shows only a single multi-spline curve). You can use either the Hermite or the Bezier basis.

You should create an original example, using your own judgement to make it visually distinct and unique. One example is to try to "write" your name using multiple multi-spline curves.

For extra credit, you can do something more ambitious. Here are some ideas:

  • Show an example of both Hermite and Bezier multi-spline curves.

  • Use the cursor data to create an interactive system that allows the user to create and modify spline curves.

  • Create a cool animated figure. Here you have an opportunity to use splines in two different ways: Both to draw the animated figure that is displayed on the screen, and as a way to smoothly vary animation values over time. I will be very impressed if you do this. :-)

Remember that you have a number of ways of animting things, including your matrix commands, modifying the input values for your splines, and using the user's mouse/cursor input.

You can also make splines in 3D, using the z coordinate.


Tuesday April 4 and Thursday April 6: Introduction to B-splines

During class on Thursday we implemented an interactive B-spline editor. You can download that code as canvas_example5.zip.

Your Homework, which is due before class on Thursday April 13, is to extend this code in various interesting ways. If you open index.html in that downloaded directory, you will see a list of suggestions for how you might do this. You should implement at least one or two of the items on that list.

As usual, there will be plenty of opportunities for extra credit if you tackle some of the more ambitious items on that list, or if you do something really spectacular that surprises and impresses me.


Tuesday April 11 and Thursday April 13: WebGL and the rendering pipeline

Your assignment, due by start of class on Thursday April 20, is to download the code from this past Tuesday's class, which is zipped into file webgl_examples1.zip, and follow the instructions in the file README.txt that you will find in that folder.

Here is the flier announcing the Walt Disney recruiting visit next Thursday, April 20, at 11am.

Also, if you are free this Friday evening, April 14, at 9:30pm, I will be introducing a showing of The Matrix at the Rubin Museum of Art. Hope to see you there!

Here is a photo I took from my phone of the notes I wrote today on the whiteboard:


Tuesday April 18: More about textures

We went over the difference between solid textures and parametric textures, and discussed projection textures, which is something in-between.

Then we went over the Summed Area texture antialiasing algorithm and the MipMapping texture antialiasing algorithm.

The notes from the lecture are below.

Rather than give a new assignment, I am going to give you a chance between now and this coming Tuesday (April 25) to catch up on existing assignments, and to forge ahead with implementing extra credit add-ons, if you'd like. Of course you can always use some of that time to catch up on assignments for your other classes, if you are so inclined. :-)


Tuesday April 25: Rendering triangles in WebGL

In class we continued developing the code for a triangle based rendering system.

At the end of the class we watched the early Walt Disney / MAGI animation test for Where the Wild Things Are.


Tuesday April 27: Rendering textures in WebGL

We continued developing the triangle based rendering pipeline. In this class we created a proper way to create shapes, via the file SHAPES.js, we showed how to render textures, and finally we showed how to create a separate Material class, so that different objects can use different shader programs.

The final version of the code we developed in class is in file webgl_april_27.zip.

As I said in class, our plan is to have one more class this coming Tuesday, May 2, and then rather than having a class on Thursday May 4, aim to have two sessions the following week in which students can present their final projects: Half on Tuesday May 9, and half on Thursday May 11. I am still checking to make sure we will have the room on Tuesday May 9.

You should now start working on a final project for the semester. Think of the amount of work you should do as roughly twice as much as you would do for one week's assignment.

You can work individually, or in groups of two or three. If you choose to work in a group, remember to split up the work in such a way that it is clear who did what, and remember that each of you is responsible for doing a full "individual project" amount of work.

For your final project, you need to use the software base that we developed in class. In addition to that, feel free if you'd like, for extra credit, to do something else using a completely different external package (such as Unity or THREE.js).


Tuesday May 2: Extra topics

On Tuesday we went over various advanced topics, with opportunities for summer research. The code I showed in class is in zipfile refine_mesh.zip.

Reminder: There is no class this Thursday. Rather than holding class this morning, instead we are holding a session this coming Tuesday, May 9, during which students can present their final projects.

Also, I received the following request about course evaluations from the department:

Dear Instructor, This email is to remind you that the students in your class have been invited to evaluate your course. Please be sure to encourage them to complete the evaluations using any personal computer or mobile device.

The CAS Faculty on Course Evaluations Committee strongly encourages you to schedule a day and set aside 10-15 minutes for students to complete the course evaluation on their laptops or mobile devices in class. The evaluation can be completed quickly; however, we are asking students to consider their responses and to give constructive feedback.

The course evaluation period will end on May 9, midnight.

If any of your students tell you that they have not received their invitation or have technical difficulty accessing the link, encourage them to contact askit@nyu.edu. General questions can be sent to ce.cas@nyu.edu.

Thank you.