Painterly Rendering for Video and Interaction

Aaron Hertzmann      Ken Perlin

Media Research Laboratory
Department of Computer Science
Courant Institute of Mathematical Sciences
New York University


We present new methods for painterly video processing. Based on our earlier still image processing technique, we ``paint over'' successive frames of animation, applying paint only in regions where the source video is changing. Image regions with minimal changes, such as due to video noise, are also left alone, using a simple difference masking technique. Optionally, brush strokes may be warped between frames using computed or procedural optical flow.

These methods produce video with a novel visual style distinct from previously demonstrated algorithms. Without optical flow, the video gives the effect of a painting that has been repeatedly updated and photographed, similar to paint-on-glass animation. We feel that this gives a subjective impression of the work of a human hand. With optical flow, the painting surface flows and deforms to follow the shape of the world.


Painterly cello animation Interactive living painting
Painterly cello animation
Interactive ``living painting.''

Related Links

NPAR 2000 paper:
Compressed Postscript (23MB)

Project Page: Painterly rendering

Project Page: Non-photorealistic rendering

Copyright © 2000 Aaron Hertzmann, Ken Perlin