image analogies

We present a new framework for processing images by example, called "image analogies." Rather than attempting to program individual filters by hand, we attempt to automatically learn filters from training data. For example, the following figure demonstrates an image analogy used to learn a painting style:

	:		::		:
A		A'		B		B'

The images on the left are training data; our system "learns" the transformation from A to A', and then applies that transformation to B to get B'. In other words, we compute B' to complete the analogy. (Only partial images are shown above; here are the full images).

Many examples and results are shown on these pages. For additional details of the algorithm, please see the paper.

Applications

We applied the image analogies approach to several different problems:

Toy filters, such as blurring or "embossing."
Texture synthesis from an example texture.
Super-resolution, inferring a high-resolution image from a low-resolutinon source.
Texture transfer, in which images are "texturized" with some arbitrary source texture.
Artistic filters, in which various drawing and painting styles, including oil, pastel, and pen-and-ink rendering, are synthesized based on scanned real-world examples.
Texture-by-numbers, in which realistic scences, composed of a variety of textures, are created using a simple "painting" interface.
Image colorization, where color is automatically added to grayscale images.

Someone else has downloaded and used our software:

Flight simulator terrain

Video

Texture-by-numbers video 25MB, Running time: 2:08. (Note: Windows Media Player sometimes will only show the first minute if you directly play the movie; save the movie to your hard drive to avoid this problem.)

Software

The Image Analogies software is available.

Paul Harrison's Resynthesizer GIMP plug-in does something similar, though the algorithm is different.

Papers

Image Analogies
A. Hertzmann, C. Jacobs, N. Oliver, B. Curless, D. Salesin.
SIGGRAPH 2001 Conference Proceedings.

Algorithms for Rendering in Artistic Styles
A. Hertzmann. Ph.D thesis. New York University. May, 2001.