Volumetric display with air-borne dust as the participating medium

Ken Perlin and Jeff Han
New York University

Significance

Currently there is no form of display that allows objects to be imaged "in thin air." There are various forms of stereoscopic or holographic display which allow an observer's two eyes to perceive two different images, as long as the observer is looking into a display screen.

There are also swept-screen volumetric displays, in which rapidly successive images are projected upon a physically rotating screen which repeatedly sweeps through a volume. In this case a volumetric image composed of points of light is directly formed in space, visible by multiple observers. The entire device needs to be enclosed in a transparent dome, for safety. Alternatively, some volumetric display devices employ two laser beams of different frequencies, focused into a cubic volume that is filled with a photo-responsive material. At any given moment, the material visibly glows at that point within the cube where the two laser beam foci meet. By optically scanning the two beams through this material, a volumetric image can be formed within the cube.

But the display described below is the first which enables the sort of scenario popularized by such films as "Forbidden Planet" and "Star Wars." In the display devices posited in those movies, an animated figure is imaged directly in the air between them, with no need for a projection screen.

Such a display, for example, allows two or more people to hold a conversation, while discussing a animated figure or other object of interest that can be floating directly in the air between them. Observers can freely pass their hands through the display volume.

The parts

  1. Infrared laser
  2. Switchable visible-light laser
  3. Optical component for merging the beams of the two lasers
  4. Computer control for switching the visible-light laser
  5. Time-varying optical beam steering mechanism
  6. Computer control for time-varying optical beam steering
  7. Linear infrared-sensitive optical position sensor
  8. Suspended dust particles in the air

What the user sees

Two or more people perceive, in the space between them, a glowing animatable object. The object is truly volumetric, and can be seen from any point of view around the object's position, as though it is floating in thin air. Observers can freely pass their hands into the space where the object appears. The volumetric image consists of fine glowing points, which imparts to it sparkling irridescent visual quality, as in the illustration below.

As the laser illuminates more dust particles in the air, the resolution of the object improves. Below are three examples of the same virtual "teddy bear," with approximately 1070, 3400 and 10600 particles illuminated, respectively.

The enabling mechanism

The key enabling mechanism is that the "display volume" in which the object appears contains a concentration of moving dust particles. The dust particles can be relatively large, on the order of 0.5-1.0 millimeter in length, so they pose no health hazard. In practice, good visual results have been achieved using lint particles, such as can be brushed off of loose woolen or synthetic fabrics. The cloud of particles can be relatively sparse, so that the individual particles are not visible to the unaided eye under normal lighting conditions.

A constant concentration of moving dust particles within the display volume is maintained through an air curtain boundary. An air curtain is a device that maintains a perpendidular laminar air flow at an open entryway, thereby preventing air from escaping through that entryway, while allowing people to freely pass through unimpeded. Air curtain technology is a standard and well practiced method of containing particulate matter in a volume of air, now in widespread use in office building entrances (to maintain temperature levels as part of an air conditioning system), and in laboratory clean rooms (to prevent the passage of airborne dust or spores into or out of open laboratory entryways).

An infrared laser is swept through the display volume, sweeping with enough repetitions per second so as to create the psychological effect of persistence of vision, generally at least 50 times per second. This sweeping movement is effected by a set of rotating or vibrating mirrors which deflect the beam in a two dimensional scanning pattern.

Coincident along the optical path of the infrared laser is a modulated visible-light laser. The two laser beams can be joined together by a dichroic prism or functionally similar optical element which reflects or transmits light depending upon the light's frequency. The two beams emerge from this optical element as a single beam, with their optical axes aligned. This second laser is capable of having its brightness modulated very rapidly over time under computer control.

The combined beam is directed toward a direction in which there are no observers, such as a floor or a ceiling or the open sky. An optical sensor is placed at a fixed lateral distance from the scanning beam. When the beam intersects a dust particle, the dust particle will scatter some of the infrared light into the detector. This sensor is a one-dimensional imaging device, which detects, and reports to a computer, at what distance from the laser source the dust particle has intersected the light beam.

A beam from infrared laser (1) strikes a dust particle (8). Some portion of the scattered light returns to the detector (7), which measures the distance of the dust particle. This measurement triggers a brief flashing of the visible laser (2) along the same beam path, thereby visibly illuminating the dust particle. The brightness of this flash depends upon the measured distance of the dust particle.
When the sensor detects an intersection between the infrared beam and a dust particle, then the computer subsystem references a shape model which is stored in a computer memory, to determine what the luminance of that model should be at the corresponding point in space. The brightness of the visible beam is set accordingly for an interval of time sufficient to illuminate the dust particle with visible light, before the scanning beam sweeps away from the particle.

As the rapidly moving dust particles intersect the sweeping beam at random intervals of time, and at random places along the beam, a volume of induced brightness is stochastically swept out within the air. An observer will see a glowing object floating in this volume.

By using more than one modulated visible laser, and by using lasers at multiple frequencies, such as a red, green and blue laser, the glowing object can be modulated to be any desired color at any point in the display volume.

One enabling mechanism for a distance detector is now described. In this embodiment, the distance detector is in-line with the laser beam itself, so that detects only a dust particle that lies along the beam, and remains insensitive to infrared reflection off of dust particles that do not lie along the beam. This can be achieved by a novel form of one-dimensional in-line detector, which consists of the following parts:

  1. The infrared laser
  2. A set of successive ring-shaped photo-detectors
  3. A convex lens with a hole in its middle
  4. The time-varying optical beam steering mechanism
  5. A computer

The beam from laser (1) travels unimpeded through a set of successive ring-shaped photo-detectors (2), and then passes unimpeded through a hole in the middle of a convex lens (3) of focal length f. the beam is then deflected by a time-varying optical beam steering mechanism (4). Returning light, which has now been scattered by a particle of dust, travels back through the optical steering mechanism (4), and is focused by the convex lens (3). After converging at a point at a distance somewhat greater than f from the lens, the light spreads out again and hits the set of ring-shaped photo-detectors (2), whose respective distances from the lens vary monotonically from 2f to 2f+ε.

When the dust particle is very far away, a greater proportion of the returning light will impinge on the detectors which are closest to the lens. When the dust particle is closer to the lens, then the distance from the lens at which light reconverges will be slightly greater. This slight shift will result in a greater proportion of the light impinging on the detectors which are furthest away from the lens. The signals from the detectors are sent to a computer (5), which analyzes the pattern of luminance received by the different detectors, and converts this pattern into an estimate of the distance of the illuminated dust particle.

Alternatively, the set of ring-shaped optical detectors (2) can be arranged in a concentric circular pattern, within a single plane of constant distance from the lens (3). Compared with the cylindrical arrangement previously described, this flat geometric arrangement has the advantage of lower cost of manufacture. In this alternate arrangement, those ring-shaped detectors which have smaller radii will receive proportionally more light when the dust particle is nearer, because in that case the light from the lens (3) will not have spread out as far by the time it impinges on the plane containing the detectors. As above, the signals from the detectors are sent to a computer (5), which analyzes the pattern of luminance received by the different detectors, and converts this pattern into an estimate of the distance of the illuminated dust particle.

Light from the dust particle is focused by lens (3) upon different rings of the concentric-ring structured detector (2). Light from a very distant dust particle will impinge on the outer ring of the detector, whereas light from a nearer dust particle will focus slightly further away from the lens, and will therefore impinge more on the inner rings of the detector.

The use of multiple laser sources

For a given density of dust particles in the air, the effective resolution and brightness of the device can be increased through the use of multiple projection units, each of which is switched independently. These multiple beams simultaneously illuminate the same dust population in the volumetric space. For a given number of moving dust particles, the use of N projection units multiplies both brightness and spatial resolution by a factor of N.