New imaging technique uses corners to see around corners
By ECE Staff
Vertical edges such as doorways or the corners of buildings are ubiquitous in everyday environments. Seeing beyond these obstacles without the use of a mirror poses a challenge, though this science fiction-like ability could help search-and-rescue workers identify hidden hazards or self-driving cars plan route trajectories. A number of methods have been developed to attempt “non-line-of-sight imaging,” but they tend to require a large number of measurements and have a small field of view, limiting their practical application. In a new paper published in Nature Communications, Professor Vivek Goyal (ECE) and his team developed a technique that can map out large-scale scenes around corners with a laser-camera system, through a small opening and from a small number of measurements.
Led by former PhD student Joshua Rapp and ECE student Charles Saunders, the paper “Seeing Around Corners with Edge-Resolved Transient Imaging” actually harnesses the presence of the vertical edge to see into the hidden space. “If you aim a laser pointer at the base of a doorway and then move the laser spot around, different amounts of the scene beyond the door frame are illuminated depending on the position of the laser,” says Goyal. “We use this to resolve the hidden space. The light that bounces back corresponds only to the part of the scene that was lit up—not the regions in shadow due to the vertical edge. Using that observation, we realized that we could subtract measurements corresponding to two different laser positions to isolate the response to light reflected from only a narrow wedge of the hidden scene.”
In addition to scanning a pulsed laser to a small number of points on an arc at the base of a vertical edge, a single photon detector focused to a fixed spot just beyond the edge captures time-resolved or “transient” measurements of the light reflected back. Differences between transient measurements from pairs of laser positions isolate the response from individual wedges of the hidden space, and computational methods reconstruct the position, orientation, and reflectivity of surfaces within each wedge. One benefit of the new approach is that it scans only a small region of the floor near the edge. Alternative methods require scanning large regions of walls, which are not always available in practical settings.
The new imaging method is the culmination of several directions of extensive research in the Goyal group. With a paper published in Science, Goyal established the field of depth imaging from extremely low light levels—as little as one photon detected per pixel—using pulsed lasers and time-resolved photon detectors. Last year, Saunders, Goyal, and former postdoc John Murray-Bruce published a paper in Nature describing a non-line-of-sight imaging technique using an ordinary digital camera to capture color images, albeit with no depth information and requiring the shape of a hidden occluding object to be known in advance. The new work takes advantage of the fact that the occluding object—the vertical edge—is visible and likewise uses probabilistic modeling of photon detections to improve the computational methods.