Engineering Researchers Develop New Camera System to See Around Corners

Breakthrough could lead to technologies that help cars avoid hidden obstacles and doctors see better detail

Jul 14, 2020

By UCLA Samueli Newsroom

A UCLA engineering professor and two colleagues in Japan have developed a new camera that can see around corners. The camera takes advantage of the same phenomenon one observes when looking through polarized sunglasses. The research paper was presented at the 2020 edition of the Conference on Computer Vision and Pattern Recognition (CVPR), the top international venue for the field. 

“Seeing around corners has been a superpower that is recently becoming possible,” said Achuta Kadambi, an assistant professor of electrical and computer engineering and an author of the study. “If the technology can be successfully applied to cameras enabling them to see around corners, it could help autonomous cars avoid accidents at blind spots, or allow biomedical engineers to create endoscopes that can help doctors see around organs.”

Setup of the experiment, new camera with polarizer on the left, hidden image, in this case a black-and-white pinwheel image, on the right.

The paper’s other co-authors are Kenichiro Tanaka and Yasuhiro Mukaigawa, both professors at the Nara Institute of Science in Technology, Japan. 

To see around corners, one needs to somehow make an ordinary wall into what is effectively a mirror — that is, a surface which reflects its surroundings. Every surface — the walls around a room, the tiles on a floor — can reflect the light of its environment. Humans cannot see all that detail with our unaided eye. But the data, in reflected light, is there and Kadambi and his colleagues used the natural polarization of light to unveil the scene.. 

“We know from high school physics that light is a wave, so imagine if we tied a jump rope to a wall and held the free end,” Kadambi said. “If we wiggle our hand up and down, the wave oscillates in a vertical orientation. If we wiggle our hand left and right, the wave oscillates in a horizontal orientation. The orientation of the oscillation is the polarization state.”

Left: Black-and-white pinwheel pattern hidden around a corner. Right: what the camera sees looking at polarized light reflected off the wall.

A polarizer, used in polarized sunglasses for example, only lets certain oscillations states through the lens.

The researchers harnessed this effect, using a polarizer in front of the camera’s lens that only lets certain oscillation states into the system. They created a novel algorithm that rearranges the polarization of light reflected off the wall into a picture that reveals everyday objects hidden around a corner.

Left: Page from a children’s book depicting an apple and hidden around a corner. Right: what the camera sees looking at polarized light reflected off the wall.

Their experiments tested letters, black-and-white patterns, colorful national flags and children’s book covers. 

In contrast to similar technologies, the researchers’ breakthrough does not require specialized hardware or additional modifications to prepare the scene. Their camera can simply leverage nature to process the hidden scene and produce an image with enough clarity and resolution to give a good idea what is around a corner.

Since polarization is a natural property of light, the researchers suggest that using it to turn walls into “mirrors” could lead to new technologies that allow someone to passively see around corners. 

Tanaka and Mukaigawa’s work on the project was supported by the Japan Society for the Promotion of Science. Kadambi’s research is supported by the National Science Foundation, a Sony Imaging Young Faculty Award, and a Google Faculty Award.

Share this article