Biomedical Imaging, Autonomous Vehicle Sensors May Get Boost from AI Designed for Physical Signals

Findings

A research team led by UCLA and the University of Rochester has demonstrated a promising evolution of an imaging system designed to capture details within “complex media,” which scatter light, from depicting structures inside body tissue to seeing obstacles through heavy fog. The system uses physics-based machine learning to improve upon an existing imaging technique.

In tests with standard calibration images obscured by complex media, the new system more than doubled the signal-to-noise ratio compared to a previous generation of the technology. The system also created images in close to real time — thousandths of a second.

Background

Today, conventional applications for seeing inside complex media depend on expensive cameras that detect just beyond the limit of visible light, into the near-infrared.

In contrast, the underlying method that the researchers improved can use relatively cheap silicon-based cameras, like those found in smartphones. Introduced 10 years ago by study co-authors from the University of Rochester, the technique relies on a special film that lets through some photons and not others to convert scattered light from the near-infrared to the visible range.

However, this method tends to produce shadows in a vignetting effect that darkens the edges of images, reducing the field of view. And images can include artifacts as lighter or darker splotches.

Method

The researchers merged the existing imaging technique with a machine learning framework called DeepTimeGate. It has two stages, starting with an algorithm trained to reconstruct images mathematically. The key addition is the second algorithm, developed at UCLA, which quickly performs a reality check, constraining results based on the fundamental rules of physics.

Impact

Sensing inside complex media in near real time using silicon-based cameras would be a boon for biomedical imaging. DeepTimeGate may lead to less expensive, more effective imaging to guide surgeries, including endoscopic procedures. Labs that test for dangerous microbes or anomalous cells in cloudy fluids such as blood could use a technology like this to analyze samples without diluting or filtering them.

Another potential application is in cameras for self-driving vehicles, to detect the surroundings through blankets of rain, fog, dust or sand. In industry, the imaging system might one day help with quality control for manufacturing that involves cloudy liquids or frosted packaging, as well as for waste removal plants.

Authors

The study was conducted through a collaboration among UCLA, the University of Rochester, Stanford University, the University of Ottawa in Canada, the Air Force Research Laboratory, Clemson University and the University of Central Florida.

The study’s leading authors are Sergio Carbajo, an associate professor of electrical and computer engineering at the UCLA Samueli School of Engineering and of physics and astronomy at the UCLA College and a member of the California NanoSystems Institute at UCLA, and Robert Boyd of the University of Rochester. The co-first authors of the study are Hao Zhang, a UCLA doctoral student who also serves as the corresponding author, and Yang Xu of the University of Rochester.

Disclosures

There are no disclosures associated with this study.

Journal

The study was published in the journal Nature Light: Science & Applications.

Funding

The study was supported by the U.S. Office of Naval Research, the National Science Foundation and the Department of Energy.