November 2, 2024

New Harvard Technology Paves the Way for Advanced Machine Vision

By

New Harvard Technology Paves The Way For Advanced Machine VisionAI Imaging Technology Concept - New Harvard Technology Paves The Way For Advanced Machine Vision

Harvard researchers have innovated a compact, single-shot polarization imaging system that simplifies traditional setups and expands applications in medical, AR, and smartphone technologies, enhancing real-time and machine learning-integrated imaging capabilities. Credit: SciTechDaily.com

Scientists have developed a compact, single-shot, and complete polarization imaging system using metasurfaces.

Think of all the information we get based on how an object interacts with wavelengths of light — a.k.a. color. Color can tell us if food is safe to eat or if a piece of metal is hot. Color is an important diagnostic tool in medicine, helping practitioners diagnose diseased tissue, inflammation, or problems in blood flow.

Companies have invested heavily to improve color in digital imaging, but wavelength is just one property of light. Polarization — how the electric field oscillates as light propagates — is also rich with information, but polarization imaging remains mostly confined to table-top laboratory settings, relying on traditional optics such as waveplates and polarizers on bulky rotational mounts.

Breakthrough in Compact Polarization Imaging

Now, researchers at the Harvard John A. Paulson School of Engineering and Applied Sciences (SEAS) have developed a compact, single-shot polarization imaging system that can provide a complete picture of polarization. By using just two thin metasurfaces, the imaging system could unlock the vast potential of polarization imaging for a range of existing and new applications, including biomedical imaging, augmented and virtual reality systems, and smartphones.

The research is published in Nature Photonics.

Mueller Matrix Imaging - New Harvard Technology Paves The Way For Advanced Machine VisionMueller Matrix Imaging - New Harvard Technology Paves The Way For Advanced Machine Vision

A unique species of beetle, Chrysina gloriosa, has a distinct response for circularly polarized light reflecting off its shell. This chiral response is correctly imaged by the new system. Credit: Aun Zaidi/Harvard SEAS

“This system, which is free of any moving parts or bulk polarization optics, will empower applications in real-time medical imaging, material characterization, machine vision, target detection, and other important areas,” said Federico Capasso, the Robert L. Wallace Professor of Applied Physics and Vinton Hayes Senior Research Fellow in Electrical Engineering at SEAS and senior author of the paper.

In previous research, Capasso and his team developed a first-of-its-kind compact polarization camera to capture so-called Stokes images, images of the polarization signature reflecting off an object – without controlling the incident illumination.

Active Polarization Imaging

“Just as the shade or even the color of an object can appear different depending on the color of the incident illumination, the polarization signature of an object depends on the polarization profile of the illumination,” said Aun Zaidi, a recent PhD graduate from Capasso’s group and first author of the paper. “In contrast to conventional polarization imaging, ‘active’ polarization imaging, known as Mueller matrix imaging, can capture the most complete polarization response of an object by controlling the incident polarization.”

Currently, Mueller matrix imaging requires a complex optical set-up with multiple rotating plates and polarizers that sequentially capture a series of images which are combined to realize a matrix representation of the image.

The simplified system developed by Capasso and his team uses two extremely thin metasurfaces — one to illuminate an object and the other to capture and analyze the light on the other side.

The first metasurface generates what’s known as polarized structured light, in which the polarization is designed to vary spatially in a unique pattern. When this polarized light reflects off or transmits through the object being illuminated, the polarization profile of the beam changes. That change is captured and analyzed by the second metasurface to construct the final image – in a single shot.

The technique allows for real-time advanced imaging, which is important for applications such as endoscopic surgery, facial recognition in smartphones, and eye tracking in AR/VR systems. It could also be combined with powerful machine-learning algorithms for applications in medical diagnostics, material classification, and pharmaceuticals.

“We have brought together two seemingly separate fields of structured light and polarized imaging to design a single system that captures the most complete polarization information. Our use of nanoengineered metasurfaces, which replace many components that would traditionally be required in a system such as this, greatly simplifies its design,” said Zaidi.

“Our single-shot and compact system provides a viable pathway for the widespread adoption of this type of imaging to empower applications requiring advanced imaging,” said Capasso.

Reference: “Metasurface-enabled single-shot and complete Mueller matrix imaging” by Aun Zaidi, Noah A. Rubin, Maryna L. Meretska, Lisa W. Li, Ahmed H. Dorrah, Joon-Suh Park and Federico Capasso, 2 May 2024, Nature Photonics.
DOI: 10.1038/s41566-024-01426-x

The Harvard Office of Technology Development has protected the intellectual property associated with this project out of Prof. Capasso’s lab and licensed the technology to Metalenz for further development.

The research was co-authored by Noah Rubin, Maryna Meretska, Lisa Li, Ahmed Dorrah and Joon-Suh Park. It was supported by the Air Force Office of Scientific Research under award Number FA9550-21-1-0312, the Office of Naval Research (ONR) under award number N00014-20-1-2450, the National Aeronautics and Space Administration (NASA) under award numbers 80NSSC21K0799 and 80NSSC20K0318, and the National Science Foundation under award no. ECCS-2025158.