Optical invention reflects the image processing power of a human eye

On a cold, sunny day, you are driving on a rural road surrounded by snow-covered fields. In an instant, your eyes process the scene, selecting individual objects to focus on (a stop sign, a barn), while the rest of the scene is blurred at the edges. Your brain stores the focused and blurred images as memories that can be viewed in your mind later when you are sitting at your desk.

Mimicking this simple, instantaneous image processing power of the human eye, Penn State electrical engineering researchers have created a metasurface: a glass slide-like optical element that uses tiny nanostructures placed at different angles to control the light. Led by corresponding author Xingjie Ni, associate professor of electrical engineering and computer science (EECS) at Penn State, the team published their invention in Natural communications.

Artificial intelligence (AI) systems require significant computing power and energy and can be slow to process images and identify objects, researchers say. In contrast, the metasurface can be used to preprocess and transform images before they are captured by a camera, allowing a computer – and AI – to process them with minimal power and data bandwidth .

The metasurface works by converting an image from the Cartesian coordinate system, where the image pixels are arranged in straight rows and columns along the x and y axes, to the log-polar system, which uses a pixel distribution like bubble.

“Like the arrangement of light receptors inside the human eye, the metasurface takes images and arranges them in a log-polar coordinate system, with denser pixels for central, focused features and more dense pixels sparse for outlying regions,” Ni said. “This allows the most important aspects of a photo to appear clearly while others remain less sharp, thereby saving data bandwidth.”

The metasurface is placed in front of a camera so that light passes through it first and transforms the image from the Cartesian system into log-polar coordinates before being digitized by a camera and transferred to a computer. Since it works using nanostructures that bend light, the metasurface requires no energy and operates at the speed of light.

“As the image of an object can vary in size or orientation, it is desirable to preprocess images to make them resilient to changes in scale and rotation,” Ni said. “This preprocessing helps AI applications recognize them as the same object more easily.”

By placing a different metasurface in front of a camera, researchers can also transform the log-polar image into the original image with Cartesian coordinates.

According to the researchers, the invention has many potential applications, including for tracking and monitoring targets to map how a car, for example, moves through a city.

“A metasurface can be used in tandem with AI systems as a preprocessor, making it easier to recognize the same car from multiple street view cameras,” Ni said. “Or if applied to a satellite, it could potentially track planes from takeoff to landing.”