(Nanowerk Spotlight) Replicating human visual perception in machines has been an ongoing challenge for engineers and researchers. While computers today can recognize images or analyze video footage, these systems remain inefficient compared to the human eye. The typical machine vision systems rely on multiple devices and separate processes to capture, store, and interpret visual data. They tend to be power-hungry, slow, and bulky. In contrast, human eyes not only detect an immense range of information but also transmit and process it with remarkable efficiency. A single visual signal can be analyzed in real-time, with the brain rapidly interpreting colors, motion, and even intent with little energy. This natural sophistication has become the benchmark for developing more efficient, integrated artificial systems.
However, attempts to mimic the human visual system have been hampered by technological limitations, particularly the need to use multiple, often complex devices to replicate the distinct roles of photoreceptors, signal transmission, and processing in the brain. Efforts to build more integrated systems have struggled with balancing the many demands of sensory input, memory retention, and computational power. Advances in material science and transistor technology are bringing scientists closer to a single, efficient solution.
At the forefront of this research is a new type of optoelectronic transistor that promises to combine multiple functions of the human visual system in one compact device. As reported in Advanced Materials (“All-In-One Optoelectronic Transistors for Bio-Inspired Visual System”), this all-in-one transistor mimics both photoreceptors in the human eye and the synaptic connections that transmit signals in the brain. Unlike conventional machine vision systems that separate image capture from processing, this technology integrates those functions into a single unit, with the potential to dramatically reduce energy consumption and improve efficiency.
Illustration of Human visual system and all-in-one transistor for bio-inspired visual system. a) Schematic representation of the workflow of the human visual system. The photoreceptor cells in the retina are responsible for converting external light signals into electrical signals. The horizontal and bipolar cells facilitate the transmission and processing of signals through synaptic connections. b) A brief illustration of the device in this work and its diverse photoelectric response characteristics. The integration of this all-in-one device provides a potential solution for the fully functional bio-inspired visual system. (Image: Reproduced with permission by Wiley-VCH Verlag)
The human eye, with its photoreceptor cells – cones for color vision and rods for light intensity – processes visual stimuli far more effectively than any artificial system developed so far. Once light enters the eye, the cones and rods convert it into electrical signals, which are transmitted through neurons in the retina and sent to the brain via synapses. This conversion and transmission of signals is seamless and efficient, allowing the brain to recognize objects, colors, and motion with speed and accuracy.
In traditional machine vision, multiple devices are required to carry out each stage of this process. CMOS (complementary metal-oxide-semiconductor) sensors capture images, storage units hold the data, and processors analyze the information. Each of these steps is managed by separate hardware, leading to inefficiencies and slower processing times. This fragmentation not only increases the size and complexity of the system but also raises its energy demands, making it impractical for many applications where compactness and low power consumption are critical.
Liu’s research provides a different approach. The key innovation here is the development of a single device that can mimic both the light-sensitive photoreceptors and the memory and signal processing functions of the brain. Central to the design is the use of indium gallium zinc oxide (IGZO), a photosensitive material that forms the transistor’s channel layer, combined with hafnium-based ferroelectric materials that allow the device to retain memory. Together, these materials allow the device to simultaneously capture, process, and store visual information, much like how the human eye and brain work together.
The device itself is capable of modulating its behavior based on electrical and optical stimuli, meaning it can adjust its sensitivity depending on the type and intensity of light it encounters. This is especially important for mimicking the different roles of rod and cone cells. Under low light conditions, for example, the device can emulate the behavior of rods, which are highly sensitive to light intensity but not color. In brighter conditions, it can switch to the function of cone cells, detecting and distinguishing between different colors. By adjusting the voltage applied to the transistor, researchers can fine-tune the device to operate as either a rod-like or cone-like sensor, making it versatile enough to handle a wide range of visual tasks.
The research team constructed a 3×3 array of these optoelectronic transistors to further test the device’s ability to replicate human vision. They conducted experiments where the array was exposed to light patterns resembling different shapes and letters. The device not only captured the image but retained it in memory, displaying a “snapshot” of the image even after the light was removed. By applying different voltages to individual transistors in the array, the researchers could alter their behavior, allowing some transistors to mimic rod cells and others to act like cone cells. This capability enabled the device to process both brightness and color information, a crucial step toward building fully functional artificial vision systems.
In addition to these sensory functions, the device also exhibits synaptic behavior, mimicking the brain’s ability to retain and process information over time. This is where the ferroelectric materials play a crucial role, allowing the transistor to “remember” the light signals it receives even after they are gone. This non-volatile memory function is particularly important for applications like real-time image recognition or autonomous driving, where it is necessary to quickly process visual information and make decisions based on it.
To demonstrate the practical applications of their technology, the research team created a simulated intelligent traffic system. In this scenario, a vehicle equipped with the bio-inspired transistor system arrives at an intersection and must decide whether to stop or proceed based on the color of the traffic light and the movement of pedestrians. The transistor system, using its color discrimination function, can recognize whether the light is red or green. If the light is green, the system further assesses whether pedestrians are crossing the street. By tracking the motion of pedestrians and calculating their speed, the system can predict whether it is safe for the vehicle to continue or if it should slow down to avoid a collision. This decision-making process is handled in real-time, without the need for separate sensors, processors, or memory units.
What makes this approach particularly promising is its potential for scaling. Unlike previous systems that require multiple components, the all-in-one optoelectronic transistor could be integrated into larger arrays, providing the foundation for highly efficient machine vision systems. By reducing the number of devices needed to capture and process visual information, this technology could lead to smaller, more energy-efficient systems suitable for a wide range of applications, from autonomous vehicles to advanced robotics and even wearable devices.
Despite these advancements, challenges remain. One key issue is ensuring the uniformity of the materials used in the transistors. Variations in the properties of these materials can affect the performance of the device, particularly when scaled up for commercial production. However, with ongoing research in materials science and fabrication techniques, the team is optimistic that these challenges can be overcome, paving the way for the widespread adoption of bio-inspired vision systems.
This research offers a glimpse into the future of intelligent systems, where machine vision operates not just with efficiency but with an elegance that mirrors natural systems. The development of an all-in-one transistor capable of mimicking the complex functions of the human visual system represents a significant step toward creating more compact, efficient, and intelligent visual technologies. As these systems evolve, they may one day match the speed, sensitivity, and energy efficiency of the human eye, opening up new possibilities in robotics, automation, and beyond.
Get our Nanotechnology Spotlight updates to your inbox!
Thank you!
You have successfully joined our subscriber list.
Become a Spotlight guest author! Join our large and growing group of guest contributors. Have you just published a scientific paper or have other exciting developments to share with the nanotechnology community? Here is how to publish on nanowerk.com.
