Nanotechnology Now – Press Release: Silicon image sensor that computes: Device speeds up, simplifies image processing for autonomous vehicles and other applications


Home > Press > Silicon image sensor that computes: Device speeds up, simplifies image processing for autonomous vehicles and other applications

SEAS researchers developed the first in-sensor processor that could be integrated into commercial silicon imaging sensor chips. The array (illustrated here) simplifies image processing for autonomous vehicles and other applications.
CREDIT
(Credit: Donhee Ham Research Group/Harvard SEAS)
SEAS researchers developed the first in-sensor processor that could be integrated into commercial silicon imaging sensor chips. The array (illustrated here) simplifies image processing for autonomous vehicles and other applications.
CREDIT
(Credit: Donhee Ham Research Group/Harvard SEAS)

Abstract:
As any driver knows, accidents can happen in the blink of an eye — so when it comes to the camera system in autonomous vehicles, processing time is critical. The time that it takes for the system to snap an image and deliver the data to the microprocessor for image processing could mean the difference between avoiding an obstacle or getting into a major accident.

Silicon image sensor that computes: Device speeds up, simplifies image processing for autonomous vehicles and other applications


Cambridge, MA | Posted on August 26th, 2022

In-sensor image processing, in which important features are extracted from raw data by the image sensor itself instead of the separate microprocessor, can speed up the visual processing. To date, demonstrations of in-sensor processing have been limited to emerging research materials which are, at least for now, difficult to incorporate into commercial systems.

Now, researchers from the Harvard John A. Paulson School of Engineering and Applied Sciences (SEAS) have developed the first in-sensor processor that could be integrated into commercial silicon imaging sensor chips ––known as complementary metal-oxide-semiconductor (CMOS) image sensors –– that are used in nearly all commercial devices that need capture visual information, including smartphones.

The research is published in Nature Electronics.

“Our work can harnesses the mainstream semiconductor electronics industry to rapidly bring in-sensor computing to a wide variety of real-world applications,” said Donhee Ham, the Gordon McKay Professor of Electrical Engineering and Applied Physics at SEAS and senior author of the paper.

Ham and his team developed a silicon photodiode array. Commercially-available image sensing chips also have a silicon photodiode array to capture images, but the team’s photodiodes are electrostatically doped, meaning that sensitivity of individual photodiodes, or pixels, to incoming light can be tuned by voltages. An array that connects multiple voltage-tunable photodiodes together can perform an analog version of multiplication and addition operations central to many image processing pipelines, extracting the relevant visual information as soon as the image is captured.

“These dynamic photodiodes can concurrently filter images as they are captured, allowing for the first stage of vision processing to be moved from the microprocessor to the sensor itself,” said Houk Jang, a postdoctoral fellow at SEAS and first author of the paper.

The silicon photodiode array can be programmed into different image filters to remove unnecessary details or noise for various applications. An imaging system in an autonomous vehicle, for example, may call for a high-pass filter to track lane markings, while other applications may call for a filter that blurs for noise reduction.

“Looking ahead, we foresee the use of this silicon-based in-sensor processor not only in machine vision applications, but also in bio-inspired applications, wherein early information processing allows for the co-location of sensor and compute units, like in the brain,” said Henry Hinton, a graduate student at SEAS and co-first author of the paper.

Next, the team aims to increase the density of photodiodes and integrate them with silicon integrated circuits.

“By replacing the standard non-programmable pixels in commercial silicon image sensors with the programmable ones developed here, imaging devices can intelligently trim out unneeded data, thus could be made more efficient in both energy and bandwidth to address the demands of the next generation of sensory applications,” said Jang.

The research was co-authored by Woo-Bin Jung, Min-Hyun Lee, Changhyun Kim, Min Park, Seoung-Ki Lee and Seongjun Park. It was supported by the Samsung Advanced Institute of Technology under Contract A30216 and by the National Science Foundation Science and Technology Center for Integrated Quantum Materials under Contract DMR-1231319.

####

For more information, please click here

Contacts:
Leah Burrows
Harvard John A. Paulson School of Engineering and Applied Sciences

Office: 617-496-1351

Copyright © Harvard John A. Paulson School of Engineering and Applied Sciences

If you have a comment, please Contact us.

Issuers of news releases, not 7th Wave, Inc. or Nanotechnology Now, are solely responsible for the accuracy of the content.

Bookmark:
Delicious
Digg
Newsvine
Google
Yahoo
Reddit
Magnoliacom
Furl
Facebook

Paper:

News and information


Mimicking termites to generate new materials August 26th, 2022


Georgia State researchers discover novel way to treat IBD with lipid nanoparticles August 26th, 2022


Bound by light: Glass nanoparticles show unexpected coupling when levitated with laser light August 26th, 2022


High-speed random number generation using self-chaotic microcavity lasers August 26th, 2022

Imaging


Dielectric metalens speed up the development of miniaturized imaging systems August 26th, 2022


An alternative to MINFLUX that enables nanometre resolution in a confocal microscope August 26th, 2022


Visualizing nanoscale structures in real time: Open-source software enables researchers to see materials in 3D while they’re still on the electron microscope August 19th, 2022


U-M researchers untangle the physics of high-temperature superconductors August 19th, 2022

Possible Futures


Georgia State researchers discover novel way to treat IBD with lipid nanoparticles August 26th, 2022


Bound by light: Glass nanoparticles show unexpected coupling when levitated with laser light August 26th, 2022


High-speed random number generation using self-chaotic microcavity lasers August 26th, 2022


Dielectric metalens speed up the development of miniaturized imaging systems August 26th, 2022

Sensors


Engineers fabricate a chip-free, wireless electronic “skin”: The device senses and wirelessly transmits signals related to pulse, sweat, and ultraviolet exposure, without bulky chips or batteries August 19th, 2022


Exploring quantum electron highways with laser light: Spiraling laser light reveals how topological insulators lose their ability to conduct electric current on their surfaces. August 19th, 2022


‘Life-like’ lasers can self-organise, adapt their structure, and cooperate July 15th, 2022


CEA-Leti Barn-Owl Inspired, Object-Localization System Uses Up to ‘5 Orders of Magnitude’ Less Energy than Existing Technology: Paper in Nature Communications Describes Neuromorphic Computing Device With ‘Virtually No Power Consumption’ When Idle, Thanks to On-Chip Non-Volatile M July 8th, 2022

Discoveries


Mimicking termites to generate new materials August 26th, 2022


Dielectric metalens speed up the development of miniaturized imaging systems August 26th, 2022


Master equation to boost quantum technologies: FQXi-funded analysis will help physicists exert exquisitely precise real-time feedback control over quantum systems August 26th, 2022


An alternative to MINFLUX that enables nanometre resolution in a confocal microscope August 26th, 2022

Announcements


Georgia State researchers discover novel way to treat IBD with lipid nanoparticles August 26th, 2022


Bound by light: Glass nanoparticles show unexpected coupling when levitated with laser light August 26th, 2022


High-speed random number generation using self-chaotic microcavity lasers August 26th, 2022


Dielectric metalens speed up the development of miniaturized imaging systems August 26th, 2022

Interviews/Book Reviews/Essays/Reports/Podcasts/Journals/White papers/Posters


Mimicking termites to generate new materials August 26th, 2022


Master equation to boost quantum technologies: FQXi-funded analysis will help physicists exert exquisitely precise real-time feedback control over quantum systems August 26th, 2022


An alternative to MINFLUX that enables nanometre resolution in a confocal microscope August 26th, 2022


Understanding outsize role of nanopores: New research reveals differences in pH, and more, about these previously mysterious environments August 26th, 2022

Automotive/Transportation


Dielectric metalens speed up the development of miniaturized imaging systems August 26th, 2022


Lithiophilic seeds and rigid arrays synergistic induced dendrite-free and stable Li anode towards long-life lithium-oxygen batteries July 22nd, 2022


A novel graphene based NiSe2 nanocrystalline array for efficient hydrogen evolution reaction July 15th, 2022


New iron catalyst could – finally! – make hydrogen fuel cells affordable: Study shows the low-cost catalyst can be a viable alternative to platinum that has stymied commercialization of the eco-friendly fuel for decades because it’s so expensive July 8th, 2022

Leave a Reply

Your email address will not be published. Required fields are marked *