The iPhone Of Tomorrow Could Have A GigaPixel Lensless Camera

Computer-assisted imaging technologies are taking smartphone photography to the next level.

It’s only been 3 years since Apple introduced dual camera phones with the release of the iPhone 7 — allowing users to capture zoom shots without adding digital grain or the bulk of a mechanical zoom system. Apple’s solution wasn’t elegant: they just cloned the camera and used a different lens. It solved the problem— but it came at a cost.

Part of the cost of adding the extra camera was financial: the original iPhone 7 camera module cost about $26, or about 9.5% of total the production cost of the phone, but the cost rose to to $32.50 for the iPhone 8+, and the triple lens on today’s iPhone 11 Pro Max brought it up to $73.50.

The 4-camera module on the upcoming iPhone 12
The 4-camera module on the upcoming iPhone 12

In addition to the cost of components, the larger camera arrays take up more internal space, add weight to the phone, increase manufacturing complexity, and make future camera system upgrades more expensive — and if Apple adds a 4th time of flight camera to the iPhone 12 Max, the issue will be compounded.

What started as a workaround is becoming a problem — but it doesn’t have to be. Advances in optical technology are driving research into lensless cameras, which could eliminate the need for multiple cameras entirely and open the door to massive improvements in smartphone photography — and the big one is the resolution.

The Biggest Obstacle To High-Res Is The Lens

The three rear cameras on the iPhone 11 Pro all take 12-megapixel images — despite other camera manufacturers experimenting with resolutions over 40 megapixels. Apple has intentionally kept their resolutions lower to avoid ‘Airy disk’ diffraction issues created by the small camera lenses & aperture on smartphones.

As has written:

“Ideally, you want the size of an Airy disk to be small enough that it doesn’t overlap from one pixel to many others. On most recent flagships, pixel sizes aren’t much smaller than the diameter of the Airy disks present in those systems. But because they use such small sensor sizes, they’ve had to limit resolution in order to avoid Airy disk overlap…”

In other words, smartphones can’t rewrite the laws of physics: and the smaller the lens and higher the pixel density, the worse image quality will be. It also means that for smartphone cameras to continue evolving, a fundamental change is needed in the design.

Introducing The Lensless Camera

The idea of a lensless camera goes back to the mid-1800’s with the invention of the pinhole camera, but for modern imaging something more elaborate is required, which is why innovators at the University of Utah, Bell Labs, Caltech, Hitachi, and Rice University are all working on lensless camera technologies. These solutions utilize innovative focal technologies combined with computer-assisted imaging software to create images without the lens:

UT Research scientists develop a lensless camera that uses a flat glass window
UT Research scientists develop a lensless camera that uses a flat glass window

University Of Utah

A story by PetaPixel describes UT research that ditches the camera lens in favor of capturing light through a standard pane of glass, and then use an AI system to construct a photo out of the light captured.

“When an ordinary digital camera sensor is pointed at an object without a lens focusing light, the resulting photo looks like an undecipherable mess of colors. But that blob of pixel data actually still has enough information for computers to understand — you just need to train an AI system in “decoding” that blob to recover what the scene looked like.”

The Bell Labs lensless LCD camera and an example photo taken with it
The Bell Labs lensless LCD camera and an example photo taken with it

Bell Labs

Back in 2013, Bell Labs invented a lensless camera that MIT Tech Review described as an “LCD panel that acts as an array of apertures that each allow light to pass through and a single sensor capable of detecting light three colours”. This simple, cheap device was built from off-the-shelf parts has many advantages over traditional cameras:

“This revolutionary lensless camera has a number of advantages over a conventional camera. First is the tiny amount of data required to create images. Without a lens, these images suffer none of the aberrations and focusing problems associated with lenses. The scene is entirely in focus and the resolution of the image depends on the size and number of the apertures and the point-like nature of the light sensor.”

Caltech’s chip-sized lensless camera with example images
Caltech’s chip-sized lensless camera with example images

Caltech

More recently, engineers at Caltech also created a lensless camera using an Optical Phased Array (OPA), which functions by changing a controlled time-delay on the individual “pixels” in a sensor array. Caltech’s Ali Hajimiri said:

“We’ve created a single thin layer of integrated silicon photonics that emulates the lens and sensor of a digital camera, reducing the thickness and cost of digital cameras. It can mimic a regular lens, but can switch from a fish-eye to a telephoto lens instantaneously — with just a simple adjustment in the way the array receives light.”

Hitacjhi’s lensless camera uses moiré & math, not glass
Hitacjhi’s lensless camera uses moiré & math, not glass

Hitachi

This well-known camera manufacturer has developed a system using Moiré fringes and mathmatical fourier transforms to capture images & video without a lens, and capable of being focused after the picture is taken, according to PetaPixel:

“The secret is a ‘film imprinted with a concentric-circle pattern’ that is laid directly on top of the image sensor. The pattern casts a shadow onto the sensor during image capture, which is then compared to a second, similar pattern that is superimposed onto the image during processing.”

The Rice University FlatCam and a sample photograph
The Rice University FlatCam and a sample photograph

Rice University

Another project covered by PetaPixel was the development of a Pinhole camera by Rice University — called the FlatCam, this lensless camera is thinner than a coin:

“the FlatCam is essentially a sensor chip that ditches the sensor in favor of a mask that has roots in the world of pinhole cameras. Instead of a single hole, however, the mask features a grid-like pattern with multiple apertures that each allow different light to hit the sensor’s surface. As with light field cameras (e.g. Lytro), the FlatCam’s photos can be refocused after exposures are made.”

Conclusion

Lensless camera technology is still in its infancy but as it continues to develop it may serve to completely replace the multi-camera arrays used in the iPhone and other smartphones on the market — which would be a massive leap forward in the world of smartphone photography.

The lens is a giant limiting factor in cameras because of its size & shape, so if you can get rid of it completely then you can utilize a larger CMOS sensor capable of capturing more light — and capable of producing larger, more detailed images & lossless digital zoom without the bulk of optics or duplication of components.

Lensless cameras like the ones above open the door to much higher resolutions, and completely new ways to focus — as well as more exotic technologies like capturing Infrared and UV wavelengths, and even applications for 3D photography from a single camera sensor.

All Rights Reserved for Android Authority

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.