Computer-assisted imaging technologies are taking smartphone photography to the next level.
It’s only been 3 years since Apple introduced dual camera phones with the release of the iPhone 7 — allowing users to capture zoom shots without adding digital grain or the bulk of a mechanical zoom system. Apple’s solution wasn’t elegant: they just cloned the camera and used a different lens. It solved the problem— but it came at a cost.
Part of the cost of adding the extra camera was financial: the original iPhone 7 camera module cost about $26, or about 9.5% of total the production cost of the phone, but the cost rose to to $32.50 for the iPhone 8+, and the triple lens on today’s iPhone 11 Pro Max brought it up to $73.50.
In addition to the cost of components, the larger camera arrays take up more internal space, add weight to the phone, increase manufacturing complexity, and make future camera system upgrades more expensive — and if Apple adds a 4th time of flight camera to the iPhone 12 Max, the issue will be compounded.
What started as a workaround is becoming a problem — but it doesn’t have to be. Advances in optical technology are driving research into lensless cameras, which could eliminate the need for multiple cameras entirely and open the door to massive improvements in smartphone photography — and the big one is the resolution.
The Biggest Obstacle To High-Res Is The Lens
The three rear cameras on the iPhone 11 Pro all take 12-megapixel images — despite other camera manufacturers experimenting with resolutions over 40 megapixels. Apple has intentionally kept their resolutions lower to avoid ‘Airy disk’ diffraction issues created by the small camera lenses & aperture on smartphones.
As has written:
“Ideally, you want the size of an Airy disk to be small enough that it doesn’t overlap from one pixel to many others. On most recent flagships, pixel sizes aren’t much smaller than the diameter of the Airy disks present in those systems. But because they use such small sensor sizes, they’ve had to limit resolution in order to avoid Airy disk overlap…”
In other words, smartphones can’t rewrite the laws of physics: and the smaller the lens and higher the pixel density, the worse image quality will be. It also means that for smartphone cameras to continue evolving, a fundamental change is needed in the design.
Introducing The Lensless Camera
The idea of a lensless camera goes back to the mid-1800’s with the invention of the pinhole camera, but for modern imaging something more elaborate is required, which is why innovators at the University of Utah, Bell Labs, Caltech, Hitachi, and Rice University are all working on lensless camera technologies. These solutions utilize innovative focal technologies combined with computer-assisted imaging software to create images without the lens:
University Of Utah
A story by PetaPixel describes UT research that ditches the camera lens in favor of capturing light through a standard pane of glass, and then use an AI system to construct a photo out of the light captured.
“When an ordinary digital camera sensor is pointed at an object without a lens focusing light, the resulting photo looks like an undecipherable mess of colors. But that blob of pixel data actually still has enough information for computers to understand — you just need to train an AI system in “decoding” that blob to recover what the scene looked like.”
Back in 2013, Bell Labs invented a lensless camera that MIT Tech Review described as an “LCD panel that acts as an array of apertures that each allow light to pass through and a single sensor capable of detecting light three colours”. This simple, cheap device was built from off-the-shelf parts has many advantages over traditional cameras:
“This revolutionary lensless camera has a number of advantages over a conventional camera. First is the tiny amount of data required to create images. Without a lens, these images suffer none of the aberrations and focusing problems associated with lenses. The scene is entirely in focus and the resolution of the image depends on the size and number of the apertures and the point-like nature of the light sensor.”
More recently, engineers at Caltech also created a lensless camera using an Optical Phased Array (OPA), which functions by changing a controlled time-delay on the individual “pixels” in a sensor array. Caltech’s Ali Hajimiri said:
“We’ve created a single thin layer of integrated silicon photonics that emulates the lens and sensor of a digital camera, reducing the thickness and cost of digital cameras. It can mimic a regular lens, but can switch from a fish-eye to a telephoto lens instantaneously — with just a simple adjustment in the way the array receives light.”
This well-known camera manufacturer has developed a system using Moiré fringes and mathmatical fourier transforms to capture images & video without a lens, and capable of being focused after the picture is taken, according to PetaPixel:
“The secret is a ‘film imprinted with a concentric-circle pattern’ that is laid directly on top of the image sensor. The pattern casts a shadow onto the sensor during image capture, which is then compared to a second, similar pattern that is superimposed onto the image during processing.”
Another project covered by PetaPixel was the development of a Pinhole camera by Rice University — called the FlatCam, this lensless camera is thinner than a coin:
“the FlatCam is essentially a sensor chip that ditches the sensor in favor of a mask that has roots in the world of pinhole cameras. Instead of a single hole, however, the mask features a grid-like pattern with multiple apertures that each allow different light to hit the sensor’s surface. As with light field cameras (e.g. Lytro), the FlatCam’s photos can be refocused after exposures are made.”
Lensless camera technology is still in its infancy but as it continues to develop it may serve to completely replace the multi-camera arrays used in the iPhone and other smartphones on the market — which would be a massive leap forward in the world of smartphone photography.
The lens is a giant limiting factor in cameras because of its size & shape, so if you can get rid of it completely then you can utilize a larger CMOS sensor capable of capturing more light — and capable of producing larger, more detailed images & lossless digital zoom without the bulk of optics or duplication of components.
Lensless cameras like the ones above open the door to much higher resolutions, and completely new ways to focus — as well as more exotic technologies like capturing Infrared and UV wavelengths, and even applications for 3D photography from a single camera sensor.
All Rights Reserved for Android Authority