Synthetic depth-of-field with a single-camera mobile phone

Shallow depth-of-field is commonly used by photographers to isolate a subject from a distracting background. However, standard cell phone cameras cannot produce such images optically, as their short focal lengths and small apertures capture nearly all-in-focus images. We present a system to computat...

Full description

Saved in:
Bibliographic Details
Published inACM transactions on graphics Vol. 37; no. 4; pp. 1 - 13
Main Authors Wadhwa, Neal, Garg, Rahul, Jacobs, David E., Feldman, Bryan E., Kanazawa, Nori, Carroll, Robert, Movshovitz-Attias, Yair, Barron, Jonathan T., Pritch, Yael, Levoy, Marc
Format Journal Article
LanguageEnglish
Published New York, NY, USA ACM 31.08.2018
Subjects
Artificial intelligence
Computational photography
Computer graphics
Computer vision
Computing methodologies
Image and video acquisition
Image manipulation
Image processing
segmentation
defocus
stereo
depth-of-field
Online AccessGet full text

Cover

More Information
Summary:Shallow depth-of-field is commonly used by photographers to isolate a subject from a distracting background. However, standard cell phone cameras cannot produce such images optically, as their short focal lengths and small apertures capture nearly all-in-focus images. We present a system to computationally synthesize shallow depth-of-field images with a single mobile camera and a single button press. If the image is of a person, we use a person segmentation network to separate the person and their accessories from the background. If available, we also use dense dual-pixel auto-focus hardware, effectively a 2-sample light field with an approximately 1 millimeter baseline, to compute a dense depth map. These two signals are combined and used to render a defocused image. Our system can process a 5.4 megapixel image in 4 seconds on a mobile phone, is fully automatic, and is robust enough to be used by non-experts. The modular nature of our system allows it to degrade naturally in the absence of a dual-pixel sensor or a human subject.
ISSN:0730-0301
1557-7368
DOI:10.1145/3197517.3201329