Video from Stills: Lensless Imaging with Rolling Shutter

• Published in: arXiv.org
• Main Authors: Antipa, Nick, Oare, Patrick, Bostan, Emrah, Ng, Ren, Waller, Laura
• Format: Paper Journal Article
• Language: English
• Published: Ithaca Cornell University Library, arXiv.org 30.05.2019
• Subjects: CMOS; Compressibility; Computer Science - Computer Vision and Pattern Recognition; Diffusers; Frames (data processing); Frames per second; Multiplexing; Pixels; Recording; Sensors; Tradeoffs; Video compression
• ISSN: 2331-8422
• DOI: 10.48550/arxiv.1905.13221

Because image sensor chips have a finite bandwidth with which to read out pixels, recording video typically requires a trade-off between frame rate and pixel count. Compressed sensing techniques can circumvent this trade-off by assuming that the image is compressible. Here, we propose using multiplexing optics to spatially compress the scene, enabling information about the whole scene to be sampled from a row of sensor pixels, which can be read off quickly via a rolling shutter CMOS sensor. Conveniently, such multiplexing can be achieved with a simple lensless, diffuser-based imaging system. Using sparse recovery methods, we are able to recover 140 video frames at over 4,500 frames per second, all from a single captured image with a rolling shutter sensor. Our proof-of-concept system uses easily-fabricated diffusers paired with an off-the-shelf sensor. The resulting prototype enables compressive encoding of high frame rate video into a single rolling shutter exposure, and exceeds the sampling-limited performance of an equivalent global shutter system for sufficiently sparse objects.