TY - JOUR
T1 - EDoF-ToF
T2 - Extended depth of field time-of-flight imaging
AU - Tan, Jasper
AU - Boominathan, Vivek
AU - Baraniuk, Richard
AU - Veeraraghavan, Ashok
N1 - Funding Information:
Funding. Air Force Office of Scientific Research (FA9550-18-1-0478); Office of Naval Research (MURI N00014-20-1-2787, N00014-18-12571, N00014-20-1-2534, Vannevar Bush Faculty Fellowship N00014-18-1-2047); National Science Foundation (CCF-1911094, CNS-1801372, IIS-1652633, IIS-1730574, IIS-1838177).
Publisher Copyright:
© 2021 Optical Society of America
PY - 2021/11/8
Y1 - 2021/11/8
N2 - Conventional continuous-wave amplitude-modulated time-of-flight (CWAM ToF) cameras suffer from a fundamental trade-off between light throughput and depth of field (DoF): a larger lens aperture allows more light collection but suffers from significantly lower DoF. However, both high light throughput, which increases signal-to-noise ratio, and a wide DoF, which enlarges the system’s applicable depth range, are valuable for CWAM ToF applications. In this work, we propose EDoF-ToF, an algorithmic method to extend the DoF of large-aperture CWAM ToF cameras by using a neural network to deblur objects outside of the lens’s narrow focal region and thus produce an all-in-focus measurement. A key component of our work is the proposed large-aperture ToF training data simulator, which models the depth-dependent blurs and partial occlusions caused by such apertures. Contrary to conventional image deblurring where the blur model is typically linear, ToF depth maps are nonlinear functions of scene intensities, resulting in a nonlinear blur model that we also derive for our simulator. Unlike extended DoF for conventional photography where depth information needs to be encoded (or made depth-invariant) using additional hardware (phase masks, focal sweeping, etc.), ToF sensor measurements naturally encode depth information, allowing a completely software solution to extended DoF. We experimentally demonstrate EDoF-ToF increasing the DoF of a conventional ToF system by 3.6 ×, effectively achieving the DoF of a smaller lens aperture that allows 22.1 × less light. Ultimately, EDoF-ToF enables CWAM ToF cameras to enjoy the benefits of both high light throughput and a wide DoF.
AB - Conventional continuous-wave amplitude-modulated time-of-flight (CWAM ToF) cameras suffer from a fundamental trade-off between light throughput and depth of field (DoF): a larger lens aperture allows more light collection but suffers from significantly lower DoF. However, both high light throughput, which increases signal-to-noise ratio, and a wide DoF, which enlarges the system’s applicable depth range, are valuable for CWAM ToF applications. In this work, we propose EDoF-ToF, an algorithmic method to extend the DoF of large-aperture CWAM ToF cameras by using a neural network to deblur objects outside of the lens’s narrow focal region and thus produce an all-in-focus measurement. A key component of our work is the proposed large-aperture ToF training data simulator, which models the depth-dependent blurs and partial occlusions caused by such apertures. Contrary to conventional image deblurring where the blur model is typically linear, ToF depth maps are nonlinear functions of scene intensities, resulting in a nonlinear blur model that we also derive for our simulator. Unlike extended DoF for conventional photography where depth information needs to be encoded (or made depth-invariant) using additional hardware (phase masks, focal sweeping, etc.), ToF sensor measurements naturally encode depth information, allowing a completely software solution to extended DoF. We experimentally demonstrate EDoF-ToF increasing the DoF of a conventional ToF system by 3.6 ×, effectively achieving the DoF of a smaller lens aperture that allows 22.1 × less light. Ultimately, EDoF-ToF enables CWAM ToF cameras to enjoy the benefits of both high light throughput and a wide DoF.
UR - http://www.scopus.com/inward/record.url?scp=85118982814&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85118982814&partnerID=8YFLogxK
U2 - 10.1364/OE.441515
DO - 10.1364/OE.441515
M3 - Article
C2 - 34808905
AN - SCOPUS:85118982814
SN - 1094-4087
VL - 29
SP - 38540
EP - 38556
JO - Optics Express
JF - Optics Express
IS - 23
ER -