Quantized Compressed Sensing by Rectified Linear Units

• Published in: IEEE transactions on information theory Vol. 67; no. 6; pp. 4125 - 4149
• Main Authors: Jung, Hans Christian, Maly, Johannes, Palzer, Lars, Stollenwerk, Alexander
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.06.2021; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Additives; Compressed sensing; Hamming distance; Hyperplanes; Mathematical analysis; Matrix methods; Measurement; Noise; Noise measurement; Q measurement; quantization; Quantization (signal); Reconstruction; rectified linear units; Robustness (mathematics); Sensors
• ISSN: 0018-9448; 1557-9654
• DOI: 10.1109/TIT.2021.3070789

This work is concerned with the problem of recovering high-dimensional signals <inline-formula> <tex-math notation="LaTeX">\mathrm {x}\in \mathbb {R}^{\text {n}} </tex-math></inline-formula> which belong to a convex set of low complexity from a small number of quantized measurements. We propose to estimate the signals via a convex program based on rectified linear units (ReLUs) for two different quantization schemes, namely one-bit and uniform multi-bit quantization. Assuming that the linear measurement process can be modelled by a sensing matrix with i.i.d. subgaussian rows, we obtain for both schemes near-optimal uniform reconstruction guarantees by adding well-designed noise to the linear measurements prior to the quantization step. In the one-bit case, we show that the program is robust against adversarial bit corruptions as well as additive noise on the linear measurements. Further, our analysis quantifies precisely how the rate-distortion relationship of the program changes depending on whether we seek reconstruction accuracies above or below the level of additive noise. The proofs rely on recent results by Dirksen and Mendelson on non-Gaussian hyperplane tessellations. Finally, we complement our theoretical analysis with numerical experiments which compare our method to other state-of-the-art methodologies.