Semantic regularization of electromagnetic inverse problems

• Published in: Nature communications Vol. 15; no. 1; pp. 3869 - 12
• Main Authors: Zhang, Hongrui, Chen, Yanjin, Wang, Zhuo, Cui, Tie Jun, del Hougne, Philipp, Li, Lianlin
• Format: Journal Article
• Language: English
• Published: London Nature Publishing Group UK 08.05.2024; Nature Publishing Group; Nature Portfolio
• Subjects: 639/166/987; 639/766/930/2735; Engineering Sciences; Humanities and Social Sciences; Ill posed problems; Image reconstruction; Inverse problems; Inverse scattering; Language; Large language models; Microwave imaging; multidisciplinary; Regularization; Science; Science (multidisciplinary); Semantics; Smart buildings
• ISSN: 2041-1723; 2041-1723
• DOI: 10.1038/s41467-024-48115-5

Solving ill-posed inverse problems typically requires regularization based on prior knowledge. To date, only prior knowledge that is formulated mathematically (e.g., sparsity of the unknown) or implicitly learned from quantitative data can be used for regularization. Thereby, semantically formulated prior knowledge derived from human reasoning and recognition is excluded. Here, we introduce and demonstrate the concept of semantic regularization based on a pre-trained large language model to overcome this vexing limitation. We study the approach, first, numerically in a prototypical 2D inverse scattering problem, and, second, experimentally in 3D and 4D compressive microwave imaging problems based on programmable metasurfaces. We highlight that semantic regularization enables new forms of highly-sought privacy protection for applications like smart homes, touchless human-machine interaction and security screening: selected subjects in the scene can be concealed, or their actions and postures can be altered in the reconstruction by manipulating the semantic prior with suitable language-based control commands. Solving ill-posed inverse problems require regularisation based on prior knowledge, which is formulated mathematically or learned from data. Here, the authors demonstrated the concept of semantic regularisation based on large language model to circumvent the current limitation.