Coarse-to-fine Disentangling Demoiréing Framework for Recaptured Screen Images

• Published in: IEEE transactions on pattern analysis and machine intelligence Vol. 45; no. 8; pp. 9439 - 9453
• Main Authors: Wang, Ce, He, Bin, Wu, Shengsen, Wan, Renjie, Shi, Boxin, Duan, Ling-Yu
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.08.2023; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Aliasing; Cameras; Computational modeling; Frequency-domain analysis; Image edge detection; Image resolution; layer separation; Moire patterns; Moire Topography; Moiré pattern; Optimization; Pipelines; Screens; signal aliasing; Visualization
• ISSN: 0162-8828; 1939-3539; 2160-9292; 1939-3539
• DOI: 10.1109/TPAMI.2023.3243310

Removing the undesired moiré patterns from images capturing the contents displayed on screens is of increasing research interest, as the need for recording and sharing the instant information conveyed by the screens is growing. Previous demoiréing methods provide limited investigations into the formation process of moiré patterns to exploit moiré-specific priors for guiding the learning of demoiréing models. In this paper, we investigate the moiré pattern formation process from the perspective of signal aliasing, and correspondingly propose a coarse-to-fine disentangling demoiréing framework. In this framework, we first disentangle the moiré pattern layer and the clean image with alleviated ill-posedness based on the derivation of our moiré image formation model. Then we refine the demoiréing results exploiting both the frequency domain features and edge attention, considering moiré patterns' property on spectrum distribution and edge intensity revealed in our aliasing based analysis. Experiments on several datasets show that the proposed method performs favorably against state-of-the-art methods. Besides, the proposed method is validated to adapt well to different data sources and scales, especially on the high-resolution moiré images.