Frequency-oriented hierarchical fusion network for single image raindrop removal

• Published in: PloS one Vol. 19; no. 5; p. e0301439
• Main Authors: Wang, Juncheng, Zhang, Jie, Guo, Shuai, Li, Bo
• Format: Journal Article
• Language: English
• Published: United States Public Library of Science 23.05.2024; Public Library of Science (PLoS)
• Subjects: Algorithms; Biology and Life Sciences; Computer and Information Sciences; Deep learning; Design; Engineering and Technology; Fourier transforms; Image processing; Image Processing, Computer-Assisted - methods; Image resolution; Image restoration; Machine learning; Methods; Neural Networks, Computer; Optical properties; Rain; Rain and rainfall; Raindrops; Research and Analysis Methods; Semantics; Social Sciences; China
• ISSN: 1932-6203; 1932-6203
• DOI: 10.1371/journal.pone.0301439

Single image raindrop removal aims at recovering high-resolution images from degraded ones. However, existing methods primarily employ pixel-level supervision between image pairs to learn spatial features, thus ignoring the more discriminative frequency information. This drawback results in the loss of high-frequency structures and the generation of diverse artifacts in the restored image. To ameliorate this deficiency, we propose a novel frequency-oriented Hierarchical Fusion Network (HFNet) for raindrop image restoration. Specifically, to compensate for spatial representation deficiencies, we design a dynamic adaptive frequency loss (DAFL), which allows the model to adaptively handle the high-frequency components that are difficult to recover. To handle spatially diverse raindrops, we propose a hierarchical fusion network to efficiently learn both contextual information and spatial features. Meanwhile, a calibrated attention mechanism is proposed to facilitate the transfer of valuable information. Comparative experiments with existing methods indicate the advantages of the proposed algorithm.