WM-MoE: Weather-aware Multi-scale Mixture-of-Experts for Blind Adverse Weather Removal
Adverse weather removal tasks like deraining, desnowing, and dehazing are usually treated as separate tasks. However, in practical autonomous driving scenarios, the type, intensity,and mixing degree of weather are unknown, so handling each task separately cannot deal with the complex practical scena...
Saved in:
Main Authors | , , , , , , , , , |
---|---|
Format | Journal Article |
Language | English |
Published |
23.03.2023
|
Subjects | |
Online Access | Get full text |
Cover
Loading…
Summary: | Adverse weather removal tasks like deraining, desnowing, and dehazing are
usually treated as separate tasks. However, in practical autonomous driving
scenarios, the type, intensity,and mixing degree of weather are unknown, so
handling each task separately cannot deal with the complex practical scenarios.
In this paper, we study the blind adverse weather removal problem.
Mixture-of-Experts (MoE) is a popular model that adopts a learnable gate to
route the input to different expert networks. The principle of MoE involves
using adaptive networks to process different types of unknown inputs.
Therefore, MoE has great potential for blind adverse weather removal. However,
the original MoE module is inadequate for coupled multiple weather types and
fails to utilize multi-scale features for better performance. To this end, we
propose a method called Weather-aware Multi-scale MoE (WM-MoE) based on
Transformer for blind weather removal. WM-MoE includes two key designs:
WEather-Aware Router (WEAR) and Multi-Scale Experts (MSE). WEAR assigns experts
for each image token based on decoupled content and weather features, which
enhances the model's capability to process multiple adverse weathers. To obtain
discriminative weather features from images, we propose Weather Guidance
Fine-grained Contrastive Learning (WGF-CL), which utilizes weather cluster
information to guide the assignment of positive and negative samples for each
image token. Since processing different weather types requires different
receptive fields, MSE leverages multi-scale features to enhance the spatial
relationship modeling capability, facilitating the high-quality restoration of
diverse weather types and intensities. Our method achieves state-of-the-art
performance in blind adverse weather removal on two public datasets and our
dataset. We also demonstrate the advantage of our method on downstream
segmentation tasks. |
---|---|
DOI: | 10.48550/arxiv.2303.13739 |