Lookahead adversarial learning for near real-time semantic segmentation

• Published in: Computer vision and image understanding Vol. 212; p. 103271
• Main Authors: Jamali-Rad, Hadi, Szabó, Attila
• Format: Journal Article
• Language: English
• Published: Elsevier Inc 01.11.2021
• Subjects: Computer vision; Conditional adversarial training; Deep learning; Semantic segmentation; Deep learning; Computer vision; Semantic segmentation; 41A10; 65D05; 65D17; 41A05; Conditional adversarial training
• ISSN: 1077-3142; 1090-235X
• DOI: 10.1016/j.cviu.2021.103271

Semantic segmentation is one of the most fundamental problems in computer vision with significant impact on a wide variety of applications. Adversarial learning is shown to be an effective approach for improving semantic segmentation quality by enforcing higher-level pixel correlations and structural information. However, state-of-the-art semantic segmentation models cannot be easily plugged into an adversarial setting because they are not designed to accommodate convergence and stability issues in adversarial networks. We bridge this gap by building a conditional adversarial network with a state-of-the-art segmentation model (DeepLabv3+) at its core. To battle the stability issues, we introduce a novel lookahead adversarial learning (LoAd) approach with an embedded label map aggregation module. We focus on semantic segmentation models that run fast at inference for near real-time field applications. Through extensive experimentation, we demonstrate that the proposed solution can alleviate divergence issues in an adversarial semantic segmentation setting and results in considerable performance improvements (+5% in some classes) on the baseline for three standard datasets. •We propose lookahead adversarial learning (LoAd) for adversarial semantic segmentation.•LoAd runs as fast as the baselines methods upon which it is applied.•This makes LoAd suitable for near real-time field applications.•Besides avoiding class confusion, LoAd improves the performance of the baseline.•LoAd also creates structurally more consistent label maps than the baselines.