MTRNet++: One-stage mask-based scene text eraser

• Published in: Computer vision and image understanding Vol. 201; p. 103066
• Main Authors: Tursun, Osman, Denman, Simon, Zeng, Rui, Sivapalan, Sabesan, Sridharan, Sridha, Fookes, Clinton
• Format: Journal Article
• Language: English
• Published: Elsevier Inc 01.12.2020
• Subjects: Generative adversarial network; Image inpainting; Inpainting; Text detection; Text removal; Inpainting; Image inpainting; Text removal; Text detection; 41A10; Generative adversarial network; 65D05; 65D17; 41A05
• ISSN: 1077-3142; 1090-235X
• DOI: 10.1016/j.cviu.2020.103066

A precise, controllable, interpretable and easily trainable text removal approach is necessary for both user-specific and large-scale text removal applications. To achieve this, we propose a one-stage mask-based text inpainting network, MTRNet++. It has a novel architecture that includes mask-refine, coarse-inpainting and fine-inpainting branches, and attention blocks. With this architecture, MTRNet++ can remove text either with or without an external mask. It achieves state-of-the-art results on both the Oxford and SCUT datasets without using external ground-truth masks. The results of ablation studies demonstrate that the proposed multi-branch architecture with attention blocks is effective and essential. It also demonstrates controllability and interpretability. •The proposed MTRNet++ has a novel one-stage mask-based architecture.•MTRNet++ achieves state-of-the-art results on the Oxford and SCUT datasets.•MTRNet++ is end-to-end trainable. It converges on a large-scale dataset within an epoch.•MTRNet++ demonstrates controllability and interpretability.•We introduced some incremental modifications regarding training losses and strategy.