Mask Optimization for Image Inpainting

• Published in: IEEE access Vol. 6; pp. 69728 - 69741
• Main Authors: Isogawa, Mariko, Mikami, Dan, Iwai, Daisuke, Kimata, Hideaki, Sato, Kosuke
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 2018; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Computational efficiency; Estimation; Image restoration; Image segmentation; Inpainting; Laboratories; learning-to-rank; Optimization; Pixels; segmentation; super pixel; Task analysis; User requirements
• ISSN: 2169-3536; 2169-3536
• DOI: 10.1109/ACCESS.2018.2877401

This paper proposes a novel approach to image inpainting that optimizes the shape of masked regions given by users. In image inpainting, which removes and restores unwanted regions in images, users draw masks to specify the regions. However, it is widely known that the users typically need to adjust the masked region by trial and error until they obtain the desired natural inpainting result, because inpainting quality is significantly affected by even a slight change in the mask. This manual masking takes a great deal of users' working time and requires considerable input. To reduce the human labor required, we propose a method for masked region optimization so that good inpainting results can be automatically obtained. To this end, our approach estimates "naturalness of inpainting" for all super pixels in inpainted images and reforms an original mask on a super-pixel basis, so that the naturalness of the inpainting result is improved. The efficacy of this approach does not depend on inpainting algorithms, thus it can be applied for every inpainting method as a plug-in. To demonstrate the effectiveness of our approach, we test our algorithm with varied images and show that it outperforms the existing inpainting methods without masked region reformation.