Feature Refinement to Improve High Resolution Image Inpainting

In this paper, we address the problem of degradation in inpainting quality of neural networks operating at high resolutions. Inpainting networks are often unable to generate globally coherent structures at resolutions higher than their training set. This is partially attributed to the receptive fiel...

Full description

Saved in:
Bibliographic Details
Published inarXiv.org
Main Authors Kulshreshtha, Prakhar, Pugh, Brian, Jiddi, Salma
Format Paper
LanguageEnglish
Published Ithaca Cornell University Library, arXiv.org 29.06.2022
Subjects
High resolution
Image resolution
Neural networks
Optimization
Online AccessGet full text

Cover

More Information
Summary:In this paper, we address the problem of degradation in inpainting quality of neural networks operating at high resolutions. Inpainting networks are often unable to generate globally coherent structures at resolutions higher than their training set. This is partially attributed to the receptive field remaining static, despite an increase in image resolution. Although downscaling the image prior to inpainting produces coherent structure, it inherently lacks detail present at higher resolutions. To get the best of both worlds, we optimize the intermediate featuremaps of a network by minimizing a multiscale consistency loss at inference. This runtime optimization improves the inpainting results and establishes a new state-of-the-art for high resolution inpainting. Code is available at: https://github.com/geomagical/lama-with-refiner/tree/refinement.
ISSN:2331-8422