Combining Markov Random Fields and Convolutional Neural Networks for Image Synthesis

This paper studies a combination of generative Markov random field (MRF) models and discriminatively trained deep convolutional neural networks (dCNNs) for synthesizing 2D images. The generative MRF acts on higher-levels of a dCNN feature pyramid, controlling the image layout at an abstract level. W...

Full description

Saved in:
Bibliographic Details
Published in2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) pp. 2479 - 2486
Main Authors Chuan Li, Wand, Michael
Format Conference Proceeding
LanguageEnglish
Published IEEE 01.06.2016
Subjects
Encoding
Image coding
Image generation
Image reconstruction
Layout
Neural networks
Online AccessGet full text

Cover

More Information
Summary:This paper studies a combination of generative Markov random field (MRF) models and discriminatively trained deep convolutional neural networks (dCNNs) for synthesizing 2D images. The generative MRF acts on higher-levels of a dCNN feature pyramid, controlling the image layout at an abstract level. We apply the method to both photographic and non-photo-realistic (artwork) synthesis tasks. The MRF regularizer prevents over-excitation artifacts and reduces implausible feature mixtures common to previous dCNN inversion approaches, permitting synthesizing photographic content with increased visual plausibility. Unlike standard MRF-based texture synthesis, the combined system can both match and adapt local features with considerable variability, yielding results far out of reach of classic generative MRF methods.
ISSN:1063-6919
DOI:10.1109/CVPR.2016.272