Recursive Conditional Generative Adversarial Networks for Video Transformation

• Published in: IEEE access Vol. 7; pp. 37807 - 37821
• Main Authors: Kim, San, Suh, Doug Young
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 2019; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Data models; Gallium nitride; generative adversarial network; Generative adversarial networks; Histograms; Image quality; Image resolution; Image sequences; Image-to-image transformation; Noise reduction; reducing flicker; Task analysis; Transformations; video transformation
• DOI: 10.1109/ACCESS.2019.2906472
• ISSN: 2169-3536

Conditional generative adversarial networks (cGANs) are used in various transformation applications, such as super-resolution, colorization, image denoising, and image inpainting. So far, cGANs have been applied to the transformation of still images, but their use could be extended to the transformation of video contents, which has a much larger market. This paper considers problems with the cGAN-based transformation of video contents. The major problem is flickering caused by the discontinuity between adjacent image frames. Several postprocessing algorithms have been proposed to reduce that effect after transformation. We propose a recursive cGAN in which the previous output frame is used as an input in addition to the current input frame to reduce the flickering effect without losing the objective quality of each image. Compared with previous postprocessing algorithms, our approach performed better in terms of various evaluation metrics for video contents.