Between the Frames - Evaluation of Various Motion Interpolation Algorithms to Improve 360° Video Quality

• Published in: 2020 IEEE International Symposium on Multimedia (ISM) pp. 65 - 73
• Main Authors: Fremerey, Stephan, Hofmeyer, Frank, Goring, Steve, Keller, Dominik, Raake, Alexander
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.12.2020
• Subjects: 360 video; Cameras; frame rate; Interpolation; motion interpolation algorithms; motion judder; omnidirectional video; Quality assessment; Quality of experience; Resists; video quality; Video sequences
• DOI: 10.1109/ISM.2020.00017

With the increasing availability of 360° video content, it becomes important to provide smoothly playing videos of high quality for end users. For this reason, we compare the influence of different Motion Interpolation (MI) algorithms on 360° video quality. After conducting a pre-test with 12 video experts in [3], we found that MI is a useful tool to increase the QoE (Quality of Experience) of omnidirectional videos. As a result of the pretest, we selected three suitable MI algorithms, namely ffmpeg Motion Compensated Interpolation (MCI), Butterflow and Super-SloMo. Subsequently, we interpolated 15 entertaining and realworld omnidirectional videos with a duration of 20 seconds from 30 fps (original framerate) to 90 fps, which is the native refresh rate of the HMD used, the HTC Vive Pro. To assess QoE, we conducted two subjective tests with 24 and 27 participants. In the first test we used a Modified Paired Comparison (M-PC) method, and in the second test the Absolute Category Rating (ACR) approach. In the M-PC test, 45 stimuli were used and in the ACR test 60. Results show that for most of the 360° videos, the interpolated versions obtained significantly higher quality scores than the lower-framerate source videos, validating our hypothesis that motion interpolation can improve the overall video quality for 360° video. As expected, it was observed that the relative comparisons in the M-PC test result in larger differences in terms of quality. Generally, the ACR method lead to similar results, while reflecting a more realistic viewing situation. In addition, we compared the different MI algorithms and can conclude that with sufficient available computing power Super-SloMo should be preferred for interpolation of omnidirectional videos, while MCI also shows a good performance.