Proposal With Alignment: A Bi-Directional Transformer for 360° Video Viewport Proposal

• Published in: IEEE transactions on circuits and systems for video technology Vol. 34; no. 11; pp. 11423 - 11437
• Main Authors: Guo, Yichen, Xu, Mai, Jiang, Lai, Deng, Xin, Zhou, Jing, Chen, Gaoxing, Sigal, Leonid
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.11.2024; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: 360 °video; Alignment; Bidirectional control; Encoders-Decoders; Feature extraction; Helmet mounted displays; Human motion; Image processing; Modules; Object tracking; Proposals; Streaming media; Task analysis; transformer; Transformers; Video compression; viewport alignment; viewport proposal

People normally watch 360 ° videos through a head-mounted display, inside which only the content of viewports can be seen. Therefore, viewport proposal, referring to detecting potential viewport candidates, plays an important role in many 360 ° video processing tasks. In this paper, we advance the v...