Wand: 360∘ video projection mapping using a 360∘ camera

• Published in: Virtual reality : the journal of the Virtual Reality Society Vol. 27; no. 3; pp. 2015 - 2027
• Main Author: Lee, Jungjin
• Format: Journal Article
• Language: English
• Published: London Springer London 01.09.2023; Springer Nature B.V
• Subjects: Artificial Intelligence; Cameras; Computer Graphics; Computer Science; Display devices; Image Processing and Computer Vision; Original Article; Parameterization; Projectors; Spherical coordinates; Structured light patterns; Surface geometry; User Interfaces and Human Computer Interaction; 360-degree camera; Video projection; Projection mapping; Automatic geometric registration; Virtual reality; Multi-projector display
• ISSN: 1359-4338; 1434-9957
• DOI: 10.1007/s10055-023-00791-2

In a surrounding projection-based display environment (e.g., a dome theater), the viewers can enjoy a 360 ∘ video with a strong sense of immersion. Building a thriving immersive environment requires two sophisticated steps. First, to generate a single seamless screen, multiple projectors constituting the surrounding display should be carefully registered to the surface. Second, a 360 ∘ video should be mapped to the projection area by considering the display surface geometry and a sweet spot (i.e., a reference viewing position) to allow viewers to perceive the correct perspectives. In this study, Wand , a novel system utilizing a consumer 360 ∘ spherical camera as a calibration device, is proposed to efficiently solve these two issues. Wand first establishes correspondences between the 360 ∘ camera and projectors using structured light patterns, and then filters out any outliers using heuristic criteria. Next, by assuming that the camera is positioned in a sweet spot, Wand solves the geometric registration of the projectors by formulating it as a simple 2D grid mesh parameterization with the correspondence constraints. Consequently, each projector mesh is directly registered into the spherical coordinates, allowing each projector to easily render a perspective-correct view from a 360 ∘ video. We applied Wand to various environments of different dimensions and shapes. The results demonstrate that our method can be used to successfully build seamless and immersive displays and provide correct perspectives at a sweet spot.