Systems and methods for capturing, transferring, and rendering viewpoint-adaptive three-dimensional (3D) personas

• Main Author: Venshtain, Simion
• Format: Patent
• Language: English
• Published: 11.05.2021
• Subjects: CALCULATING; COMPUTING; COUNTING; ELECTRIC COMMUNICATION TECHNIQUE; ELECTRIC DIGITAL DATA PROCESSING; ELECTRICITY; IMAGE DATA PROCESSING OR GENERATION, IN GENERAL; PHYSICS; PICTORIAL COMMUNICATION, e.g. TELEVISION; TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHICCOMMUNICATION

Systems and methods relate to receiving a plurality of video streams captured of a subject by a plurality of video cameras, each video stream including video frames time-synchronized according to a shared frame rate, each video camera having a known vantage point in a predetermined coordinate system; obtaining at least one three-dimensional (3D) mesh of the subject at the shared frame rate, the 3D mesh time-synchronized with the video frames of the video streams, the at least one mesh including a plurality of vertices with known locations in the predetermined coordinate system; calculating one or more lists of visible-vertices at the shared frame rate, each list including a subset of the plurality of vertices of the at least one 3D mesh of the subject, the subset being a function of the location of the known vantage point associated with at least one of the plurality of video cameras; generating one or more time-synchronized data streams at the shared frame rate, the one or more time-synchronized data streams including: one or more video streams encoding at least one of the plurality of video streams; and one or more geometric-data streams including the calculated one or more visible-vertices lists; and transmitting the one or more time-synchronized data streams to a receiver for rendering of a viewpoint-adaptive 3D persona of the subject.