Machine-learning based video compression

• Main Authors: Bourdev, Lubomir, Anderson, Alexander, Lew, Carissa, Nair, Sanjay, Branson, Steve, Rippel, Oren
• Format: Patent
• Language: English
• Published: 16.06.2020
• Subjects: CALCULATING; COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS; COMPUTING; COUNTING; ELECTRIC COMMUNICATION TECHNIQUE; ELECTRICITY; PHYSICS; PICTORIAL COMMUNICATION, e.g. TELEVISION

An encoder system trains a compression model that includes an autoencoder model and a frame extractor model. The encoding portion of the autoencoder is coupled to receive a set of target frames and a previous state tensor for the set of target frames and generate compressed code. The decoding portion of the autoencoder is coupled to receive the compressed code and the previous state tensor for the set of frames and generate a next state tensor for the set of target frames. The frame extractor model is coupled to receive the next state tensor and generate a set of reconstructed frames that correspond to the set of target frames by performing one or more operations on the state tensor. The state tensor for the set of frames includes information from frames of the video that can be used by the frame extractor to generate reconstructed frames.