Machine learning based rate distortion optimizer for video compression

• Main Authors: SRINIVASAMURTHY, NAGARAJAN, MALAYATH NARENDRA, NAGARAJAN, ANANTHA, SIDDARAMANNA, MOHAN, SHINGE PRASHANT SUKUMAR, BHATTI PAWAN KUMAR
• Format: Patent
• Language: Chinese; English
• Published: 22.09.2023
• Subjects: ELECTRIC COMMUNICATION TECHNIQUE; ELECTRICITY; PICTORIAL COMMUNICATION, e.g. TELEVISION

Systems and techniques for data encoding are described that use a machine learning method to generate a distortion prediction Dhat and a prediction bit rate Rhat, and perform rate distortion optimization (RDO) using Dhat and Rhat. For example, in response to one or more neural networks receiving a residual portion of a block of a video frame as an input, a video encoder may generate a distortion prediction Dhat and a bitrate residual prediction Rreshat based on an output of the one or more neural networks. The video encoder may determine a bitrate metadata prediction, Rmeta, based on metadata associated with the compression mode, and determine R as a sum of Rreshat and Rmeta. The video encoder may determine a rate distortion cost prediction, Jhat, as a function of Dhat and Rhat, and may determine a prediction mode for the compressed block based on Jhat. 描述了用于数据编码的系统和技术，其使用机器学习方法来生成失真预测D_hat和预测比特率R_hat，并使用D_hat和R_hat来执行率失真优化(RDO)。例如，响应于一个或多个神经网络接收视频帧的块的残差部分作为输入，视频编码器可以基于一个或多个神经网络的输出来生成失真预测D_hat和比特率残差预测Rres_hat