Adaptive spatial down-sampling method based on object occupancy distribution for video coding for machines

• Published in: EURASIP journal on image and video processing Vol. 2024; no. 1; pp. 36 - 17
• Main Authors: An, Eun-bin, Kim, Ayoung, Jung, Soon-heung, Kwak, Sangwoon, Lee, Jin Young, Cheong, Won-Sik, Choo, Hyon-Gon, Seo, Kwang-deok
• Format: Journal Article
• Language: English
• Published: Cham Springer International Publishing 10.10.2024; Springer Nature B.V; SpringerOpen
• Subjects: Adaptive sampling; Biometrics; Codec; Coding; Data analysis; Demand analysis; Engineering; Error analysis; Error reduction; Image Processing and Computer Vision; Machine vision; Object occupancy distribution; Pattern Recognition; Performance evaluation; Sampling methods; Signal,Image and Speech Processing; Spatial data; Spatial down-sampling; Video coding for machines; Video data; Video object detection; Vision systems; Visual coding for humans and machines; Visual fields; Spatial down-sampling; Video coding for machines; Machine vision; Video object detection; Object occupancy distribution
• ISSN: 1687-5281; 1687-5176; 1687-5281
• DOI: 10.1186/s13640-024-00647-y

As the performance of machine vision continues to improve, it is being used in various industrial fields to analyze and generate massive amounts of video data. Although the demand for and consumption of video data by machines has increased significantly, video coding for machines needs to be improved. It is therefore necessary to consider a new codec that differs from conventional codecs based on the human visual system (HVS). Spatial down-sampling plays a critical role in video coding for machines because it reduces the volume of the video data to be processed while maintaining the shape of the data’s features that are important for the machine to reference when processing the video. An effective method of determining the intensity of spatial down-sampling as an efficient coding tool for machines is still in the early stages. Here, we propose a method of determining an optimal scale factor for spatial down-sampling by collecting and analyzing information on the number of objects and the ratio of the area occupied by the object within a picture. We compare the data reduction ratio to the machine accuracy error ratio ( DRAER ) to evaluate the performance of the proposed method. By applying the proposed method, the DRAER was found to be a maximum of 21.40 dB and a minimum of 11.94 dB . This shows that video coding gain for the machines could be achieved through the proposed method while maintaining the accuracy of machine vision tasks.