Hands-On Guidance for Distilling Object Detectors

Knowledge distillation can lead to deploy-friendly networks against the plagued computational complexity problem, but previous methods neglect the feature hierarchy in detectors. Motivated by this, we propose a general framework for detection distillation. Our method, called Hands-on Guidance Distil...

Full description

Saved in:
Bibliographic Details
Published in2021 IEEE International Conference on Multimedia and Expo (ICME) pp. 1 - 6
Main Authors Qin, Yangyang, Ling, Hefei, He, Zhenghai, Shi, Yuxuan, Wu, Lei
Format Conference Proceeding
LanguageEnglish
Published IEEE 05.07.2021
Subjects
Absorption
Conferences
Detectors
Feature extraction
knowledge distillation
Knowledge engineering
Loss measurement
model compression
Object detection
Robustness
Online AccessGet full text

Cover

More Information
Summary:Knowledge distillation can lead to deploy-friendly networks against the plagued computational complexity problem, but previous methods neglect the feature hierarchy in detectors. Motivated by this, we propose a general framework for detection distillation. Our method, called Hands-on Guidance Distillation, distills the latent knowledge of all stage features for imposing more comprehensive supervision, and focuses on the essence simultaneously for promoting more intense knowledge absorption. Specifically, a series of novel mechanisms are designed elaborately, including correspondence establishment for consistency, hands-on imitation loss measure and re-weighted optimization from both micro and macro perspectives. We conduct extensive evaluations with different distillation configurations over VOC and COCO datasets, which show better performance on accuracy and speed trade-offs. Meanwhile, feasibility experiments on different structural networks further prove the robustness of our HGD.
ISSN:1945-788X
DOI:10.1109/ICME51207.2021.9428248