Fast Video Object Segmentation via Mask Transfer Network
Accuracy and processing speed are two important factors that affect the use of video object segmentation (VOS) in real applications. With the advanced techniques of deep neural networks, the accuracy has been significantly improved, however, the speed is still far below the real-time needs because o...
Saved in:
Main Authors | , , |
---|---|
Format | Journal Article |
Language | English |
Published |
28.08.2019
|
Subjects | |
Online Access | Get full text |
Cover
Loading…
Summary: | Accuracy and processing speed are two important factors that affect the use
of video object segmentation (VOS) in real applications. With the advanced
techniques of deep neural networks, the accuracy has been significantly
improved, however, the speed is still far below the real-time needs because of
the complicated network design, such as the requirement of the first frame
fine-tuning step. To overcome this limitation, we propose a novel mask transfer
network (MTN), which can greatly boost the processing speed of VOS and also
achieve a reasonable accuracy. The basic idea of MTN is to transfer the
reference mask to the target frame via an efficient global pixel matching
strategy. The global pixel matching between the reference frame and the target
frame is to ensure good matching results. To enhance the matching speed, we
perform the matching on a downsampled feature map with 1/32 of the original
frame size. At the same time, to preserve the detailed mask information in such
a small feature map, a mask network is designed to encode the annotated mask
information with 512 channels. Finally, an efficient feature warping method is
used to transfer the encoded reference mask to the target frame. Based on this
design, our method avoids the fine-tuning step on the first frame and does not
rely on the temporal cues and particular object categories. Therefore, it runs
very fast and can be conveniently trained only with images, as well as being
robust to unseen objects. Experiments on the DAVIS datasets demonstrate that
MTN can achieve a speed of 37 fps, and also shows a competitive accuracy in
comparison to the state-of-the-art methods. |
---|---|
DOI: | 10.48550/arxiv.1908.10717 |