Video-SwinUNet: Spatio-temporal Deep Learning Framework for VFSS Instance Segmentation

This paper presents a deep learning framework for medical video segmentation. Convolution neural network (CNN) and transformer-based methods have achieved great milestones in medical image segmentation tasks due to their incredible semantic feature encoding and global information comprehension abili...

Full description

Saved in:
Bibliographic Details
Published in2023 IEEE International Conference on Image Processing (ICIP) pp. 2470 - 2474
Main Authors Zeng, Chengxi, Yang, Xinyu, Smithard, David, Mirmehdi, Majid, Gambaruto, Alberto M, Burghardt, Tilo
Format Conference Proceeding
LanguageEnglish
Published IEEE 08.10.2023
Subjects
Deep learning
Dysphagia
Feature extraction
Image segmentation
Neural networks
Performance gain
Pipelines
Semantics
Swin Transformer
Video Tracking
Videofluoroscopy
Online AccessGet full text

Cover

More Information
Summary:This paper presents a deep learning framework for medical video segmentation. Convolution neural network (CNN) and transformer-based methods have achieved great milestones in medical image segmentation tasks due to their incredible semantic feature encoding and global information comprehension abilities. However, most existing approaches ignore a salient aspect of medical video data - the temporal dimension. Our proposed framework explicitly extracts features from neighbouring frames across the temporal dimension and incorporates them with a temporal feature blender, which then tokenises the high-level spatio-temporal feature to form a strong global feature encoded via a Swin Transformer. The final segmentation results are produced via a UNet-like encoder-decoder architecture. Our model outperforms other approaches by a significant margin and improves the segmentation benchmarks on the VFSS2022 dataset, achieving a dice coefficient of 0.8986 and 0.8186 for the two datasets tested. Our studies also show the efficacy of the temporal feature blending scheme and cross-dataset transferability of learned capabilities. Code and models are available at https://github.com/SimonZeng7108/Video-SwinUNet.
DOI:10.1109/ICIP49359.2023.10222212