Click-pixel Cognition Fusion Network with Balanced Cut for Interactive Image Segmentation

• Published in: IEEE transactions on image processing Vol. 33; p. 1
• Main Authors: Lin, Jiacheng, Xiao, Zhiqiang, Wei, Xiaohui, Duan, Puhong, He, Xuan, Dian, Renwei, Li, Zhiyong, Li, Shutao
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.01.2024; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: balanced normalized focal loss; Click-pixel fusion; Cognition; Cognition & reasoning; Decision analysis; Decision theory; Image segmentation; interactive image segmentation; pixel imbalance problem; Pixels; Source code
• ISSN: 1057-7149; 1941-0042
• DOI: 10.1109/TIP.2023.3338003

Interactive image segmentation (IIS) has been widely used in various fields, such as medicine, industry, etc . However, some core issues, such as pixel imbalance, remain unresolved so far. Different from existing methods based on pre-processing or post-processing, we analyze the cause of pixel imbalance in depth from the two perspectives of pixel number and pixel difficulty. Based on this, a novel and unified Click-pixel Cognition Fusion network with Balanced Cut (CCF-BC) is proposed in this paper. On the one hand, the Click-pixel Cognition Fusion (CCF) module, inspired by the human cognition mechanism, is designed to increase the number of click-related pixels (namely, positive pixels) being correctly segmented, where the click and visual information are fully fused by using a progressive three-tier interaction strategy. On the other hand, a general loss, Balanced Normalized Focal Loss (BNFL), is proposed. Its core is to use a group of control coefficients related to sample gradients and forces the network to pay more attention to positive and hard-to-segment pixels during training. As a result, BNFL always tends to obtain a balanced cut of positive and negative samples in the decision space. Theoretical analysis shows that the commonly used Focal and BCE losses can be regarded as special cases of BNFL. Experiment results of five well-recognized datasets have shown the superiority of the proposed CCF-BC method compared to other state-of-the-art methods. Code will be publicly available at https://github.com/lab206/CCF-BC.