Disentangled Capsule Routing for Fast Part-Object Relational Saliency

• Published in: IEEE transactions on image processing Vol. 31; pp. 6719 - 6732
• Main Authors: Liu, Yi, Zhang, Dingwen, Liu, Nian, Xu, Shoukun, Han, Jungong
• Format: Journal Article
• Language: English
• Published: New York IEEE 2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Accuracy; capsule network; Complexity; Detectors; disentangled capsule routing; Feature extraction; Inference; multi-level information integration; Object detection; part-object relationship; Routing; Salience; Salient object detection; Semantics; Task analysis; Visualization
• ISSN: 1057-7149; 1941-0042
• DOI: 10.1109/TIP.2022.3215887

Recently, the Part-Object Relational (POR) saliency underpinned by the Capsule Network (CapsNet) has been demonstrated to be an effective modeling mechanism to improve the saliency detection accuracy. However, it is widely known that the current capsule routing operations have huge computational complexity, which seriously limited the usability of the POR saliency models in real-time applications. To this end, this paper takes an early step towards a fast POR saliency inference by proposing a novel disentangled part-object relational network. Concretely, we disentangle horizontal routing and vertical routing from the original omnidirectional capsule routing, thus generating Disentangled Capsule Routing (DCR). This mechanism enjoys two advantages. On one hand, DCR that disentangles orthogonal 1D (i.e., vertical and horizontal) routing greatly reduces parameters and routing complexity, resulting in much faster inference than omnidirectional 2D routing adopted by existing CapsNets. On the other hand, thanks to the light POR cues explored by DCR, we could conveniently integrate the part-object routing process to different feature layers in CNN, rather than just applying it to the small-scaled one as in previous works. This helps to increase saliency inference accuracy. Compared to previous POR saliency detectors, DPORTNet infers visual saliency <inline-formula> <tex-math notation="LaTeX">\left ({{5 \sim 9} }\right) \times </tex-math></inline-formula> faster, and is more accurate. DPORTNet is available under the open-source license at https://github.com/liuyi1989/DCR .