Graph-based cell pattern recognition for merging the multi-modal optical microscopic image of neurons

• Published in: Computer methods and programs in biomedicine Vol. 256; p. 108392
• Main Authors: Li, Wenwei, Chen, Wu, Dai, Zimin, Chai, Xiaokang, An, Sile, Guan, Zhuang, Zhou, Wei, Chen, Jianwei, Gong, Hui, Luo, Qingming, Feng, Zhao, Li, Anan
• Format: Journal Article
• Language: English
• Published: Ireland Elsevier B.V 01.11.2024
• Subjects: Cell pattern recognition; Graph matching; High-order graph; Image registration; Image registration; Cell pattern recognition; High-order graph; Graph matching
• ISSN: 0169-2607; 1872-7565; 1872-7565
• DOI: 10.1016/j.cmpb.2024.108392

•Introduced a method that uses the distribution of neurons as a similarity measure, achieving cross-modal data matching between functional and structural imaging.•Employed a high-order graph model where triplets of neuronal nodes form hyperedges, measuring the similarity of hyperedges based on the angles of triangles, enhancing the method's robustness to scaling transformations.•Employ nonlinear optimization strategies to address the challenge of local-to-global matching.•Integrated matched neuronal positional information with image similarity to construct a joint probability model, further enhancing the accuracy of the matching process. A deep understanding of neuron structure and function is crucial for elucidating brain mechanisms, diagnosing and treating diseases. Optical microscopy, pivotal in neuroscience, illuminates neuronal shapes, projections, and electrical activities. To explore the projection of specific functional neurons, scientists have been developing optical-based multimodal imaging strategies to simultaneously capture dynamic in vivo signals and static ex vivo structures from the same neuron. However, the original position of neurons is highly susceptible to displacement during ex vivo imaging, presenting a significant challenge for integrating multimodal information at the single-neuron level. This study introduces a graph-model-based approach for cell image matching, facilitating precise and automated pairing of sparsely labeled neurons across different optical microscopic images. It has been shown that utilizing neuron distribution as a matching feature can mitigate modal differences, the high-order graph model can address scale inconsistency, and the nonlinear iteration can resolve discrepancies in neuron density. This strategy was applied to the connectivity study of the mouse visual cortex, performing cell matching between the two-photon calcium image and the HD-fMOST brain-wide anatomical image sets. Experimental results demonstrate 96.67% precision, 85.29% recall rate, and 90.63% F1 Score, comparable to expert technicians. This study builds a bridge between functional and structural imaging, offering crucial technical support for neuron classification and circuitry analysis.