SiMaLSTM-SNP: novel semantic relatedness learning model preserving both Siamese networks and membrane computing

• Published in: The Journal of supercomputing Vol. 80; no. 3; pp. 3382 - 3411
• Main Authors: Gu, Xu, Chen, Xiaoliang, Lu, Peng, Lan, Xiang, Li, Xianyong, Du, Yajun
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.02.2024; Springer Nature B.V
• Subjects: Artificial neural networks; Compilers; Computation; Computer Science; Data processing; Deep learning; Interpreters; Membranes; Natural language processing; Processor Architectures; Programming Languages; Semantics; Sentences; Attention mechanism; Long short-term memory networks; Spiking neural P system; Semantic relatedness
• ISSN: 0920-8542; 1573-0484
• DOI: 10.1007/s11227-023-05592-7

Semantic relatedness is one of the most significant aspects of natural language processing. It has been identified as a critical technology for developing intelligent systems like Siri, Microsoft Ice, Cortana, and Xiaoai. In 2014, SemEval ranked SR as the top task. While many existing studies have focused on analyzing the entailment of single phrases, advancements in deep learning have made it possible to analyze complete sentences or texts. While the natural parallelism of membrane computing has shown promise for data processing, harnessing this potential to advance semantic relatedness remains an open problem yet to be tackled. This paper proposes a novel Siamese Manhattan LSTM-SNP approach (SiMaLSTM-SNP) for the SR problem. The approach uses a collaborative Word2vec and 10-Layer Attention strategy to represent and extract sentence pairs and a Siamese LSTM-SNP structure to calculate the hidden states of sentences. The multi-head self-attention layer identifies text associations and redistributes hidden state weights. The last hidden state is extracted, and the relatedness score is calculated using the Manhattan distance. The experiments demonstrate that SiMaLSTM-SNP outperforms 17 classical SR baselines and 7 novel approaches on the standard datasets SICK and STS in terms of mean square error performance. This indicates that SiMaLSTM-SNP can accurately capture the semantic distinction between two sentences and effectively preserve their semantic information.