ATTENTION-BASED NATURAL LANGUAGE PROCESSING

• Main Authors: WANG, Zhi Hu, ZHAO, Shiwan, GUO, HongLei, SU, Zhong, AN, Bang
• Format: Patent
• Language: English
• Published: 12.11.2020
• Subjects: CALCULATING; COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS; COMPUTING; COUNTING; PHYSICS

Embodiments of the present disclosure relate to attention-based neural language processing. In an embodiment, a method is disclosed. According to the method, a sentence graph is generated from a sentence containing words. The sentence graph comprises nodes representing words and edges connecting the nodes, at least one of the edges being constructed to indicate a syntactic relationship between words represented by nodes connected therebetween. Word representations for the words are determined based on the sentence graph by applying an attention mechanism on respective ones of the nodes and respective sets of neighbor nodes for the nodes. A set of neighbor nodes for a node has edges connected to the node. A sentence representation for the sentence is determined based on the word representations for use in a natural language processing task related to the sentence. In other embodiments, a system and a computer program product are disclosed.