Natural language processing using an ontology-based concept embedding model

• Main Authors: Brendan Bull, Paul Lewis Felt, Andrew Hicks
• Format: Patent
• Language: English
• Published: 13.09.2023
• Subjects: CALCULATING; COMPUTING; COUNTING; ELECTRIC DIGITAL DATA PROCESSING; PHYSICS

A method 300, system, or program for a computer system to perform natural language processing (NLP) by generating a vector space model based on an ontology of concepts 310: training examples are extracted for concepts of a hierarchical ontology (200, fig.2A), wherein the training examples are based on neighbouring concepts 330; a plurality of vectors, each including one or more features, are initialized 340 and each vector corresponds to a concept, then a vector space model is generated by iteratively modifying the concept vectors to optimize a loss function 350; finally, natural language processing 360 is performed using the vector space model. Concepts can be assessed using cosine similarity between a concept vector and a mean vector of at least one of each of a parent concept and child concept. The vector space model may be a continuous bag of words model. The iterative optimisation can use both positive and negative training examples in order to optimise the loss function. The adjustment of vectors may also use a gradient descent algorithm.