LATENT TOKEN REPRESENTATIONS FOR PASSAGE AND ANSWER SCORING IN QUESTION ANSWERING SYSTEMS

• Main Authors: KATZ, EDWARD G, BELLER, Charles E, ACKERMANN, CHRISTOPHER F, BOXWELL, STEPHEN A, Summers, Kristen M
• Format: Patent
• Language: English
• Published: 30.05.2019

The present invention may receive the question, a plurality of the candidate answers, and a plurality of documents associated with the plurality of candidate answers in the natural language. Then the present invention may tokenize the question, the plurality of the candidate answers, and the plurality of the documents into a corresponding n-gram sequence. The present invention may map n-gram elements from the tokenized question to the n-gram elements of the plurality of the tokenized candidate answers and the plurality of the tokenized documents using the latent token representation technique. The present invention may score the plurality of tokenized candidate answers based on the latent token representation technique. Then, the present invention may determine the precise answer based on the plurality of scored candidate answers.