融合局部特征的多知识库常识问答模型
TP18; 当前的多知识库融合常识推理模型的输入和特征组合的方式过于简单,导致模型丢失了一些与问题和答案相关的重要信息,限制了融合外部知识的常识推理模型的效果.另外,在进行常识问答的任务时,预训练语言模型输出的问题和答案表示存在的向量各向异性问题没有得到解决.这些问题都是导致常识问答推理性能不够高的因素.针对以上问题,提出了一种基于局部特征融合的多知识库常识问答模型,改进外部知识库和问答文本的融合方式.模型将局部的问题和答案特征融入预训练语言模型全局特征,以丰富模型的特征信息,并在预测层结合了多种维度的特征进行预测;模型对于待匹配的问题和答案句子表示进行了白化处理,然后执行匹配任务.通过白化操...
Saved in:
| Published in | 计算机工程与应用 Vol. 60; no. 12; pp. 129 - 135 |
|---|---|
| Main Authors | , , |
| Format | Journal Article |
| Language | Chinese |
| Published |
上海师范大学 信息与机电工程学院,上海 201418
15.06.2024
|
| Subjects | |
| Online Access | Get full text |
| ISSN | 1002-8331 |
| DOI | 10.3778/j.issn.1002-8331.2303-0080 |
Cover
| Abstract | TP18; 当前的多知识库融合常识推理模型的输入和特征组合的方式过于简单,导致模型丢失了一些与问题和答案相关的重要信息,限制了融合外部知识的常识推理模型的效果.另外,在进行常识问答的任务时,预训练语言模型输出的问题和答案表示存在的向量各向异性问题没有得到解决.这些问题都是导致常识问答推理性能不够高的因素.针对以上问题,提出了一种基于局部特征融合的多知识库常识问答模型,改进外部知识库和问答文本的融合方式.模型将局部的问题和答案特征融入预训练语言模型全局特征,以丰富模型的特征信息,并在预测层结合了多种维度的特征进行预测;模型对于待匹配的问题和答案句子表示进行了白化处理,然后执行匹配任务.通过白化操作,模型增强了句子表示的各向同性,提升了句子向量的表征能力;还探索了不同预训练编码器(如:ALBERT、ELECTRA)在模型上的效果,以加强对知识文本的特征抽取能力,并证明了模型的稳定性.实验结果证明,在相同BERT-base编码器的实验下,模型的准确率达到78.6%,相较于基线模型,准确率提升了3.5个百分点;在ELECTRA-base编码器的实验下,模型的准确率达到80.1%. |
|---|---|
| AbstractList | TP18; 当前的多知识库融合常识推理模型的输入和特征组合的方式过于简单,导致模型丢失了一些与问题和答案相关的重要信息,限制了融合外部知识的常识推理模型的效果.另外,在进行常识问答的任务时,预训练语言模型输出的问题和答案表示存在的向量各向异性问题没有得到解决.这些问题都是导致常识问答推理性能不够高的因素.针对以上问题,提出了一种基于局部特征融合的多知识库常识问答模型,改进外部知识库和问答文本的融合方式.模型将局部的问题和答案特征融入预训练语言模型全局特征,以丰富模型的特征信息,并在预测层结合了多种维度的特征进行预测;模型对于待匹配的问题和答案句子表示进行了白化处理,然后执行匹配任务.通过白化操作,模型增强了句子表示的各向同性,提升了句子向量的表征能力;还探索了不同预训练编码器(如:ALBERT、ELECTRA)在模型上的效果,以加强对知识文本的特征抽取能力,并证明了模型的稳定性.实验结果证明,在相同BERT-base编码器的实验下,模型的准确率达到78.6%,相较于基线模型,准确率提升了3.5个百分点;在ELECTRA-base编码器的实验下,模型的准确率达到80.1%. |
| Abstract_FL | The input and feature combination of the current commonsense reasoning model based on multi-knowledge base fusion is too simple,resulting in the loss of some important information related to questions and answers,which limits the effect of the commonsense reasoning model integrating external knowledge.In addition,during the commonsense question and answer task,the problem of vector anisotropy in the output of the pre-training language model and the answer representation has not been solved.These problems are the factors that lead to the poor reasoning performance of commonsense question answering.To solve the above problems,this paper proposes a multi-knowledge base common-sense question answering model based on local feature fusion,which improves the fusion of external knowledge bases and question-answer texts.The model integrates the local question and answer features into the global features of the pre-trained language model to enrich the feature information of the model,and combines the features of multiple dimensions in the prediction layer for prediction.The model for the questions and answers to be matched.Sentence representations are whitened and then the matching task is performed.Through the whitening operation,the model enhances the isotropy of the sentence representation and improves the representation ability of the sentence vector.This paper also explores the effect of different pre-trained encoders(such as,ALBERT,ELECTRA)on the model to strengthen knowledge.The feature extraction ability of text is strengtened,and the stability of the model is proved.The experimental results show that under the same BERT-base encoder experiment,the accuracy of the model reaches 78.6%,which is 3.5 percentage points higher than the baseline model.In the experiment of ELECTRA-base encoder,the accuracy reaches 80.1%. |
| Author | 汪春梅 袁非牛 田雨晴 |
| AuthorAffiliation | 上海师范大学 信息与机电工程学院,上海 201418 |
| AuthorAffiliation_xml | – name: 上海师范大学 信息与机电工程学院,上海 201418 |
| Author_FL | YUAN Feiniu WANG Chunmei TIAN Yuqing |
| Author_FL_xml | – sequence: 1 fullname: TIAN Yuqing – sequence: 2 fullname: WANG Chunmei – sequence: 3 fullname: YUAN Feiniu |
| Author_xml | – sequence: 1 fullname: 田雨晴 – sequence: 2 fullname: 汪春梅 – sequence: 3 fullname: 袁非牛 |
| BookMark | eNo9jT1LQzEYRjNUsNb-CTeHXN_k7W2SUYpfUHDRudzEpPQiKRhE7qYoooggKAgdKoJCFxW36uC_ibf9FxYUp-dwhvMskIrve0vIEoMEhZAredILwScMgFOJyBKOgBRAQoVU_-08qYfQ05AyFKlAVSU4GV7Hm4v4fjw9HZWXH_HrpBycxadB-fA8eTuPn7dxPJ7B9P61fLn7Hj3G4dUimXPZfrD1v62R3fW1ndYmbW9vbLVW2zQwaHLaQMOFVIw7oZW00lqtubFgBexlyjmdSmHBKFTaWC6azGlEkBKFNIKlDmtk-bd7lHmX-W4n7x8e-NljJw951xRFwYE3GAfG8Qdxw12I |
| ClassificationCodes | TP18 |
| ContentType | Journal Article |
| Copyright | Copyright © Wanfang Data Co. Ltd. All Rights Reserved. |
| Copyright_xml | – notice: Copyright © Wanfang Data Co. Ltd. All Rights Reserved. |
| DBID | 2B. 4A8 92I 93N PSX TCJ |
| DOI | 10.3778/j.issn.1002-8331.2303-0080 |
| DatabaseName | Wanfang Data Journals - Hong Kong WANFANG Data Centre Wanfang Data Journals 万方数据期刊 - 香港版 China Online Journals (COJ) China Online Journals (COJ) |
| DatabaseTitleList | |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering |
| DocumentTitle_FL | Multi-Knowledge Base Common Sense Question Answering Model Based on Local Feature Fusion |
| EndPage | 135 |
| ExternalDocumentID | jsjgcyyy202412012 |
| GroupedDBID | -0Y 2B. 4A8 5XA 5XJ 92H 92I 93N ABJNI ACGFS ALMA_UNASSIGNED_HOLDINGS CCEZO CUBFJ CW9 PSX TCJ TGT U1G U5S |
| ID | FETCH-LOGICAL-s1062-43c278912f7b98e8eebb2ce0e70da9ffb587e0c939bce2761fb33088378c715f3 |
| ISSN | 1002-8331 |
| IngestDate | Thu May 29 04:10:55 EDT 2025 |
| IsPeerReviewed | false |
| IsScholarly | false |
| Issue | 12 |
| Keywords | whitening of sentence representations common sense question and answering 局部特征融合预测 常识问答 local feature fusion prediction 知识库融合 knowledge base fusion 向量白化 |
| Language | Chinese |
| LinkModel | OpenURL |
| MergedId | FETCHMERGED-LOGICAL-s1062-43c278912f7b98e8eebb2ce0e70da9ffb587e0c939bce2761fb33088378c715f3 |
| PageCount | 7 |
| ParticipantIDs | wanfang_journals_jsjgcyyy202412012 |
| PublicationCentury | 2000 |
| PublicationDate | 2024-06-15 |
| PublicationDateYYYYMMDD | 2024-06-15 |
| PublicationDate_xml | – month: 06 year: 2024 text: 2024-06-15 day: 15 |
| PublicationDecade | 2020 |
| PublicationTitle | 计算机工程与应用 |
| PublicationTitle_FL | Computer Engineering and Applications |
| PublicationYear | 2024 |
| Publisher | 上海师范大学 信息与机电工程学院,上海 201418 |
| Publisher_xml | – name: 上海师范大学 信息与机电工程学院,上海 201418 |
| SSID | ssib051375739 ssib001102935 ssj0000561668 ssib023646291 ssib057620132 |
| Score | 1.9565256 |
| Snippet | TP18;... |
| SourceID | wanfang |
| SourceType | Aggregation Database |
| StartPage | 129 |
| Title | 融合局部特征的多知识库常识问答模型 |
| URI | https://d.wanfangdata.com.cn/periodical/jsjgcyyy202412012 |
| Volume | 60 |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwnR1NaxUxcKntRQ_iJ35TxJzK1k32I8kx-94-ilgvttBbeZtmKz08wbaH9qQoYhFBUBB6qAgKvah4qx78N-tr_4Uzs_vePmyFKjyWMElmJjObN5NlZuJ5t7qh6DoRLPkyTgo_KrT2u9YWvlgSYO5cEXOLB8XZe8nMfHRnIV4YO7E1ErW0vpZP280j80r-R6sAA71iluw_aHaIFADQBv3CEzQMz2PpmGWK6YypNstipgOmFDZSjuELmWYqZAYgkinNUk1dMJgjRBumIoSYCNsI6TATI0LTYSqhwYbpkBoKf02XZloyk-Es02Ya8CRIyHBiA0ikoy4vTcyol2bBXBivW4gfkUuiKxGDSlkWIS2VDRiIiLcIegevRg1IaYk6pSUCPlhi1AxJUArGUI8iAsCiYNW9PdUQRQBOWNrIdiUonY5-CBERBmxVqaD06g4YJNQpsV9JSJGEVAulXsnVUBdIyCRTNK1DQkiQKEiyWelQGtXC4qPEUiMibjVyLlqHWZkSGFA7amjIEg2y1WpLVN2sMNhxYsSu8PqzUOWi8KrCy5_WL5RSkfVDCtNDChjrH_p4NGhs_jASc2V1ZdlubGygQDkwCd7MhJASIx4mTHv27v3GtwZXVDe-NV48kIim0FLMQxnLpsQsnGNFUJcFrQvtJzypU1NrzqoiwMj27b8zTdl0vaLbWx5x_ObOeKfrE9ukqbbfWW9s88E579RIHc_zXri_86p8_aL89vjg6W5_63v580l_-1n5cbv__tP-1-fljzfl3h40Dt596X9--2v3Q7nz8oI338nmWjN-fRuJv8oDSi20mDXORSFzrZxyLs-FdYGTwVJXF0UeK-kCq0OdWydkwos8DMGGh1JZyeMivOiN9x723CVv0gaugENtkCtdRJFwOTr54FhbaS0c-N1l72a95MX632Z18ZCirhxn0FXvZLNVrnnja4_W3XXwotfyG7V-fwNaR5FI |
| linkProvider | EBSCOhost |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=%E8%9E%8D%E5%90%88%E5%B1%80%E9%83%A8%E7%89%B9%E5%BE%81%E7%9A%84%E5%A4%9A%E7%9F%A5%E8%AF%86%E5%BA%93%E5%B8%B8%E8%AF%86%E9%97%AE%E7%AD%94%E6%A8%A1%E5%9E%8B&rft.jtitle=%E8%AE%A1%E7%AE%97%E6%9C%BA%E5%B7%A5%E7%A8%8B%E4%B8%8E%E5%BA%94%E7%94%A8&rft.au=%E7%94%B0%E9%9B%A8%E6%99%B4&rft.au=%E6%B1%AA%E6%98%A5%E6%A2%85&rft.au=%E8%A2%81%E9%9D%9E%E7%89%9B&rft.date=2024-06-15&rft.pub=%E4%B8%8A%E6%B5%B7%E5%B8%88%E8%8C%83%E5%A4%A7%E5%AD%A6+%E4%BF%A1%E6%81%AF%E4%B8%8E%E6%9C%BA%E7%94%B5%E5%B7%A5%E7%A8%8B%E5%AD%A6%E9%99%A2%2C%E4%B8%8A%E6%B5%B7+201418&rft.issn=1002-8331&rft.volume=60&rft.issue=12&rft.spage=129&rft.epage=135&rft_id=info:doi/10.3778%2Fj.issn.1002-8331.2303-0080&rft.externalDocID=jsjgcyyy202412012 |
| thumbnail_s | http://utb.summon.serialssolutions.com/2.0.0/image/custom?url=http%3A%2F%2Fwww.wanfangdata.com.cn%2Fimages%2FPeriodicalImages%2Fjsjgcyyy%2Fjsjgcyyy.jpg |