METHOD AND APPARATUS FOR IMPROVING PERFORMANCE OF ATTENTION-BASED SEQUENCE-TO-SEQUENCE MODEL

To provide a method and apparatus for improving performance of an attention-based sequence-to-sequence model.SOLUTION: A method for improving performance of an attention-based sequence-to-sequence model includes the steps of: determining an output sequence corresponding to an input sequence on the b...

Full description

Saved in:
Bibliographic Details
Main Author LEE MIN-JOONG
Format Patent
LanguageEnglish
Japanese
Published 13.12.2021
Subjects
CALCULATING
COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
COMPUTING
COUNTING
PHYSICS
Online AccessGet full text

Cover

More Information
Summary:To provide a method and apparatus for improving performance of an attention-based sequence-to-sequence model.SOLUTION: A method for improving performance of an attention-based sequence-to-sequence model includes the steps of: determining an output sequence corresponding to an input sequence on the basis of the attention-based sequence-to-sequence model; selecting one or more target attention heads among a plurality of attention heads; detecting one or more error output tokens among output tokens constituting the output sequence on the basis of the target attention head; and correcting the output sequence on the basis of the error output token.SELECTED DRAWING: Figure 5 【課題】アテンション基盤シーケンスツーシーケンスモデルの性能向上方法及び装置を提供する。【解決手段】一実施形態に係るアテンション基盤シーケンスツーシーケンスモデルの性能向上方法は、アテンション基盤シーケンスツーシーケンスモデルに基づいて入力シーケンスに対応する出力シーケンスを決定するステップと、複数のアテンションヘッドのうち、１つ以上のターゲットアテンションヘッドを選択するステップと、前記出力シーケンスを構成する出力トークンのうち、前記ターゲットアテンションヘッドに基づいて１つ以上のエラー出力トークンを検出するステップと、前記エラー出力トークンに基づいて前記出力シーケンスを補正するステップを含む。【選択図】図５
Bibliography:Application Number: JP20210068894