Distinguish Sense from Nonsense: Out-of-Scope Detection for Virtual Assistants
Out of Scope (OOS) detection in Conversational AI solutions enables a chatbot to handle a conversation gracefully when it is unable to make sense of the end-user query. Accurately tagging a query as out-of-domain is particularly hard in scenarios when the chatbot is not equipped to handle a topic wh...
Saved in:
| Main Authors | , , , , |
|---|---|
| Format | Journal Article |
| Language | English |
| Published |
16.01.2023
|
| Subjects | |
| Online Access | Get full text |
Cover
Loading…
| Abstract | Out of Scope (OOS) detection in Conversational AI solutions enables a chatbot
to handle a conversation gracefully when it is unable to make sense of the
end-user query. Accurately tagging a query as out-of-domain is particularly
hard in scenarios when the chatbot is not equipped to handle a topic which has
semantic overlap with an existing topic it is trained on. We propose a simple
yet effective OOS detection method that outperforms standard OOS detection
methods in a real-world deployment of virtual assistants. We discuss the
various design and deployment considerations for a cloud platform solution to
train virtual assistants and deploy them at scale. Additionally, we propose a
collection of datasets that replicates real-world scenarios and show
comprehensive results in various settings using both offline and online
evaluation metrics. |
|---|---|
| AbstractList | Out of Scope (OOS) detection in Conversational AI solutions enables a chatbot
to handle a conversation gracefully when it is unable to make sense of the
end-user query. Accurately tagging a query as out-of-domain is particularly
hard in scenarios when the chatbot is not equipped to handle a topic which has
semantic overlap with an existing topic it is trained on. We propose a simple
yet effective OOS detection method that outperforms standard OOS detection
methods in a real-world deployment of virtual assistants. We discuss the
various design and deployment considerations for a cloud platform solution to
train virtual assistants and deploy them at scale. Additionally, we propose a
collection of datasets that replicates real-world scenarios and show
comprehensive results in various settings using both offline and online
evaluation metrics. |
| Author | Qian, Cheng Kunc, Ladislav Wang, Gengyu Potdar, Saloni Qi, Haode |
| Author_xml | – sequence: 1 givenname: Cheng surname: Qian fullname: Qian, Cheng – sequence: 2 givenname: Haode surname: Qi fullname: Qi, Haode – sequence: 3 givenname: Gengyu surname: Wang fullname: Wang, Gengyu – sequence: 4 givenname: Ladislav surname: Kunc fullname: Kunc, Ladislav – sequence: 5 givenname: Saloni surname: Potdar fullname: Potdar, Saloni |
| BackLink | https://doi.org/10.48550/arXiv.2301.06544$$DView paper in arXiv |
| BookMark | eNotj7FOwzAURT3AAIUPYMI_kJDYz4nDVrVAkap2aMUa2c4zWGrtynYQ_D2hIF3p3OlI55pc-OCRkLu6KkEKUT2o-OU-S8aruqwaAXBFNkuXsvPvo0sfdIc-IbUxHOkmTHfaI92OuQi22JlwQrrEjCa74KkNkb65mEd1oPOUJovyOd2QS6sOCW__OSP756f9YlWsty-vi_m6UE0LhW4kqywYCQaMQIbQcUStWCdrPWBnBHBUskXT1rxrtJFCDwMAAxB2GDSfkfs_7TmoP0V3VPG7_w3rz2H8B4PRS3I |
| ContentType | Journal Article |
| Copyright | http://arxiv.org/licenses/nonexclusive-distrib/1.0 |
| Copyright_xml | – notice: http://arxiv.org/licenses/nonexclusive-distrib/1.0 |
| DBID | AKY GOX |
| DOI | 10.48550/arxiv.2301.06544 |
| DatabaseName | arXiv Computer Science arXiv.org |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: GOX name: arXiv.org url: http://arxiv.org/find sourceTypes: Open Access Repository |
| DeliveryMethod | fulltext_linktorsrc |
| ExternalDocumentID | 2301_06544 |
| GroupedDBID | AKY GOX |
| ID | FETCH-LOGICAL-a674-b6820f4c84c4c5e2e493eeba2981bde9c543ea87ec71396bc85bdd442445fddb3 |
| IEDL.DBID | GOX |
| IngestDate | Mon Jan 08 05:37:57 EST 2024 |
| IsDoiOpenAccess | true |
| IsOpenAccess | true |
| IsPeerReviewed | false |
| IsScholarly | false |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-a674-b6820f4c84c4c5e2e493eeba2981bde9c543ea87ec71396bc85bdd442445fddb3 |
| OpenAccessLink | https://arxiv.org/abs/2301.06544 |
| ParticipantIDs | arxiv_primary_2301_06544 |
| PublicationCentury | 2000 |
| PublicationDate | 2023-01-16 |
| PublicationDateYYYYMMDD | 2023-01-16 |
| PublicationDate_xml | – month: 01 year: 2023 text: 2023-01-16 day: 16 |
| PublicationDecade | 2020 |
| PublicationYear | 2023 |
| Score | 1.8709615 |
| SecondaryResourceType | preprint |
| Snippet | Out of Scope (OOS) detection in Conversational AI solutions enables a chatbot
to handle a conversation gracefully when it is unable to make sense of the... |
| SourceID | arxiv |
| SourceType | Open Access Repository |
| SubjectTerms | Computer Science - Computation and Language |
| Title | Distinguish Sense from Nonsense: Out-of-Scope Detection for Virtual Assistants |
| URI | https://arxiv.org/abs/2301.06544 |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwdV09T8MwELXaTiwIBKh8ygOrASd2YrMhSqkY0qEFdav8cYEuKWrTip_PXVIEC5In27LkZ1n3zr53x9i1isE66bQAnxmh0lAKkzqL190a0A7IZlG0RZGNXtXLTM86jP9oYdzqa7Ft8wP79S3yY3lD8kfVZd0koZCt5_Gs_ZxsUnHt5v_OQ47ZdP0xEsMDtr9jd_yhPY5D1oHqiBUDukfV-2ax_uATdBuBk6iDFxTIjO2ejze1WJZiQgoRPoC6CY-qOPJJ_rZYkcKDI4zE9Kp6fcymw6fp40jsyhgIl-VKIAbJXamCUUEFDQkomwJ4l1hkjBFs0CoFZ3II6C_azAejfYyKBGi6jNGnJ6xXLSvoMy4ROAAZTMiDirL0PrPSugTXjcb7cMr6zebnn22mijnhMm9wOft_6JztUQ11eleQ2QXr1asNXKKlrf1VA_c3S0p_0g |
| link.rule.ids | 228,230,783,888 |
| linkProvider | Cornell University |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Distinguish+Sense+from+Nonsense%3A+Out-of-Scope+Detection+for+Virtual+Assistants&rft.au=Qian%2C+Cheng&rft.au=Qi%2C+Haode&rft.au=Wang%2C+Gengyu&rft.au=Kunc%2C+Ladislav&rft.date=2023-01-16&rft_id=info:doi/10.48550%2Farxiv.2301.06544&rft.externalDocID=2301_06544 |