Evidence to Generate (E2G): A Single-agent Two-step Prompting for Context Grounded and Retrieval Augmented Reasoning
While chain-of-thought (CoT) prompting has revolutionized how LLMs perform reasoning tasks, its current methods and variations (e.g, Self-consistency, ReACT, Reflexion, Tree-of-Thoughts (ToT), Cumulative Reasoning (CR)) suffer from limitations like slowness, limited context grounding, hallucination...
Saved in:
Published in | arXiv.org |
---|---|
Main Author | |
Format | Paper |
Language | English |
Published |
Ithaca
Cornell University Library, arXiv.org
11.01.2024
|
Subjects | |
Online Access | Get full text |
Cover
Loading…
Abstract | While chain-of-thought (CoT) prompting has revolutionized how LLMs perform reasoning tasks, its current methods and variations (e.g, Self-consistency, ReACT, Reflexion, Tree-of-Thoughts (ToT), Cumulative Reasoning (CR)) suffer from limitations like slowness, limited context grounding, hallucination and inconsistent outputs. To overcome these challenges, we introduce Evidence to Generate (E2G), a novel single-agent, two-step prompting framework. Instead of unverified reasoning claims, this innovative approach leverages the power of "evidence for decision making" by first focusing exclusively on the thought sequences (the series of intermediate steps) explicitly mentioned in the context which then serve as extracted evidence, guiding the LLM's output generation process with greater precision and efficiency. This simple yet powerful approach unlocks the true potential of chain-of-thought like prompting, paving the way for faster, more reliable, and more contextually aware reasoning in LLMs. \tool achieves remarkable results robustly across a wide range of knowledge-intensive reasoning and generation tasks, surpassing baseline approaches with state-of-the-art LLMs. For example, (i) on LogiQA benchmark using GPT-4 as backbone model, \tool achieves a new state-of-the Accuracy of 53.8% exceeding CoT by 18%, ToT by 11%, CR by 9% (ii) a variant of E2G with PaLM2 outperforms the variable-shot performance of Gemini Ultra by 0.9 F1 points, reaching an F1 score of 83.3 on a subset of DROP. |
---|---|
AbstractList | While chain-of-thought (CoT) prompting has revolutionized how LLMs perform reasoning tasks, its current methods and variations (e.g, Self-consistency, ReACT, Reflexion, Tree-of-Thoughts (ToT), Cumulative Reasoning (CR)) suffer from limitations like slowness, limited context grounding, hallucination and inconsistent outputs. To overcome these challenges, we introduce Evidence to Generate (E2G), a novel single-agent, two-step prompting framework. Instead of unverified reasoning claims, this innovative approach leverages the power of "evidence for decision making" by first focusing exclusively on the thought sequences (the series of intermediate steps) explicitly mentioned in the context which then serve as extracted evidence, guiding the LLM's output generation process with greater precision and efficiency. This simple yet powerful approach unlocks the true potential of chain-of-thought like prompting, paving the way for faster, more reliable, and more contextually aware reasoning in LLMs. \tool achieves remarkable results robustly across a wide range of knowledge-intensive reasoning and generation tasks, surpassing baseline approaches with state-of-the-art LLMs. For example, (i) on LogiQA benchmark using GPT-4 as backbone model, \tool achieves a new state-of-the Accuracy of 53.8% exceeding CoT by 18%, ToT by 11%, CR by 9% (ii) a variant of E2G with PaLM2 outperforms the variable-shot performance of Gemini Ultra by 0.9 F1 points, reaching an F1 score of 83.3 on a subset of DROP. |
Author | Md Rizwan Parvez |
Author_xml | – sequence: 1 fullname: Md Rizwan Parvez |
BookMark | eNqNi0sKwkAQBQdR8HuHBje6CMQZ428nEuNS1L0Mpg2R2B1nOurxjeABXD14VdVVTWLChupoYybBYqp1Ww28v4VhqGdzHUWmoyR-5inSBUEYEiR0VhBGsU7GK1jDMaeswMBmSAKnFwdesIS943spNYIrO9gwCb4FEscVpZiCpRQOKC7Hpy1gXWX3usbvZz1TnfVV62oLj4Pf9tRwG582u6B0_KjQy_nGlaManfVyYqIonJqF-c_6AKATTIg |
ContentType | Paper |
Copyright | 2024. This work is published under http://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License. |
Copyright_xml | – notice: 2024. This work is published under http://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License. |
DBID | 8FE 8FG ABJCF ABUWG AFKRA AZQEC BENPR BGLVJ CCPQU DWQXO HCIFZ L6V M7S PIMPY PQEST PQQKQ PQUKI PRINS PTHSS |
DatabaseName | ProQuest SciTech Collection ProQuest Technology Collection Materials Science & Engineering Database (Proquest) ProQuest Central (Alumni) ProQuest Central ProQuest Central Essentials AUTh Library subscriptions: ProQuest Central Technology Collection ProQuest One Community College ProQuest Central SciTech Premium Collection ProQuest Engineering Collection Engineering Database Publicly Available Content (ProQuest) ProQuest One Academic Eastern Edition (DO NOT USE) ProQuest One Academic ProQuest One Academic UKI Edition ProQuest Central China Engineering Collection |
DatabaseTitle | Publicly Available Content Database Engineering Database Technology Collection ProQuest Central Essentials ProQuest One Academic Eastern Edition ProQuest Central (Alumni Edition) SciTech Premium Collection ProQuest One Community College ProQuest Technology Collection ProQuest SciTech Collection ProQuest Central China ProQuest Central ProQuest Engineering Collection ProQuest One Academic UKI Edition ProQuest Central Korea Materials Science & Engineering Collection ProQuest One Academic Engineering Collection |
DatabaseTitleList | Publicly Available Content Database |
Database_xml | – sequence: 1 dbid: 8FG name: ProQuest Technology Collection url: https://search.proquest.com/technologycollection1 sourceTypes: Aggregation Database |
DeliveryMethod | fulltext_linktorsrc |
Discipline | Physics |
EISSN | 2331-8422 |
Genre | Working Paper/Pre-Print |
GroupedDBID | 8FE 8FG ABJCF ABUWG AFKRA ALMA_UNASSIGNED_HOLDINGS AZQEC BENPR BGLVJ CCPQU DWQXO FRJ HCIFZ L6V M7S M~E PIMPY PQEST PQQKQ PQUKI PRINS PTHSS |
ID | FETCH-proquest_journals_29135504383 |
IEDL.DBID | 8FG |
IngestDate | Wed Sep 25 00:22:14 EDT 2024 |
IsOpenAccess | true |
IsPeerReviewed | false |
IsScholarly | false |
Language | English |
LinkModel | DirectLink |
MergedId | FETCHMERGED-proquest_journals_29135504383 |
OpenAccessLink | https://www.proquest.com/docview/2913550438/abstract/?pq-origsite=%requestingapplication% |
PQID | 2913550438 |
PQPubID | 2050157 |
ParticipantIDs | proquest_journals_2913550438 |
PublicationCentury | 2000 |
PublicationDate | 20240111 |
PublicationDateYYYYMMDD | 2024-01-11 |
PublicationDate_xml | – month: 01 year: 2024 text: 20240111 day: 11 |
PublicationDecade | 2020 |
PublicationPlace | Ithaca |
PublicationPlace_xml | – name: Ithaca |
PublicationTitle | arXiv.org |
PublicationYear | 2024 |
Publisher | Cornell University Library, arXiv.org |
Publisher_xml | – name: Cornell University Library, arXiv.org |
SSID | ssj0002672553 |
Score | 3.5044477 |
SecondaryResourceType | preprint |
Snippet | While chain-of-thought (CoT) prompting has revolutionized how LLMs perform reasoning tasks, its current methods and variations (e.g, Self-consistency, ReACT,... |
SourceID | proquest |
SourceType | Aggregation Database |
SubjectTerms | Context Large language models Reasoning Sequences |
Title | Evidence to Generate (E2G): A Single-agent Two-step Prompting for Context Grounded and Retrieval Augmented Reasoning |
URI | https://www.proquest.com/docview/2913550438/abstract/ |
hasFullText | 1 |
inHoldings | 1 |
isFullTextHit | |
isPrint | |
link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwfV1LSwMxEB5qi-DNJz5qGdCDHsJ2t7vbrhepsm0RWkqt0FvJbrJeZLe2KXrytzsTu3oQekwCISST-YZvXgDXrpStZqK1CJt-Jvys0xSdJGuLjGxZFaksVdbRPhyFgxf_aRbMKjAoc2E4rLLUiVZRqyJljtzxIpegkf1WjkyYBUiNc794F9w_iv2sm2YaO1BzuSYe54z3-r9sixe2yXZu_VO4FkV6-1Aby4VeHkBF54ewa4Mv09URmLK1J5oCf-pAG403sde_vcMuPhO6vGkhOQkKpx-FoIdZ4HhJP5ljlpHMTrRFpj4NMpXEnDbKXOHEdssiUcLu-tUW3-Q5ubIM7DFc9eLp40CUR51vxGo1_7uE1glU8yLXp4CRp9l35iulCJKkjNqBm9LLkO2QBJHvn0F9207n25cvYM8jHGfWwXXrUDXLtb4kHDZJw15xA2oP8Wg8odHwK_4Goq6T_w |
link.rule.ids | 786,790,12792,21416,33408,33779,43635,43840 |
linkProvider | ProQuest |
linkToHtml | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwfV1LT8MwDLZgE4IbTzEYYAkOcIjoI3uUCxpoo8A2TaNIu01pk3JBbVk7wc_HCS0ckHaNpSgPx5_12bEBLmwhXCtUirUtHjMedy3WDeMOi8mXlZ6MI2kC7aNx23_lT7PWrCTc8jKtsrKJxlDLNNIc-bXj2QSNOm51m30w3TVKR1fLFhrrUCcBt2pQv-uPJ9NflsVpd8hndv8ZWoMeg22oT0SmFjuwppJd2DBJl1G-B0XV0hOLFH_qPxcKL_vOw9UN9vCFUOVdMaE_P2HwmTK6kAwnC3rBOlcZyd1EU1zqq0BNIWkuG0UicWq6ZJEKYW_5Zopu6jGRG-Z1H84H_eDeZ9VS56U65fO_zbsHUEvSRB0Ceo7SMTMupSQoEsLrtOyIboR8hrDlcd6A5qqZjlaLz2DTD0bD-fBx_HwMWw5huWYebLsJtWKxVCeExUV4Wh74N_tWkcg |
openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Evidence+to+Generate+%28E2G%29%3A+A+Single-agent+Two-step+Prompting+for+Context+Grounded+and+Retrieval+Augmented+Reasoning&rft.jtitle=arXiv.org&rft.au=Md+Rizwan+Parvez&rft.date=2024-01-11&rft.pub=Cornell+University+Library%2C+arXiv.org&rft.eissn=2331-8422 |