Parameter Estimation in Softmax Decision-Making Models With Linear Objective Functions
We contribute to the development of a systematic means to infer features of human decision-making from behavioral data. Motivated by the common use of softmax selection in models of human decision-making, we study the maximum-likelihood (ML) parameter estimation problem for softmax decision-making m...
Saved in:
| Published in | IEEE transactions on automation science and engineering Vol. 13; no. 1; pp. 54 - 67 |
|---|---|
| Main Authors | , |
| Format | Journal Article |
| Language | English |
| Published |
New York
IEEE
01.01.2016
The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Subjects | |
| Online Access | Get full text |
Cover
Loading…
| Abstract | We contribute to the development of a systematic means to infer features of human decision-making from behavioral data. Motivated by the common use of softmax selection in models of human decision-making, we study the maximum-likelihood (ML) parameter estimation problem for softmax decision-making models with linear objective functions. We present conditions under which the likelihood function is convex. These allow us to provide sufficient conditions for convergence of the resulting ML estimator and to construct its asymptotic distribution. In the case of models with nonlinear objective functions, we show how the estimator can be applied by linearizing about a nominal parameter value. We apply the estimator to fit the stochastic Upper Credible Limit (UCL) model of human decision-making to human subject data. The fits show statistically significant differences in behavior across related, but distinct, tasks. |
|---|---|
| AbstractList | We contribute to the development of a systematic means to infer features of human decision-making from behavioral data. Motivated by the common use of softmax selection in models of human decision-making, we study the maximum-likelihood (ML) parameter estimation problem for softmax decision-making models with linear objective functions. We present conditions under which the likelihood function is convex. These allow us to provide sufficient conditions for convergence of the resulting ML estimator and to construct its asymptotic distribution. In the case of models with nonlinear objective functions, we show how the estimator can be applied by linearizing about a nominal parameter value. We apply the estimator to fit the stochastic Upper Credible Limit (UCL) model of human decision-making to human subject data. The fits show statistically significant differences in behavior across related, but distinct, tasks. |
| Author | Reverdy, Paul Leonard, Naomi Ehrich |
| Author_xml | – sequence: 1 givenname: Paul surname: Reverdy fullname: Reverdy, Paul email: preverdy@seas.upenn.edu organization: Dept. of Electr. & Syst. Eng., Univ. of Pennsylvania, Philadelphia, PA, USA – sequence: 2 givenname: Naomi Ehrich surname: Leonard fullname: Leonard, Naomi Ehrich email: naomi@princeton.edu organization: Dept. of Mech. & Aerosp. Eng., Princeton Univ., Princeton, NJ, USA |
| BookMark | eNp9kE1LxDAQhoMo-PkDxEvAi5euSdOkyVF0_YAVBVc9ltl0qlm7qSZZ0X9v64oHD55mGJ73ZXi2ybrvPBKyz9mIc2aOpyd341HOuBzlhTF5UayRLS6lzkSpxfqwFzKTRspNsh3jnLG80IZtkYdbCLDAhIGOY3ILSK7z1Hl61zVpAR_0DK2L_S27hhfnn-h1V2Mb6aNLz3TiPEKgN7M52uTekZ4vvR0K4i7ZaKCNuPczd8j9-Xh6eplNbi6uTk8mmRVGpQwsRyzzupQCi9qqRtaWoVWqVrWWvEGNUuZ1o2wDGkHLGWgAoaGBmhUcxA45WvW-hu5tiTFVCxctti147Jax4qVWXBkmWI8e_kHn3TL4_ruekoYrkRvVU-WKsqGLMWBTWZe-paQArq04qwbf1eC7GnxXP777JP-TfA29z_D5b-ZglXGI-MuXQihZcvEF5h6Ovg |
| CODEN | ITASC7 |
| CitedBy_id | crossref_primary_10_3389_fnins_2019_01292 crossref_primary_10_1007_s11042_021_11669_3 crossref_primary_10_1177_0037549716667834 crossref_primary_10_26599_BSA_2022_9050007 crossref_primary_10_1007_s42113_021_00112_3 crossref_primary_10_1016_j_image_2021_116410 crossref_primary_10_1088_1742_6596_1550_3_032066 crossref_primary_10_1109_TMC_2024_3373501 crossref_primary_10_1109_TCE_2023_3331770 crossref_primary_10_1155_2018_9867061 crossref_primary_10_1007_s11012_017_0748_4 crossref_primary_10_1002_minf_201700111 crossref_primary_10_1007_s11042_023_14866_4 crossref_primary_10_3390_jimaging6030008 crossref_primary_10_1111_cogs_12695 crossref_primary_10_1177_0361198119851085 crossref_primary_10_1109_LSP_2018_2860238 crossref_primary_10_3390_vaccines10111838 crossref_primary_10_3390_vaccines10050756 crossref_primary_10_1016_j_ijepes_2019_105812 crossref_primary_10_1109_TCCN_2020_2971688 crossref_primary_10_1007_s11042_022_12125_6 crossref_primary_10_1007_s42113_024_00229_1 crossref_primary_10_1007_s11042_020_08654_7 crossref_primary_10_1186_s13640_018_0332_4 crossref_primary_10_1109_TASE_2023_3307758 crossref_primary_10_1007_s00607_020_00883_w crossref_primary_10_1007_s10670_024_00913_5 crossref_primary_10_1109_TVCG_2022_3148745 crossref_primary_10_1088_2399_6528_ac75ad crossref_primary_10_1109_ACCESS_2019_2944411 crossref_primary_10_1186_s12915_023_01774_0 crossref_primary_10_1016_j_cie_2024_110209 |
| Cites_doi | 10.1901/jeab.2005.110-04 10.1146/annurev.neuro.29.051605.112903 10.1109/JPROC.2011.2173815 10.1371/journal.pcbi.1003015 10.1109/ACC.2011.5991446 10.1037/a0038199 10.1090/S0025-5718-1970-0274029-X 10.1093/comjnl/13.3.317 10.1038/nature04766 10.3758/s13415-013-0220-4 10.1109/JPROC.2014.2307024 10.1177/1745691610393980 10.1093/imamat/6.1.76 10.1090/S0025-5718-1970-0258249-6 10.1098/rstb.2007.2098 10.1145/279943.279964 10.1016/j.neuron.2010.04.016 10.1007/BF00048682 10.1109/JPROC.2011.2166437 10.1109/TPAMI.2005.127 10.1007/BF00992698 10.1126/science.1115270 10.1090/S0002-9904-1952-09620-8 10.2307/1427186 10.1093/acprof:oso/9780199600434.003.0001 10.1016/S1573-4412(05)80005-4 |
| ContentType | Journal Article |
| Copyright | Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) Jan 2016 |
| Copyright_xml | – notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) Jan 2016 |
| DBID | 97E RIA RIE AAYXX CITATION 7SC 7SP 7TB 8FD FR3 JQ2 L7M L~C L~D F28 |
| DOI | 10.1109/TASE.2015.2499244 |
| DatabaseName | IEEE Xplore (IEEE) IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef Computer and Information Systems Abstracts Electronics & Communications Abstracts Mechanical & Transportation Engineering Abstracts Technology Research Database Engineering Research Database ProQuest Computer Science Collection Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional ANTE: Abstracts in New Technology & Engineering |
| DatabaseTitle | CrossRef Technology Research Database Computer and Information Systems Abstracts – Academic Mechanical & Transportation Engineering Abstracts Electronics & Communications Abstracts ProQuest Computer Science Collection Computer and Information Systems Abstracts Engineering Research Database Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Professional ANTE: Abstracts in New Technology & Engineering |
| DatabaseTitleList | Technology Research Database Technology Research Database |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://proxy.k.utb.cz/login?url=https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering |
| EISSN | 1558-3783 |
| EndPage | 67 |
| ExternalDocumentID | 3930427351 10_1109_TASE_2015_2499244 7336571 |
| Genre | orig-research Feature |
| GrantInformation_xml | – fundername: Office of Naval Research (ONR) under grantid: N00014-09-1-1074; N00014-14-1-0635 |
| GroupedDBID | -~X 0R~ 29I 4.4 5GY 5VS 6IK 97E AAJGR AARMG AASAJ AAWTH ABAZT ABQJQ ABVLG ACGFO ACGFS ACIWK AENEX AETIX AGQYO AGSQL AHBIQ AIBXA AKJIK AKQYR ALMA_UNASSIGNED_HOLDINGS ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ CS3 DU5 EBS EJD F5P HZ~ H~9 IFIPE IPLJI JAVBF LAI M43 O9- OCL PQQKQ RIA RIE RNS AAYXX CITATION RIG 7SC 7SP 7TB 8FD FR3 JQ2 L7M L~C L~D F28 |
| ID | FETCH-LOGICAL-c396t-ac1ee72d753e4dc6f5dc0ec66d6d851fe8e552df6cfa8ea85ba8aa38afad041a3 |
| IEDL.DBID | RIE |
| ISSN | 1545-5955 |
| IngestDate | Sun Aug 24 04:05:53 EDT 2025 Sun Jun 29 15:18:30 EDT 2025 Tue Jul 01 02:56:28 EDT 2025 Thu Apr 24 22:54:56 EDT 2025 Tue Aug 26 16:43:04 EDT 2025 |
| IsPeerReviewed | false |
| IsScholarly | true |
| Issue | 1 |
| Language | English |
| License | https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c396t-ac1ee72d753e4dc6f5dc0ec66d6d851fe8e552df6cfa8ea85ba8aa38afad041a3 |
| Notes | SourceType-Scholarly Journals-1 ObjectType-Feature-1 content type line 14 ObjectType-Article-1 ObjectType-Feature-2 content type line 23 |
| PQID | 1759163296 |
| PQPubID | 27623 |
| PageCount | 14 |
| ParticipantIDs | proquest_miscellaneous_1786169030 proquest_journals_1759163296 ieee_primary_7336571 crossref_citationtrail_10_1109_TASE_2015_2499244 crossref_primary_10_1109_TASE_2015_2499244 |
| ProviderPackageCode | CITATION AAYXX |
| PublicationCentury | 2000 |
| PublicationDate | 2016-01-01 |
| PublicationDateYYYYMMDD | 2016-01-01 |
| PublicationDate_xml | – month: 01 year: 2016 text: 2016-01-01 day: 01 |
| PublicationDecade | 2010 |
| PublicationPlace | New York |
| PublicationPlace_xml | – name: New York |
| PublicationTitle | IEEE transactions on automation science and engineering |
| PublicationTitleAbbrev | TASE |
| PublicationYear | 2016 |
| Publisher | IEEE The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Publisher_xml | – name: IEEE – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| References | ref35 ref12 ref37 ref15 wilson (ref38) 2013 ref30 ref32 ref10 goldberger (ref11) 1991 ref2 ref1 (ref34) 2015 ref16 ref19 ref18 sutton (ref33) 1998 kay (ref14) 1993 newey (ref22) 1994; 4 russell (ref28) 1998 gimpel (ref9) 2010 ref24 samejima (ref29) 2005; 310 ref26 ref20 kaufmann (ref13) 2012 ref21 ref27 ng (ref23) 2000 srivastava (ref31) 2015 ref8 ref7 ref4 ref3 ref6 ref5 reverdy (ref25) 2014 welch (ref36) 1947; 34 mcfadden (ref17) 1974 |
| References_xml | – ident: ref16 doi: 10.1901/jeab.2005.110-04 – ident: ref19 doi: 10.1146/annurev.neuro.29.051605.112903 – start-page: 592 year: 2012 ident: ref13 article-title: On Bayesian upper confidence bounds for bandit problems publication-title: Proc Int Conf Artif Intell Statist – ident: ref32 doi: 10.1109/JPROC.2011.2173815 – ident: ref20 doi: 10.1371/journal.pcbi.1003015 – year: 1991 ident: ref11 publication-title: Econometrics Course[M] – ident: ref24 doi: 10.1109/ACC.2011.5991446 – ident: ref37 doi: 10.1037/a0038199 – ident: ref30 doi: 10.1090/S0025-5718-1970-0274029-X – ident: ref8 doi: 10.1093/comjnl/13.3.317 – ident: ref7 doi: 10.1038/nature04766 – start-page: 663 year: 2000 ident: ref23 article-title: Algorithms for inverse reinforcement learning publication-title: Proc Int Conf Mach Learn – ident: ref5 doi: 10.3758/s13415-013-0220-4 – ident: ref26 doi: 10.1109/JPROC.2014.2307024 – year: 2014 ident: ref25 publication-title: Human-inspired algorithms for search a framework for human-machine multi-armed bandit problems – year: 2015 ident: ref31 article-title: Correlated multiarmed bandit problem: Bayesian algorithms and regret analysis – ident: ref3 doi: 10.1177/1745691610393980 – year: 1993 ident: ref14 publication-title: Fundamentals of Statistical Signal Processing Volume I Estimation Theory – ident: ref2 doi: 10.1093/imamat/6.1.76 – start-page: 41s year: 2013 ident: ref38 article-title: Is model fitting necessary for model-based fMRI? publication-title: Proc Multi-Disciplinary Conf Reinforcement Learn Decision Making – ident: ref12 doi: 10.1090/S0025-5718-1970-0258249-6 – ident: ref4 doi: 10.1098/rstb.2007.2098 – start-page: 101 year: 1998 ident: ref28 article-title: Learning agents for uncertain environments publication-title: Proc 11th Annu Conf Computational Learning Theory doi: 10.1145/279943.279964 – ident: ref10 doi: 10.1016/j.neuron.2010.04.016 – ident: ref1 doi: 10.1007/BF00048682 – year: 1998 ident: ref33 publication-title: Introduction to Reinforcement Learning – start-page: 105 year: 1974 ident: ref17 publication-title: Frontiers in Econometrics – ident: ref21 doi: 10.1109/JPROC.2011.2166437 – year: 2015 ident: ref34 publication-title: Fminunc – volume: 34 start-page: 28 year: 1947 ident: ref36 article-title: The generalization of ?Student?s? problem when several different population variances are involved publication-title: Biometrika – ident: ref15 doi: 10.1109/TPAMI.2005.127 – ident: ref35 doi: 10.1007/BF00992698 – volume: 310 start-page: 1337 year: 2005 ident: ref29 article-title: Representation of action-specific reward values in the striatum publication-title: Science doi: 10.1126/science.1115270 – ident: ref27 doi: 10.1090/S0002-9904-1952-09620-8 – ident: ref18 doi: 10.2307/1427186 – ident: ref6 doi: 10.1093/acprof:oso/9780199600434.003.0001 – volume: 4 start-page: 2111 year: 1994 ident: ref22 publication-title: Handbook of Econometrics doi: 10.1016/S1573-4412(05)80005-4 – year: 2010 ident: ref9 publication-title: ?Softmax-margin training for structured log-linear models ? |
| SSID | ssj0024890 |
| Score | 2.321352 |
| Snippet | We contribute to the development of a systematic means to infer features of human decision-making from behavioral data. Motivated by the common use of softmax... |
| SourceID | proquest crossref ieee |
| SourceType | Aggregation Database Enrichment Source Index Database Publisher |
| StartPage | 54 |
| SubjectTerms | Asymptotic methods Asymptotic properties Automation Biological system modeling Convergence Data models Decision making estimation Estimators Human Human behavior Linear programming Mathematical models Maximum likelihood method Parameter estimation Stochastic models Stochastic processes |
| Title | Parameter Estimation in Softmax Decision-Making Models With Linear Objective Functions |
| URI | https://ieeexplore.ieee.org/document/7336571 https://www.proquest.com/docview/1759163296 https://www.proquest.com/docview/1786169030 |
| Volume | 13 |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV1LT9wwEB4Bp_ZAaSnqlodcqSeEl2T9WOeIYFeo0rZIQMstcuyJSkuzVclKiF_P2PEuFFDVW6Q4D83Dnm-eAB9ROC8FWj7I0XBZZ5JbpZBbkmXMtTbKBz_k5LM-PpefLtTFEuwtamEQMSafYT9cxli-n7pZcJXth9Z9KhSMLxNw62q17vvqmehPCRYBV4VSKYKZZ8X-2cHpKCRxqT5hDcIb8q8zKA5VebITx-Nl_Aom8x_rskp-9mdt1Xe3j3o2_u-fr8FqsjPZQScYr2EJmzfw8kH3wXX4emJDZhYRlo1I0bsaRnbZsFPamn_ZG3aU5u_wSRxZxcLctKtr9u2y_c4Iw5KOsC_Vj27HZGM6IKMMv4Xz8ejs8JinMQvciUK33LoccTjwBFxQeqdr5V2GTmuvPdljNRpUauBr7Wpr0BpVWWOtMLa2PpO5FRuw0kwbfAcsr6QjjImZJZSnvCjy2hsjjBoKOdRY9yCbE750qQd5GIVxVUYskhVl4FUZeFUmXvVgd_HI764Bx78WrwfaLxYmsvdga87dMqnodUl2E5nGYlDoHnxY3CblChET2-B0FtYYHeKIInv__Js34QV9P_lktmCl_TPDbbJS2moniucdjgrkMQ |
| linkProvider | IEEE |
| linkToHtml | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjR3bbtMw9GgaD2wP3Ma0wgBP4gnNXVJf6jxO0KqwdSCtG3uLHPtEDEaKWCohvp5jxy2XIcRbpDiRde73A_AchfNSoOWDHA2XdSa5VQq5JVrGXGujfIhDTk_05Ey-uVAXa7C_6oVBxFh8hv3wGHP5fu4WIVR2EEb3qdAwfov0vsq7bq2fk_VMjKgEm4CrQqmUw8yz4mB2eDoKZVyqT94GeRzyNy0U16rckMVRwYzvwnR5ta6u5FN_0VZ99_2PqY3_e_d7cCdZmuywI437sIbNA9j8Zf7gFpy_s6E2i0DLRsTqXRcju2zYKQnnz_Ybe5U28PBpXFrFwua0q2v2_rL9wMiLJS5hb6uPncxkY1KRkYofwtl4NHs54WnRAnei0C23LkccDjy5Lii907XyLkOntdeeLLIaDSo18LV2tTVojaqssVYYW1ufydyKbVhv5g3uAMsr6cjLxMySn6e8KPLaGyOMGgo51Fj3IFsCvnRpCnlYhnFVRm8kK8qAqzLgqky46sGL1SdfuhEc_zq8FWC_OpjA3oPdJXbLxKTXJVlOZByLQaF7sLd6TewVcia2wfkinDE6ZBJF9ujvf34Gtyez6XF5_Prk6DFs0F1ShGYX1tuvC3xCNktbPY2k-gObi-d6 |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Parameter+Estimation+in+Softmax+Decision-Making+Models+With+Linear+Objective+Functions&rft.jtitle=IEEE+transactions+on+automation+science+and+engineering&rft.au=Reverdy%2C+Paul&rft.au=Leonard%2C+Naomi+Ehrich&rft.date=2016-01-01&rft.issn=1545-5955&rft.eissn=1558-3783&rft.volume=13&rft.issue=1&rft.spage=54&rft.epage=67&rft_id=info:doi/10.1109%2FTASE.2015.2499244&rft.externalDBID=n%2Fa&rft.externalDocID=10_1109_TASE_2015_2499244 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1545-5955&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1545-5955&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1545-5955&client=summon |