Lower bounds for non-convex stochastic optimization
We lower bound the complexity of finding ϵ -stationary points (with gradient norm at most ϵ ) using stochastic first-order methods. In a well-studied model where algorithms access smooth, potentially non-convex functions through queries to an unbiased stochastic gradient oracle with bounded variance...
Saved in:
| Published in | Mathematical programming Vol. 199; no. 1-2; pp. 165 - 214 |
|---|---|
| Main Authors | , , , , , |
| Format | Journal Article |
| Language | English |
| Published |
Berlin/Heidelberg
Springer Berlin Heidelberg
01.05.2023
Springer Springer Nature B.V |
| Subjects | |
| Online Access | Get full text |
Cover
Loading…
| Abstract | We lower bound the complexity of finding
ϵ
-stationary points (with gradient norm at most
ϵ
) using stochastic first-order methods. In a well-studied model where algorithms access smooth, potentially non-convex functions through queries to an unbiased stochastic gradient oracle with bounded variance, we prove that (in the worst case) any algorithm requires at least
ϵ
-
4
queries to find an
ϵ
-stationary point. The lower bound is tight, and establishes that stochastic gradient descent is minimax optimal in this model. In a more restrictive model where the noisy gradient estimates satisfy a mean-squared smoothness property, we prove a lower bound of
ϵ
-
3
queries, establishing the optimality of recently proposed variance reduction techniques. |
|---|---|
| AbstractList | We lower bound the complexity of finding ϵ-stationary points (with gradient norm at most ϵ) using stochastic first-order methods. In a well-studied model where algorithms access smooth, potentially non-convex functions through queries to an unbiased stochastic gradient oracle with bounded variance, we prove that (in the worst case) any algorithm requires at least ϵ-4 queries to find an ϵ-stationary point. The lower bound is tight, and establishes that stochastic gradient descent is minimax optimal in this model. In a more restrictive model where the noisy gradient estimates satisfy a mean-squared smoothness property, we prove a lower bound of ϵ-3 queries, establishing the optimality of recently proposed variance reduction techniques. We lower bound the complexity of finding ϵ -stationary points (with gradient norm at most ϵ ) using stochastic first-order methods. In a well-studied model where algorithms access smooth, potentially non-convex functions through queries to an unbiased stochastic gradient oracle with bounded variance, we prove that (in the worst case) any algorithm requires at least ϵ - 4 queries to find an ϵ -stationary point. The lower bound is tight, and establishes that stochastic gradient descent is minimax optimal in this model. In a more restrictive model where the noisy gradient estimates satisfy a mean-squared smoothness property, we prove a lower bound of ϵ - 3 queries, establishing the optimality of recently proposed variance reduction techniques. We lower bound the complexity of finding [Formula omitted]-stationary points (with gradient norm at most [Formula omitted]) using stochastic first-order methods. In a well-studied model where algorithms access smooth, potentially non-convex functions through queries to an unbiased stochastic gradient oracle with bounded variance, we prove that (in the worst case) any algorithm requires at least [Formula omitted] queries to find an [Formula omitted]-stationary point. The lower bound is tight, and establishes that stochastic gradient descent is minimax optimal in this model. In a more restrictive model where the noisy gradient estimates satisfy a mean-squared smoothness property, we prove a lower bound of [Formula omitted] queries, establishing the optimality of recently proposed variance reduction techniques. |
| Audience | Academic |
| Author | Carmon, Yair Woodworth, Blake Foster, Dylan J. Duchi, John C. Arjevani, Yossi Srebro, Nathan |
| Author_xml | – sequence: 1 givenname: Yossi surname: Arjevani fullname: Arjevani, Yossi organization: The Hebrew University – sequence: 2 givenname: Yair orcidid: 0000-0001-5731-8640 surname: Carmon fullname: Carmon, Yair email: ycarmon@tauex.tau.ac.il organization: Tel Aviv University – sequence: 3 givenname: John C. surname: Duchi fullname: Duchi, John C. organization: Stanford University – sequence: 4 givenname: Dylan J. surname: Foster fullname: Foster, Dylan J. organization: Microsoft Research New England – sequence: 5 givenname: Nathan surname: Srebro fullname: Srebro, Nathan organization: TTIC – sequence: 6 givenname: Blake surname: Woodworth fullname: Woodworth, Blake organization: Inria |
| BookMark | eNp9kE1LAzEQhoNUsK3-AU8LnrfObJJN9liKX1DwoueQpklNaTc12fr1641dQfBQAhkY5pmXeUZk0IbWEnKJMEEAcZ0QEEQJVVUCyvyLEzJERuuS1awekCFAxUteI5yRUUprAEAq5ZDQeXi3sViEfbtMhQuxyJtLE9o3-1GkLpgXnTpvirDr_NZ_6c6H9pycOr1J9uK3jsnz7c3T7L6cP949zKbz0lAuu5ILI5xzNcemRgPcSpZL5QSyiqJGa3BpeCM1X7imrhxdMEcl4xoRJUdNx-Sq37uL4XVvU6fWYR_bHKkqCYxCIznLU5N-aqU3VvnWhS5qk9_Sbn0-xDqf-1PBas4QOM9A1QMmhpSidWoX_VbHT4Wgfmyq3qbKNtXBphIZkv8g47uDjZzmN8dR2qMp57QrG__OOEJ9A-SEiWI |
| CitedBy_id | crossref_primary_10_1109_TIFS_2024_3516548 crossref_primary_10_1109_TSP_2024_3511452 crossref_primary_10_1186_s40537_023_00828_y crossref_primary_10_1007_s10589_023_00521_z crossref_primary_10_1016_j_jco_2024_101889 crossref_primary_10_3390_su16051947 crossref_primary_10_1109_TPAMI_2024_3423382 crossref_primary_10_1109_TSP_2024_3374106 crossref_primary_10_1287_moor_2022_0193 crossref_primary_10_1007_s10957_024_02547_7 crossref_primary_10_1080_02331934_2024_2367635 crossref_primary_10_1109_TPAMI_2024_3382294 crossref_primary_10_1109_TSP_2022_3143372 crossref_primary_10_1007_s10994_024_06573_4 crossref_primary_10_2478_ijanmc_2024_0040 crossref_primary_10_1109_TWC_2024_3497998 crossref_primary_10_3390_math12111664 crossref_primary_10_1007_s10957_023_02193_5 crossref_primary_10_1109_TPAMI_2021_3112139 |
| Cites_doi | 10.1109/TIT.2011.2182178 10.1109/TIT.2017.2701343 10.1109/TIT.2011.2154375 10.1007/s10208-017-9365-9 10.1016/j.jco.2011.06.001 10.1137/16M1080173 10.1007/BF02592948 10.1137/0803004 10.1007/s10208-019-09429-9 10.1006/jcom.1994.1025 10.1137/120880811 10.1007/s10107-019-01431-x 10.1137/090774100 10.1007/s10107-006-0706-8 10.1214/aos/1193342380 10.1007/978-1-4419-8853-9 10.1007/978-1-4612-1880-7_29 10.1109/ALLERTON.2016.7852377 10.1109/SFCS.1977.24 |
| ContentType | Journal Article |
| Copyright | Springer-Verlag GmbH Germany, part of Springer Nature and Mathematical Optimization Society 2022 COPYRIGHT 2023 Springer Springer-Verlag GmbH Germany, part of Springer Nature and Mathematical Optimization Society 2022. |
| Copyright_xml | – notice: Springer-Verlag GmbH Germany, part of Springer Nature and Mathematical Optimization Society 2022 – notice: COPYRIGHT 2023 Springer – notice: Springer-Verlag GmbH Germany, part of Springer Nature and Mathematical Optimization Society 2022. |
| DBID | AAYXX CITATION 7SC 8FD JQ2 L7M L~C L~D |
| DOI | 10.1007/s10107-022-01822-7 |
| DatabaseName | CrossRef Computer and Information Systems Abstracts Technology Research Database ProQuest Computer Science Collection Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional |
| DatabaseTitle | CrossRef Computer and Information Systems Abstracts Technology Research Database Computer and Information Systems Abstracts – Academic Advanced Technologies Database with Aerospace ProQuest Computer Science Collection Computer and Information Systems Abstracts Professional |
| DatabaseTitleList | Computer and Information Systems Abstracts |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering Mathematics |
| EISSN | 1436-4646 |
| EndPage | 214 |
| ExternalDocumentID | A746541055 10_1007_s10107_022_01822_7 |
| GrantInformation_xml | – fundername: Office of Naval Research Global grantid: ONR-YIP N00014-19-1-2288 funderid: http://dx.doi.org/10.13039/100007297 – fundername: Alfred P. Sloan Foundation funderid: http://dx.doi.org/10.13039/100000879 – fundername: Google funderid: http://dx.doi.org/10.13039/100006785 – fundername: Division of Computing and Communication Foundations grantid: 1553086 funderid: http://dx.doi.org/10.13039/100000143 – fundername: Division of Computing and Communication Foundations grantid: 1740751 funderid: http://dx.doi.org/10.13039/100000143 |
| GroupedDBID | --K --Z -52 -5D -5G -BR -EM -Y2 -~C -~X .4S .86 .DC .VR 06D 0R~ 0VY 199 1B1 1N0 1OL 1SB 203 28- 29M 2J2 2JN 2JY 2KG 2KM 2LR 2P1 2VQ 2~H 30V 3V. 4.4 406 408 409 40D 40E 5GY 5QI 5VS 67Z 6NX 6TJ 78A 7WY 88I 8AO 8FE 8FG 8FL 8TC 8UJ 8VB 95- 95. 95~ 96X AAAVM AABHQ AACDK AAHNG AAIAL AAJBT AAJKR AANZL AARHV AARTL AASML AATNV AATVU AAUYE AAWCG AAYIU AAYQN AAYTO AAYZH ABAKF ABBBX ABBXA ABDBF ABDZT ABECU ABFTV ABHLI ABHQN ABJCF ABJNI ABJOX ABKCH ABKTR ABMNI ABMQK ABNWP ABQBU ABQSL ABSXP ABTEG ABTHY ABTKH ABTMW ABULA ABUWG ABWNU ABXPI ACAOD ACBXY ACDTI ACGFS ACGOD ACHSB ACHXU ACIWK ACKNC ACMDZ ACMLO ACNCT ACOKC ACOMO ACPIV ACUHS ACZOJ ADHHG ADHIR ADIMF ADINQ ADKNI ADKPE ADRFC ADTPH ADURQ ADYFF ADZKW AEBTG AEFIE AEFQL AEGAL AEGNC AEJHL AEJRE AEKMD AEMOZ AEMSY AENEX AEOHA AEPYU AESKC AETLH AEVLU AEXYK AFBBN AFEXP AFFNX AFGCZ AFKRA AFLOW AFQWF AFWTZ AFZKB AGAYW AGDGC AGGDS AGJBK AGMZJ AGQEE AGQMX AGRTI AGWIL AGWZB AGYKE AHAVH AHBYD AHKAY AHQJS AHSBF AHYZX AIAKS AIGIU AIIXL AILAN AITGF AJBLW AJRNO AJZVZ AKVCP ALMA_UNASSIGNED_HOLDINGS ALWAN AMKLP AMXSW AMYLF AMYQR AOCGG ARAPS ARCSS ARMRJ ASPBG AVWKF AXYYD AYJHY AZFZN AZQEC B-. B0M BA0 BAPOH BBWZM BDATZ BENPR BEZIV BGLVJ BGNMA BPHCQ BSONS CAG CCPQU COF CS3 CSCUP DDRTE DL5 DNIVK DPUIP DU5 DWQXO EAD EAP EBA EBLON EBR EBS EBU ECS EDO EIOEI EJD EMI EMK EPL ESBYG EST ESX FEDTE FERAY FFXSO FIGPU FINBP FNLPD FRNLG FRRFC FSGXE FWDCC GGCAI GGRSB GJIRD GNUQQ GNWQR GQ6 GQ7 GQ8 GROUPED_ABI_INFORM_COMPLETE GXS H13 HCIFZ HF~ HG5 HG6 HMJXF HQYDN HRMNR HVGLF HZ~ H~9 I-F I09 IAO IHE IJ- IKXTQ ITM IWAJR IXC IZIGR IZQ I~X I~Z J-C J0Z JBSCW JCJTX JZLTJ K1G K60 K6V K6~ K7- KDC KOV KOW L6V LAS LLZTM M0C M0N M2P M4Y M7S MA- N2Q N9A NB0 NDZJH NPVJJ NQ- NQJWS NU0 O9- O93 O9G O9I O9J OAM P19 P2P P62 P9R PF0 PQBIZ PQBZA PQQKQ PROAC PT4 PT5 PTHSS Q2X QOK QOS QWB R4E R89 R9I RHV RIG RNI RNS ROL RPX RPZ RSV RZK S16 S1Z S26 S27 S28 S3B SAP SCLPG SDD SDH SDM SHX SISQX SJYHP SMT SNE SNPRN SNX SOHCF SOJ SPISZ SRMVM SSLCW STPWE SZN T13 T16 TH9 TN5 TSG TSK TSV TUC TUS U2A UG4 UOJIU UTJUX UZXMN VC2 VFIZW W23 W48 WH7 WK8 XPP YLTOR Z45 Z5O Z7R Z7S Z7X Z7Y Z7Z Z81 Z83 Z86 Z88 Z8M Z8N Z8R Z8T Z8W Z92 ZL0 ZMTXR ZWQNP ~02 ~8M ~EX AAPKM AAYXX ABBRH ABDBE ABFSG ACSTC ADHKG ADXHL AEZWR AFDZB AFHIU AFOHR AGQPQ AHPBZ AHWEU AIXLP AMVHM ATHPR AYFIA CITATION PHGZM PHGZT AEIIB 7SC 8FD ABRTQ JQ2 L7M L~C L~D |
| ID | FETCH-LOGICAL-c358t-57c7fff651961c05e841c02f714231a1ec1dc598a5bf962f3b4f3845a111851a3 |
| IEDL.DBID | U2A |
| ISSN | 0025-5610 |
| IngestDate | Fri Jul 25 19:49:27 EDT 2025 Tue Jun 10 21:06:55 EDT 2025 Thu Apr 24 23:05:51 EDT 2025 Tue Jul 01 02:15:14 EDT 2025 Fri Feb 21 02:43:30 EST 2025 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 1-2 |
| Keywords | 90C06 68Q25 90C15 90C26 90C30 90C60 |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c358t-57c7fff651961c05e841c02f714231a1ec1dc598a5bf962f3b4f3845a111851a3 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14 |
| ORCID | 0000-0001-5731-8640 |
| PQID | 2804309854 |
| PQPubID | 25307 |
| PageCount | 50 |
| ParticipantIDs | proquest_journals_2804309854 gale_infotracacademiconefile_A746541055 crossref_primary_10_1007_s10107_022_01822_7 crossref_citationtrail_10_1007_s10107_022_01822_7 springer_journals_10_1007_s10107_022_01822_7 |
| ProviderPackageCode | CITATION AAYXX |
| PublicationCentury | 2000 |
| PublicationDate | 2023-05-01 |
| PublicationDateYYYYMMDD | 2023-05-01 |
| PublicationDate_xml | – month: 05 year: 2023 text: 2023-05-01 day: 01 |
| PublicationDecade | 2020 |
| PublicationPlace | Berlin/Heidelberg |
| PublicationPlace_xml | – name: Berlin/Heidelberg – name: Heidelberg |
| PublicationSubtitle | A Publication of the Mathematical Optimization Society |
| PublicationTitle | Mathematical programming |
| PublicationTitleAbbrev | Math. Program |
| PublicationYear | 2023 |
| Publisher | Springer Berlin Heidelberg Springer Springer Nature B.V |
| Publisher_xml | – name: Springer Berlin Heidelberg – name: Springer – name: Springer Nature B.V |
| References | Agarwal, Bartlett, Ravikumar, Wainwright (CR1) 2012; 5 CR37 CR36 CR32 Cartis, Gould, Toint (CR18) 2012; 28 Ma, Wang, Chi, Chen (CR33) 2019 Nesterov (CR39) 1983; 27 CR2 Cartis, Gould, Toint (CR19) 2012; 88 Murty, Kabadi (CR34) 1987; 39 CR4 CR3 CR6 CR5 CR8 CR7 CR9 CR49 CR47 CR46 CR43 CR42 CR40 Carmon, Duchi, Hinder, Sidford (CR15) 2019; 184 LeCam (CR31) 1973; 1 Cartis, Gould, Toint (CR17) 2010; 20 CR14 CR13 CR57 Vavasis (CR48) 1993; 3 CR56 CR55 CR10 CR54 CR53 CR52 CR51 CR50 Sun, Qu, Wright (CR45) 2018; 18 Braun, Guzmán, Pokutta (CR12) 2017; 63 Carmon, Duchi, Hinder, Sidford (CR16) 2021; 185 Nesterov, Polyak (CR38) 2006; 108 CR29 CR28 CR27 CR26 CR25 Bottou, Curtis, Nocedal (CR11) 2018; 60 CR24 CR23 CR22 CR21 CR20 Raginsky, Rakhlin (CR41) 2011; 57 Ghadimi, Lan (CR30) 2013; 23 Shalev-Shwartz, Zhang (CR44) 2013; 14 Nemirovski (CR35) 1994; 10 1822_CR14 1822_CR13 1822_CR57 1822_CR56 1822_CR55 1822_CR10 1822_CR54 1822_CR53 A Agarwal (1822_CR1) 2012; 5 S Shalev-Shwartz (1822_CR44) 2013; 14 1822_CR52 C Cartis (1822_CR18) 2012; 28 1822_CR51 1822_CR50 1822_CR8 1822_CR7 1822_CR9 KG Murty (1822_CR34) 1987; 39 M Raginsky (1822_CR41) 2011; 57 1822_CR49 1822_CR47 1822_CR46 1822_CR4 1822_CR3 1822_CR43 1822_CR6 C Cartis (1822_CR19) 2012; 88 S Ghadimi (1822_CR30) 2013; 23 YE Nesterov (1822_CR39) 1983; 27 1822_CR42 1822_CR5 1822_CR40 1822_CR2 G Braun (1822_CR12) 2017; 63 1822_CR37 L Bottou (1822_CR11) 2018; 60 1822_CR36 1822_CR32 L LeCam (1822_CR31) 1973; 1 Y Nesterov (1822_CR38) 2006; 108 C Ma (1822_CR33) 2019 SA Vavasis (1822_CR48) 1993; 3 J Sun (1822_CR45) 2018; 18 Y Carmon (1822_CR15) 2019; 184 Y Carmon (1822_CR16) 2021; 185 C Cartis (1822_CR17) 2010; 20 1822_CR29 1822_CR28 1822_CR27 1822_CR26 1822_CR25 1822_CR24 1822_CR23 1822_CR22 1822_CR21 1822_CR20 A Nemirovski (1822_CR35) 1994; 10 |
| References_xml | – ident: CR22 – volume: 5 start-page: 3235 issue: 58 year: 2012 end-page: 3249 ident: CR1 article-title: Information-theoretic lower bounds on the oracle complexity of stochastic convex optimization publication-title: IEEE Trans. Inf. Theory doi: 10.1109/TIT.2011.2182178 – ident: CR49 – ident: CR4 – ident: CR51 – volume: 14 start-page: 567 year: 2013 end-page: 599 ident: CR44 article-title: Stochastic dual coordinate ascent methods for regularized loss minimization publication-title: J. Mach. Learn. Res. – ident: CR29 – ident: CR54 – ident: CR8 – ident: CR25 – ident: CR42 – ident: CR21 – ident: CR46 – volume: 63 start-page: 4709 issue: 7 year: 2017 end-page: 4724 ident: CR12 article-title: Lower bounds on the oracle complexity of nonsmooth convex optimization via information theory publication-title: IEEE Trans. Inf. Theory doi: 10.1109/TIT.2017.2701343 – ident: CR50 – volume: 57 start-page: 7036 issue: 10 year: 2011 end-page: 7056 ident: CR41 article-title: Information-based complexity, feedback and dynamics in convex programming publication-title: IEEE Trans. Inf. Theory doi: 10.1109/TIT.2011.2154375 – ident: CR9 – ident: CR57 – volume: 18 start-page: 1131 issue: 5 year: 2018 end-page: 1198 ident: CR45 article-title: A geometric analysis of phase retrieval publication-title: Found. Comput. Math. doi: 10.1007/s10208-017-9365-9 – ident: CR32 – ident: CR36 – ident: CR5 – volume: 28 start-page: 93 issue: 1 year: 2012 end-page: 108 ident: CR18 article-title: Complexity bounds for second-order optimality in unconstrained optimization publication-title: J. Complex. doi: 10.1016/j.jco.2011.06.001 – volume: 60 start-page: 223 issue: 2 year: 2018 end-page: 311 ident: CR11 article-title: Optimization methods for large-scale learning publication-title: SIAM Rev. doi: 10.1137/16M1080173 – ident: CR26 – volume: 39 start-page: 117 issue: 2 year: 1987 end-page: 129 ident: CR34 article-title: Some np-complete problems in quadratic and nonlinear programming publication-title: Math. progr. doi: 10.1007/BF02592948 – volume: 3 start-page: 60 issue: 1 year: 1993 end-page: 80 ident: CR48 article-title: Black-box complexity of local minimization publication-title: SIAM J. Opt. doi: 10.1137/0803004 – ident: CR43 – year: 2019 ident: CR33 article-title: Implicit regularization in nonconvex statistical estimation: gradient descent converges linearly for phase retrieval, matrix completion and blind deconvolution publication-title: Found. Comput. Math. doi: 10.1007/s10208-019-09429-9 – ident: CR47 – ident: CR14 – ident: CR2 – volume: 10 start-page: 451 issue: 4 year: 1994 end-page: 463 ident: CR35 article-title: On parallel complexity of nonsmooth convex optimization publication-title: J. Complex. doi: 10.1006/jcom.1994.1025 – ident: CR37 – ident: CR53 – volume: 88 start-page: 1 year: 2012 end-page: 10 ident: CR19 article-title: How much patience to you have?: a worst-case perspective on smooth noncovex optimization publication-title: Optima – ident: CR10 – volume: 23 start-page: 2341 issue: 4 year: 2013 end-page: 2368 ident: CR30 article-title: Stochastic first-and zeroth-order methods for nonconvex stochastic programming publication-title: SIAM J. Opt. doi: 10.1137/120880811 – volume: 185 start-page: 315 issue: 1 year: 2021 end-page: 355 ident: CR16 article-title: Lower bounds for finding stationary points II: First-order methods publication-title: Math. Progr. doi: 10.1007/s10107-019-01431-x – ident: CR6 – ident: CR56 – ident: CR40 – ident: CR27 – ident: CR23 – volume: 27 start-page: 372 issue: 2 year: 1983 end-page: 376 ident: CR39 article-title: A method for solving the convex programming problem with convergence rate publication-title: Sov. Math. Dokl. – volume: 20 start-page: 2833 issue: 6 year: 2010 end-page: 2852 ident: CR17 article-title: On the complexity of steepest descent, newton’s and regularized newton’s methods for nonconvex unconstrained optimization problems publication-title: Siam J. Opt. doi: 10.1137/090774100 – ident: CR3 – volume: 108 start-page: 177 issue: 1 year: 2006 end-page: 205 ident: CR38 article-title: Cubic regularization of newton method and its global performance publication-title: Math. Progr. doi: 10.1007/s10107-006-0706-8 – ident: CR52 – ident: CR13 – ident: CR55 – ident: CR7 – ident: CR28 – ident: CR24 – volume: 1 start-page: 38 issue: 1 year: 1973 end-page: 53 ident: CR31 article-title: Convergence of estimates under dimensionality restrictions publication-title: Ann. Stat. doi: 10.1214/aos/1193342380 – ident: CR20 – volume: 184 start-page: 71 issue: 1 year: 2019 end-page: 120 ident: CR15 article-title: Lower bounds for finding stationary points I publication-title: Math. Progr. – ident: 1822_CR14 – ident: 1822_CR5 – ident: 1822_CR10 – ident: 1822_CR9 – ident: 1822_CR20 – ident: 1822_CR56 – volume: 5 start-page: 3235 issue: 58 year: 2012 ident: 1822_CR1 publication-title: IEEE Trans. Inf. Theory doi: 10.1109/TIT.2011.2182178 – ident: 1822_CR52 – ident: 1822_CR28 – ident: 1822_CR24 – ident: 1822_CR43 – ident: 1822_CR2 – ident: 1822_CR47 – ident: 1822_CR8 – volume: 39 start-page: 117 issue: 2 year: 1987 ident: 1822_CR34 publication-title: Math. progr. doi: 10.1007/BF02592948 – volume: 27 start-page: 372 issue: 2 year: 1983 ident: 1822_CR39 publication-title: Sov. Math. Dokl. – ident: 1822_CR40 – ident: 1822_CR13 – volume: 28 start-page: 93 issue: 1 year: 2012 ident: 1822_CR18 publication-title: J. Complex. doi: 10.1016/j.jco.2011.06.001 – ident: 1822_CR55 – ident: 1822_CR37 doi: 10.1007/978-1-4419-8853-9 – volume: 57 start-page: 7036 issue: 10 year: 2011 ident: 1822_CR41 publication-title: IEEE Trans. Inf. Theory doi: 10.1109/TIT.2011.2154375 – volume: 18 start-page: 1131 issue: 5 year: 2018 ident: 1822_CR45 publication-title: Found. Comput. Math. doi: 10.1007/s10208-017-9365-9 – ident: 1822_CR51 – volume: 108 start-page: 177 issue: 1 year: 2006 ident: 1822_CR38 publication-title: Math. Progr. doi: 10.1007/s10107-006-0706-8 – ident: 1822_CR27 – volume: 63 start-page: 4709 issue: 7 year: 2017 ident: 1822_CR12 publication-title: IEEE Trans. Inf. Theory doi: 10.1109/TIT.2017.2701343 – ident: 1822_CR23 – volume: 23 start-page: 2341 issue: 4 year: 2013 ident: 1822_CR30 publication-title: SIAM J. Opt. doi: 10.1137/120880811 – volume: 60 start-page: 223 issue: 2 year: 2018 ident: 1822_CR11 publication-title: SIAM Rev. doi: 10.1137/16M1080173 – ident: 1822_CR54 doi: 10.1007/978-1-4612-1880-7_29 – ident: 1822_CR7 – volume: 185 start-page: 315 issue: 1 year: 2021 ident: 1822_CR16 publication-title: Math. Progr. doi: 10.1007/s10107-019-01431-x – volume: 3 start-page: 60 issue: 1 year: 1993 ident: 1822_CR48 publication-title: SIAM J. Opt. doi: 10.1137/0803004 – volume: 184 start-page: 71 issue: 1 year: 2019 ident: 1822_CR15 publication-title: Math. Progr. – ident: 1822_CR26 – ident: 1822_CR49 – ident: 1822_CR50 – volume: 88 start-page: 1 year: 2012 ident: 1822_CR19 publication-title: Optima – ident: 1822_CR22 – volume: 14 start-page: 567 year: 2013 ident: 1822_CR44 publication-title: J. Mach. Learn. Res. – ident: 1822_CR4 – year: 2019 ident: 1822_CR33 publication-title: Found. Comput. Math. doi: 10.1007/s10208-019-09429-9 – ident: 1822_CR42 doi: 10.1109/ALLERTON.2016.7852377 – ident: 1822_CR36 – ident: 1822_CR6 – ident: 1822_CR32 – ident: 1822_CR57 – ident: 1822_CR25 – volume: 1 start-page: 38 issue: 1 year: 1973 ident: 1822_CR31 publication-title: Ann. Stat. doi: 10.1214/aos/1193342380 – ident: 1822_CR53 doi: 10.1109/SFCS.1977.24 – ident: 1822_CR21 – volume: 10 start-page: 451 issue: 4 year: 1994 ident: 1822_CR35 publication-title: J. Complex. doi: 10.1006/jcom.1994.1025 – ident: 1822_CR3 – volume: 20 start-page: 2833 issue: 6 year: 2010 ident: 1822_CR17 publication-title: Siam J. Opt. doi: 10.1137/090774100 – ident: 1822_CR29 – ident: 1822_CR46 |
| SSID | ssj0001388 |
| Score | 2.6840336 |
| Snippet | We lower bound the complexity of finding
ϵ
-stationary points (with gradient norm at most
ϵ
) using stochastic first-order methods. In a well-studied model... We lower bound the complexity of finding [Formula omitted]-stationary points (with gradient norm at most [Formula omitted]) using stochastic first-order... We lower bound the complexity of finding ϵ-stationary points (with gradient norm at most ϵ) using stochastic first-order methods. In a well-studied model where... |
| SourceID | proquest gale crossref springer |
| SourceType | Aggregation Database Enrichment Source Index Database Publisher |
| StartPage | 165 |
| SubjectTerms | Algorithms Analysis Calculus of Variations and Optimal Control; Optimization Combinatorics Full Length Paper Lower bounds Mathematical and Computational Physics Mathematical Methods in Physics Mathematics Mathematics and Statistics Mathematics of Computing Minimax technique Numerical Analysis Optimization Queries Smoothness Theoretical |
| Title | Lower bounds for non-convex stochastic optimization |
| URI | https://link.springer.com/article/10.1007/s10107-022-01822-7 https://www.proquest.com/docview/2804309854 |
| Volume | 199 |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LT8MwDLZgu8AB8RSFMfWAxAEqkTZp0mMHGxMwTkwapyjNGnHgJTokfj52127jKXGqqqZJ5TT258T-DHCYOdRxuUsCZmLauslsoBKKBOCMo3sgcu5oH3JwE_eH_HIkRlVSWFFHu9dHkqWmXkh2Y7StFlIoAZq1QC5DU6DvToFcwzCd6V8WKVUXaiV0UKXK_NzHJ3P0VSl_Ox0tjU5vHdYqtOin0-ndgKX8aRNWFzgE8W4wI14ttiC6pqpnfkbFkgofAamP7n1Qhpa_-4jz7L0hYmb_GTXFY5WCuQ3DXvf2rB9UdRECGwk1CYS00jkXI_iKmT0VueJ4CZ1kiI2YYbllYysSZUTmkjh0UcZdpLgwqNcQYJloBxo4eL4L_jgPuUOIlZ1awxMpEiPR_xnLLIqMRePlAavFo21FGk61Kx70nO6YRKpRpLoUqZYeHM_eeZlSZvzZ-oikrmk9Yc_WVGkB-H3ETKVTSYxvVMbTg1Y9MbpaaIUOFZGWJUpwD07qyZo__n3cvf8134cVKjQ_DXVsQWPy-pYfIByZZG1opp3zTo-uF3dX3Xb5N34AT9jUqA |
| linkProvider | Springer Nature |
| linkToHtml | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwtV07T8MwED7xGIAB8RTlmQHEAJFwYsfOwFDxUAstE5XYjOPaYoAW0SLg__BDuUuTlrfEwBRFcWzrbN99tu--A9jOPOo459OQmYSObjIbqpQ8ATjjuD0Qjns6h2xeJLUWP7sSV2PwWsbC5N7u5ZVkrqnfBbsxOlaLyJUAzVooC1fKc_fyhBu13mH9GEd1J4pOTy6PamGRSyC0sVD9UEgrvfcJApaE2QPhFMdH5CVDPMEMc5a1rUiVEZlPk8jHGfex4sKgLkBQYmKsdxwmsbCitdOKqkN9z2KlysSwhEaK0Jzv-_zB_H02Al9uY3MjdzoHswU6DaqD6TQPY66zADPvOAvxrTkkeu0tQtygLGtBRsmZegEC4KDT7YS5K_tzgLjS3hgigg66qJnuipDPJWj9i-yWYQIbdysQtF3EPUK67MAankqRGon7rbbM4thYNJYVYKV4tC1IyilXxq0e0SuTSDWKVOci1bICe8N_7gcUHb-W3iWpa1q_WLM1RRgC9o-YsHRVEsMcpQ2twHo5MLpY2D0dKSJJS5XgFdgvB2v0-ed2V_9WfAumapfNhm7UL87XYJqS3A_cLNdhov_w6DYQCvWzzXwmBnD931P_DX9qDVY |
| linkToPdf | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwtV07T8MwED5BkRAMiKcoFMgAYoCIOrFjZ2CogIq3GKjEZhzXFgMURIuAf8VP5C5NyhuJgSmK4tjW2b77bN99B7CaedRxzqchMwkd3WQ2VCl5AnDGcXsgHPd0Dnlymuy3-OGFuBiClzIWJvd2L68k-zENxNLU6W3dtf3Wu8A3RkdsEbkVoIkLZeFWeeSeH3HT1t0-2MURXoui5t75zn5Y5BUIbSxULxTSSu99guAlYbYunOL4iLxkiC2YYc6ythWpMiLzaRL5OOM-VlwY1AsIUEyM9Q7DCKfoY1xBragx0P0sVqpMEkvIpAjT-b7PH0zhZ4Pw5WY2N3jNSZgokGrQ6E-tKRhynWkYf8dfiG8nA9LX7gzEx5RxLcgoUVM3QDAcdG47Ye7W_hQgxrRXhkihg1vUUjdF-OcstP5FdnNQwcbdPARtF3GP8C6rW8NTKVIjce_VllkcG4uGswqsFI-2BWE55c241m9UyyRSjSLVuUi1rMLG4J-7Pl3Hr6XXSeqa1jLWbE0RkoD9I1Ys3ZDENkcpRKtQKwdGF4u8qyNFhGmpErwKm-VgvX3-ud2FvxVfgdGz3aY-Pjg9WoQxynff97isQaV3_-CWEBX1suV8IgZw-d8z_xWgCRGJ |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Lower+bounds+for+non-convex+stochastic+optimization&rft.jtitle=Mathematical+programming&rft.au=Arjevani+Yossi&rft.au=Carmon+Yair&rft.au=Duchi%2C+John+C&rft.au=Foster%2C+Dylan+J&rft.date=2023-05-01&rft.pub=Springer+Nature+B.V&rft.issn=0025-5610&rft.eissn=1436-4646&rft.volume=199&rft.issue=1-2&rft.spage=165&rft.epage=214&rft_id=info:doi/10.1007%2Fs10107-022-01822-7&rft.externalDBID=NO_FULL_TEXT |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0025-5610&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0025-5610&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0025-5610&client=summon |