Sparsity in long-time control of neural ODEs
We consider the neural ODE and optimal control perspective of supervised learning, with ℓ1-control penalties, where rather than only minimizing a final cost (the empirical risk) for the state, we integrate this cost over the entire time horizon. We prove that any optimal control (for this cost) vani...
Saved in:
| Published in | Systems & control letters Vol. 172; p. 105452 |
|---|---|
| Main Authors | , |
| Format | Journal Article |
| Language | English |
| Published |
Elsevier B.V
01.02.2023
|
| Subjects | |
| Online Access | Get full text |
Cover
Loading…
| Abstract | We consider the neural ODE and optimal control perspective of supervised learning, with ℓ1-control penalties, where rather than only minimizing a final cost (the empirical risk) for the state, we integrate this cost over the entire time horizon. We prove that any optimal control (for this cost) vanishes beyond some positive stopping time. When seen in the discrete-time context, this result entails an ordered sparsity pattern for the parameters of the associated residual neural network: ordered in the sense that these parameters are all 0 beyond a certain layer. Furthermore, we provide a polynomial stability estimate for the empirical risk with respect to the time horizon. This can be seen as a turnpike property, for nonsmooth dynamics and functionals with ℓ1 penalties, and without any smallness assumptions on the data, both of which are new in the literature. |
|---|---|
| AbstractList | We consider the neural ODE and optimal control perspective of supervised learning, with ℓ1-control penalties, where rather than only minimizing a final cost (the empirical risk) for the state, we integrate this cost over the entire time horizon. We prove that any optimal control (for this cost) vanishes beyond some positive stopping time. When seen in the discrete-time context, this result entails an ordered sparsity pattern for the parameters of the associated residual neural network: ordered in the sense that these parameters are all 0 beyond a certain layer. Furthermore, we provide a polynomial stability estimate for the empirical risk with respect to the time horizon. This can be seen as a turnpike property, for nonsmooth dynamics and functionals with ℓ1 penalties, and without any smallness assumptions on the data, both of which are new in the literature. |
| ArticleNumber | 105452 |
| Author | Esteve-Yagüe, Carlos Geshkovski, Borjan |
| Author_xml | – sequence: 1 givenname: Carlos surname: Esteve-Yagüe fullname: Esteve-Yagüe, Carlos email: ce423@cam.ac.uk organization: Department of Applied Mathematics and Theoretical Physics, University of Cambridge, Cambridge, CB3 0WA, UK – sequence: 2 givenname: Borjan orcidid: 0000-0002-7890-3352 surname: Geshkovski fullname: Geshkovski, Borjan email: borjan@mit.edu organization: Department of Mathematics, Massachusetts Institute of Technology, Simons Building, Room 246C, 77 Massachusetts Avenue, Cambridge, MA, 02139-4307, USA |
| BookMark | eNqFj01LAzEQQINUsK3-Bdkf4NZJskl2wYNS6wcUelDPIc3OSso2KUkU-u_dUr146WlgmPeYNyEjHzwSck1hRoHK280s7ZMNvscZA8aGpagEOyNjWitWqkbIERkPh6qUDaUXZJLSBgAYcD4mN287E5PL-8L5og_-s8xui8WgyzH0RegKj1_R9MXqcZEuyXln-oRXv3NKPp4W7_OXcrl6fp0_LEvLmyqX1ALWIIwwaq0qC4quWSU6VCA73nBJFVStNbXBWhhOpWiblnVMNEwZUIbyKbk7em0MKUXstHXZZHd4yrheU9CHcr3Rf-X6UK6P5QMu_-G76LYm7k-D90cQh7hvh1En69BbbF1Em3Ub3CnFD8SDeC0 |
| CitedBy_id | crossref_primary_10_1016_j_matpur_2023_10_005 crossref_primary_10_1016_j_sysconle_2025_106069 crossref_primary_10_1137_21M1411433 crossref_primary_10_1016_j_eswa_2024_126041 crossref_primary_10_1016_j_neunet_2024_106640 |
| Cites_doi | 10.1016/j.sysconle.2022.105182 10.1111/j.2517-6161.1996.tb02080.x 10.1016/j.ejcon.2017.02.001 10.1051/cocv/2019038 10.3934/jcd.2019009 10.3934/mcrf.2013.3.447 10.1098/rsta.2013.0400 10.1088/1361-6420/aa9a90 10.1017/S0956792521000139 10.1142/S0218202515400059 10.1137/18M1223083 10.1007/s10957-016-1016-9 10.4171/JEMS/1221 10.1016/j.jde.2014.09.005 10.1017/S0962492922000046 10.1137/18M118709X 10.1002/oca.781 10.1109/CVPR.2016.90 10.1109/TAC.2022.3190051 10.4171/jems/245 10.1145/3446776 10.1088/1361-6544/ac4e61 10.1109/TAC.2022.3181222 10.1137/0907087 10.1007/s40304-017-0103-z 10.1098/rsta.2015.0203 10.1002/oca.2126 |
| ContentType | Journal Article |
| Copyright | 2022 Elsevier B.V. |
| Copyright_xml | – notice: 2022 Elsevier B.V. |
| DBID | AAYXX CITATION |
| DOI | 10.1016/j.sysconle.2022.105452 |
| DatabaseName | CrossRef |
| DatabaseTitle | CrossRef |
| DatabaseTitleList | |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering |
| EISSN | 1872-7956 |
| ExternalDocumentID | 10_1016_j_sysconle_2022_105452 S0167691122002298 |
| GroupedDBID | --K --M -~X .DC .~1 0R~ 123 1B1 1RT 1~. 1~5 29Q 4.4 457 4G. 5VS 7-5 71M 8P~ 9JN 9JO AAAKF AAAKG AABNK AACTN AAEDT AAEDW AAIAV AAIKJ AAKOC AALRI AAOAW AAQFI AAQXK AARIN AAXUO ABFNM ABJNI ABMAC ABTAH ABUCO ABXDB ABYKQ ACDAQ ACGFS ACNNM ACRLP ADBBV ADEZE ADIYS ADMUD ADTZH AEBSH AECPX AEKER AENEX AFKWA AFTJW AGHFR AGUBO AGYEJ AHHHB AHJVU AIEXJ AIKHN AITUG AJBFU AJOXV ALMA_UNASSIGNED_HOLDINGS AMFUW AMRAJ APLSM ASPBG AVWKF AXJTR AZFZN BJAXD BKOJK BLXMC CS3 DU5 EBS EFJIC EFLBG EJD EO8 EO9 EP2 EP3 FDB FEDTE FGOYB FIRID FNPLU FYGXN G-2 G-Q GBLVA HAMUX HVGLF HZ~ IHE J1W JJJVA KOM LY1 LY7 M41 MO0 N9A O-L O9- OAUVE OZT P-8 P-9 P2P PC. Q38 R2- RIG ROL RPZ SDF SDG SDP SDS SES SET SEW SPC SPCBC SSB SSD SST SSZ T5K TN5 WH7 WUQ XPP ZMT ZY4 ~G- AATTM AAXKI AAYWO AAYXX ABWVN ACRPL ACVFH ADCNI ADNMO AEIPS AEUPX AFJKZ AFPUW AFXIZ AGCQF AGQPQ AGRNS AIGII AIIUN AKBMS AKRWK AKYEP ANKPU APXCP BNPGV CITATION SSH |
| ID | FETCH-LOGICAL-c394t-1c0e805a5a7b74c071b245fe706f39361704dca8ae85a3165d9d2f25927a07a13 |
| IEDL.DBID | .~1 |
| ISSN | 0167-6911 |
| IngestDate | Thu Apr 24 22:57:55 EDT 2025 Tue Jul 01 03:29:11 EDT 2025 Fri Feb 23 02:38:07 EST 2024 |
| IsDoiOpenAccess | false |
| IsOpenAccess | true |
| IsPeerReviewed | true |
| IsScholarly | true |
| Keywords | Neural ODEs Learning Sparsity Optimal Control |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c394t-1c0e805a5a7b74c071b245fe706f39361704dca8ae85a3165d9d2f25927a07a13 |
| ORCID | 0000-0002-7890-3352 |
| OpenAccessLink | https://hal.science/hal-03154149v2/file/manuscript.pdf |
| ParticipantIDs | crossref_citationtrail_10_1016_j_sysconle_2022_105452 crossref_primary_10_1016_j_sysconle_2022_105452 elsevier_sciencedirect_doi_10_1016_j_sysconle_2022_105452 |
| ProviderPackageCode | CITATION AAYXX |
| PublicationCentury | 2000 |
| PublicationDate | February 2023 2023-02-00 |
| PublicationDateYYYYMMDD | 2023-02-01 |
| PublicationDate_xml | – month: 02 year: 2023 text: February 2023 |
| PublicationDecade | 2020 |
| PublicationTitle | Systems & control letters |
| PublicationYear | 2023 |
| Publisher | Elsevier B.V |
| Publisher_xml | – name: Elsevier B.V |
| References | Caponigro, Fornasier, Piccoli, Trélat (b34) 2013; 3 Zhang, Bengio, Hardt, Recht, Vinyals (b3) 2021; 64 T.Q. Chen, Y. Rubanova, J. Bettencourt, D.K. Duvenaud, Neural ordinary differential equations, in: Advances in Neural Information Processing Systems, 2018, pp. 6571–6583. E. Dupont, A. Doucet, Y.W. Teh, Augmented Neural ODEs, in: Advances in Neural Information Processing Systems, 2019, pp. 3134–3144. Esteve-Yagüe, Geshkovski, Pighin, Zuazua (b11) 2021 Vossen, Maurer (b33) 2006; 27 Li, Lin, Shen (b21) 2022 Ruiz-Balet, Zuazua (b23) 2021 Caponigro, Fornasier, Piccoli, Trélat (b36) 2015; 25 Yoon, Shin, Yang (b42) 2022 Haber, Ruthotto (b5) 2017; 34 Faulwasser, Bonvin (b20) 2017; 35 Ruiz-Balet, Affili, Zuazua (b24) 2022; 162 Tabuada, Gharesifard (b26) 2022 Alt, Schneider (b29) 2015; 36 Kidger (b45) 2022 Kalise, Kunisch, Rao (b31) 2017; 172 Li, Chen, Tai, E (b39) 2017; 18 Bölcskei, Grohs, Kutyniok, Petersen (b47) 2020 Esteve-Yagüe, Geshkovski, Pighin, Zuazua (b16) 2022; 35 Yeh (b48) 2006 Elsayed, Krishnan, Mobahi, Regan, Bengio (b9) 2018; 31 Fornasier, Piccoli, Rossi (b35) 2014; 372 Geshkovski, Zuazua (b8) 2022; 31 Grathwohl, Chen, Bettencourt, Sutskever, Duvenaud (b43) 2018 Geshkovski (b27) 2021 E (b4) 2017; 5 Zuazua (b28) 2010; 13 Geshkovski, Zuazua (b30) 2022 K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 770–778. Papamakarios, Nalisnick, Rezende, Mohamed, Lakshminarayanan (b44) 2021; 22 Celledoni, Ehrhardt, Etmann, McLachlan, Owren, Schönlieb, Sherry (b38) 2021; 32 Tibshirani (b2) 1996; 58 Grüne, Schaller, Schiela (b10) 2019; 57 Trélat, Zuazua (b14) 2015; 258 Gugat, Schuster, Zuazua (b19) 2021 Y. Rubanova, R.T. Chen, D.K. Duvenaud, Latent ordinary differential equations for irregularly-sampled time series, in: Advances in Neural Information Processing Systems, 2019, pp. 5320–5330. Faulwasser, Grüne (b15) 2022; 23 Effland, Kobler, Kunisch, Pock (b18) 2020 Chizat, Bach (b13) 2018; 31 Benning, Celledoni, Ehrhardt, Owren, Schönlieb (b40) 2019; 6 Kalise, Kunisch, Rao (b32) 2020; 26 Agrachev, Sarychev (b22) 2021 Faulwasser, Hempel, Streif (b17) 2021 Goodfellow, Bengio, Courville (b12) 2016 Santosa, Symes (b1) 1986; 7 Bárcena-Petisco (b25) 2021 Mallat (b46) 2016; 374 Li (10.1016/j.sysconle.2022.105452_b39) 2017; 18 Faulwasser (10.1016/j.sysconle.2022.105452_b15) 2022; 23 Tibshirani (10.1016/j.sysconle.2022.105452_b2) 1996; 58 Esteve-Yagüe (10.1016/j.sysconle.2022.105452_b11) 2021 Caponigro (10.1016/j.sysconle.2022.105452_b34) 2013; 3 Mallat (10.1016/j.sysconle.2022.105452_b46) 2016; 374 E (10.1016/j.sysconle.2022.105452_b4) 2017; 5 Zhang (10.1016/j.sysconle.2022.105452_b3) 2021; 64 Chizat (10.1016/j.sysconle.2022.105452_b13) 2018; 31 Geshkovski (10.1016/j.sysconle.2022.105452_b8) 2022; 31 Li (10.1016/j.sysconle.2022.105452_b21) 2022 Caponigro (10.1016/j.sysconle.2022.105452_b36) 2015; 25 Gugat (10.1016/j.sysconle.2022.105452_b19) 2021 Goodfellow (10.1016/j.sysconle.2022.105452_b12) 2016 Agrachev (10.1016/j.sysconle.2022.105452_b22) 2021 Haber (10.1016/j.sysconle.2022.105452_b5) 2017; 34 Trélat (10.1016/j.sysconle.2022.105452_b14) 2015; 258 Grathwohl (10.1016/j.sysconle.2022.105452_b43) 2018 Zuazua (10.1016/j.sysconle.2022.105452_b28) 2010; 13 Faulwasser (10.1016/j.sysconle.2022.105452_b20) 2017; 35 Vossen (10.1016/j.sysconle.2022.105452_b33) 2006; 27 Kalise (10.1016/j.sysconle.2022.105452_b31) 2017; 172 Celledoni (10.1016/j.sysconle.2022.105452_b38) 2021; 32 Tabuada (10.1016/j.sysconle.2022.105452_b26) 2022 10.1016/j.sysconle.2022.105452_b37 Papamakarios (10.1016/j.sysconle.2022.105452_b44) 2021; 22 Kalise (10.1016/j.sysconle.2022.105452_b32) 2020; 26 Yoon (10.1016/j.sysconle.2022.105452_b42) 2022 Bárcena-Petisco (10.1016/j.sysconle.2022.105452_b25) 2021 Elsayed (10.1016/j.sysconle.2022.105452_b9) 2018; 31 Grüne (10.1016/j.sysconle.2022.105452_b10) 2019; 57 Santosa (10.1016/j.sysconle.2022.105452_b1) 1986; 7 Geshkovski (10.1016/j.sysconle.2022.105452_b30) 2022 10.1016/j.sysconle.2022.105452_b41 Benning (10.1016/j.sysconle.2022.105452_b40) 2019; 6 Geshkovski (10.1016/j.sysconle.2022.105452_b27) 2021 10.1016/j.sysconle.2022.105452_b7 10.1016/j.sysconle.2022.105452_b6 Ruiz-Balet (10.1016/j.sysconle.2022.105452_b23) 2021 Fornasier (10.1016/j.sysconle.2022.105452_b35) 2014; 372 Yeh (10.1016/j.sysconle.2022.105452_b48) 2006 Alt (10.1016/j.sysconle.2022.105452_b29) 2015; 36 Faulwasser (10.1016/j.sysconle.2022.105452_b17) 2021 Ruiz-Balet (10.1016/j.sysconle.2022.105452_b24) 2022; 162 Kidger (10.1016/j.sysconle.2022.105452_b45) 2022 Effland (10.1016/j.sysconle.2022.105452_b18) 2020 Bölcskei (10.1016/j.sysconle.2022.105452_b47) 2020 Esteve-Yagüe (10.1016/j.sysconle.2022.105452_b16) 2022; 35 |
| References_xml | – volume: 31 year: 2018 ident: b9 article-title: Large margin deep networks for classification publication-title: Adv. Neural Inf. Process. Syst. – year: 2021 ident: b27 article-title: Control in Moving Interfaces and Deep Learning – volume: 162 year: 2022 ident: b24 article-title: Interpolation and approximation via momentum ResNets and neural ODEs publication-title: Systems Control Lett. – volume: 22 start-page: 1 year: 2021 end-page: 64 ident: b44 article-title: Normalizing flows for probabilistic modeling and inference publication-title: J. Mach. Learn. Res. – volume: 7 start-page: 1307 year: 1986 end-page: 1330 ident: b1 article-title: Linear inversion of band-limited reflection seismograms publication-title: SIAM J. Sci. Stat. Comput. – year: 2022 ident: b26 article-title: Universal approximation power of deep residual neural networks through the lens of control publication-title: IEEE Trans. Automat. Control – start-page: 1 year: 2021 end-page: 20 ident: b22 article-title: Control on the manifolds of mappings with a view to the deep learning publication-title: J. Dyn. Control Syst. – volume: 18 start-page: 5998 year: 2017 end-page: 6026 ident: b39 article-title: Maximum principle based algorithms for deep learning publication-title: J. Mach. Learn. Res. – reference: K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 770–778. – volume: 36 start-page: 512 year: 2015 end-page: 534 ident: b29 article-title: Linear-quadratic control problems with publication-title: Optimal Control Appl. Methods – volume: 3 start-page: 447 year: 2013 end-page: 466 ident: b34 article-title: Sparse stabilization and optimal control of the Cucker-Smale model publication-title: Math. Control Relat. Fields – year: 2022 ident: b21 article-title: Deep learning via dynamical systems: An approximation perspective publication-title: J. Eur. Math. Soc. – year: 2021 ident: b25 article-title: Optimal control for neural ODE in a long time horizon and applications to the classification and simultaneous controllability problems – year: 2021 ident: b23 article-title: Neural ODE control for classification, approximation and transport – start-page: 17 year: 2021 end-page: 41 ident: b19 article-title: The finite-time turnpike phenomenon for optimal control problems: Stabilization by non-smooth tracking terms publication-title: Stabilization of Distributed Parameter Systems: Design Methods and Applications – volume: 35 start-page: 1652 year: 2022 ident: b16 article-title: Turnpike in Lipschitz—nonlinear optimal control publication-title: Nonlinearity – year: 2021 ident: b11 article-title: Large-time asymptotics in deep learning – volume: 258 start-page: 81 year: 2015 end-page: 114 ident: b14 article-title: The turnpike property in finite-dimensional nonlinear optimal control publication-title: J. Differential Equations – reference: T.Q. Chen, Y. Rubanova, J. Bettencourt, D.K. Duvenaud, Neural ordinary differential equations, in: Advances in Neural Information Processing Systems, 2018, pp. 6571–6583. – year: 2016 ident: b12 article-title: Deep Learning – volume: 13 start-page: 85 year: 2010 end-page: 117 ident: b28 article-title: Switching control publication-title: J. Eur. Math. Soc. – volume: 26 start-page: 61 year: 2020 ident: b32 article-title: Sparse and switching infinite horizon optimal controls with mixed-norm penalizations publication-title: ESAIM Control Optim. Calc. Var. – volume: 23 start-page: 367 year: 2022 ident: b15 article-title: Turnpike properties in optimal control publication-title: Numer. Control: A – year: 2018 ident: b43 article-title: Ffjord: Free-form continuous dynamics for scalable reversible generative models – reference: Y. Rubanova, R.T. Chen, D.K. Duvenaud, Latent ordinary differential equations for irregularly-sampled time series, in: Advances in Neural Information Processing Systems, 2019, pp. 5320–5330. – volume: 32 start-page: 888 year: 2021 end-page: 936 ident: b38 article-title: Structure-preserving deep learning publication-title: European J. Appl. Math. – year: 2021 ident: b17 article-title: On the turnpike to design of deep neural nets: Explicit depth bounds – start-page: 1 year: 2020 end-page: 21 ident: b18 article-title: Variational networks: An optimal control approach to early stopping variational methods for image restoration publication-title: J. Math. Imaging Vision – reference: E. Dupont, A. Doucet, Y.W. Teh, Augmented Neural ODEs, in: Advances in Neural Information Processing Systems, 2019, pp. 3134–3144. – volume: 27 start-page: 301 year: 2006 end-page: 321 ident: b33 article-title: On publication-title: Optim. Control Appl. Methods – volume: 35 start-page: 34 year: 2017 end-page: 41 ident: b20 article-title: Exact turnpike properties and economic NMPC publication-title: Eur. J. Control – year: 2006 ident: b48 article-title: Real Analysis: Theory of Measure and Integration – volume: 172 start-page: 481 year: 2017 end-page: 517 ident: b31 article-title: Infinite horizon sparse optimal control publication-title: J. Optim. Theory Appl. – volume: 6 start-page: 171 year: 2019 ident: b40 article-title: Deep learning as optimal control problems: Models and numerical methods publication-title: J. Comput. Dyn. – volume: 31 start-page: 135 year: 2022 end-page: 263 ident: b8 article-title: Turnpike in optimal control of PDEs, ResNets, and beyond publication-title: Acta Numer. – volume: 31 year: 2018 ident: b13 article-title: On the global convergence of gradient descent for over-parameterized models using optimal transport publication-title: Adv. Neural Inf. Process. Syst. – volume: 57 start-page: 2753 year: 2019 end-page: 2774 ident: b10 article-title: Sensitivity analysis of optimal control for a class of parabolic PDEs motivated by model predictive control publication-title: SIAM J. Control Optim. – volume: 58 start-page: 267 year: 1996 end-page: 288 ident: b2 article-title: Regression shrinkage and selection via the Lasso publication-title: J. R. Stat. Soc. Ser. B Stat. Methodol. – year: 2022 ident: b45 article-title: On neural differential equations – volume: 374 year: 2016 ident: b46 article-title: Understanding deep convolutional networks publication-title: Phil. Trans. R. Soc. A – year: 2022 ident: b30 article-title: Optimal actuator design via Brunovsky’s normal form publication-title: IEEE Trans. Automat. Control – volume: 25 start-page: 521 year: 2015 end-page: 564 ident: b36 article-title: Sparse stabilization and control of alignment models publication-title: Math. Models Methods Appl. Sci. – year: 2022 ident: b42 article-title: Learning polymorphic Neural ODEs with time-evolving mixture publication-title: IEEE Trans. Pattern Anal. Mach. Intell. – start-page: 8 year: 2020 end-page: 45 ident: b47 article-title: Optimal approximation with sparsely connected deep neural networks publication-title: SIAM J. Math. Data Sci. – volume: 64 start-page: 107 year: 2021 end-page: 115 ident: b3 article-title: Understanding deep learning (still) requires rethinking generalization publication-title: Commun. ACM – volume: 372 year: 2014 ident: b35 article-title: Mean-field sparse optimal control publication-title: Phil. Trans. R. Soc. A – volume: 5 start-page: 1 year: 2017 end-page: 11 ident: b4 article-title: A proposal on machine learning via dynamical systems publication-title: Commun. Math. Stat. – volume: 34 year: 2017 ident: b5 article-title: Stable architectures for deep neural networks publication-title: Inverse Problems – volume: 162 year: 2022 ident: 10.1016/j.sysconle.2022.105452_b24 article-title: Interpolation and approximation via momentum ResNets and neural ODEs publication-title: Systems Control Lett. doi: 10.1016/j.sysconle.2022.105182 – year: 2021 ident: 10.1016/j.sysconle.2022.105452_b11 – volume: 58 start-page: 267 issue: 1 year: 1996 ident: 10.1016/j.sysconle.2022.105452_b2 article-title: Regression shrinkage and selection via the Lasso publication-title: J. R. Stat. Soc. Ser. B Stat. Methodol. doi: 10.1111/j.2517-6161.1996.tb02080.x – volume: 35 start-page: 34 year: 2017 ident: 10.1016/j.sysconle.2022.105452_b20 article-title: Exact turnpike properties and economic NMPC publication-title: Eur. J. Control doi: 10.1016/j.ejcon.2017.02.001 – year: 2021 ident: 10.1016/j.sysconle.2022.105452_b25 – year: 2022 ident: 10.1016/j.sysconle.2022.105452_b45 – volume: 26 start-page: 61 year: 2020 ident: 10.1016/j.sysconle.2022.105452_b32 article-title: Sparse and switching infinite horizon optimal controls with mixed-norm penalizations publication-title: ESAIM Control Optim. Calc. Var. doi: 10.1051/cocv/2019038 – volume: 6 start-page: 171 issue: 2 year: 2019 ident: 10.1016/j.sysconle.2022.105452_b40 article-title: Deep learning as optimal control problems: Models and numerical methods publication-title: J. Comput. Dyn. doi: 10.3934/jcd.2019009 – volume: 3 start-page: 447 issue: 4 year: 2013 ident: 10.1016/j.sysconle.2022.105452_b34 article-title: Sparse stabilization and optimal control of the Cucker-Smale model publication-title: Math. Control Relat. Fields doi: 10.3934/mcrf.2013.3.447 – volume: 372 issue: 2028 year: 2014 ident: 10.1016/j.sysconle.2022.105452_b35 article-title: Mean-field sparse optimal control publication-title: Phil. Trans. R. Soc. A doi: 10.1098/rsta.2013.0400 – ident: 10.1016/j.sysconle.2022.105452_b6 – volume: 34 issue: 1 year: 2017 ident: 10.1016/j.sysconle.2022.105452_b5 article-title: Stable architectures for deep neural networks publication-title: Inverse Problems doi: 10.1088/1361-6420/aa9a90 – volume: 32 start-page: 888 issue: 5 year: 2021 ident: 10.1016/j.sysconle.2022.105452_b38 article-title: Structure-preserving deep learning publication-title: European J. Appl. Math. doi: 10.1017/S0956792521000139 – volume: 22 start-page: 1 issue: 57 year: 2021 ident: 10.1016/j.sysconle.2022.105452_b44 article-title: Normalizing flows for probabilistic modeling and inference publication-title: J. Mach. Learn. Res. – year: 2021 ident: 10.1016/j.sysconle.2022.105452_b23 – volume: 25 start-page: 521 issue: 03 year: 2015 ident: 10.1016/j.sysconle.2022.105452_b36 article-title: Sparse stabilization and control of alignment models publication-title: Math. Models Methods Appl. Sci. doi: 10.1142/S0218202515400059 – volume: 18 start-page: 5998 issue: 1 year: 2017 ident: 10.1016/j.sysconle.2022.105452_b39 article-title: Maximum principle based algorithms for deep learning publication-title: J. Mach. Learn. Res. – ident: 10.1016/j.sysconle.2022.105452_b37 – volume: 31 year: 2018 ident: 10.1016/j.sysconle.2022.105452_b9 article-title: Large margin deep networks for classification publication-title: Adv. Neural Inf. Process. Syst. – volume: 57 start-page: 2753 issue: 4 year: 2019 ident: 10.1016/j.sysconle.2022.105452_b10 article-title: Sensitivity analysis of optimal control for a class of parabolic PDEs motivated by model predictive control publication-title: SIAM J. Control Optim. doi: 10.1137/18M1223083 – volume: 172 start-page: 481 issue: 2 year: 2017 ident: 10.1016/j.sysconle.2022.105452_b31 article-title: Infinite horizon sparse optimal control publication-title: J. Optim. Theory Appl. doi: 10.1007/s10957-016-1016-9 – volume: 31 year: 2018 ident: 10.1016/j.sysconle.2022.105452_b13 article-title: On the global convergence of gradient descent for over-parameterized models using optimal transport publication-title: Adv. Neural Inf. Process. Syst. – year: 2022 ident: 10.1016/j.sysconle.2022.105452_b21 article-title: Deep learning via dynamical systems: An approximation perspective publication-title: J. Eur. Math. Soc. doi: 10.4171/JEMS/1221 – volume: 258 start-page: 81 issue: 1 year: 2015 ident: 10.1016/j.sysconle.2022.105452_b14 article-title: The turnpike property in finite-dimensional nonlinear optimal control publication-title: J. Differential Equations doi: 10.1016/j.jde.2014.09.005 – volume: 31 start-page: 135 year: 2022 ident: 10.1016/j.sysconle.2022.105452_b8 article-title: Turnpike in optimal control of PDEs, ResNets, and beyond publication-title: Acta Numer. doi: 10.1017/S0962492922000046 – year: 2021 ident: 10.1016/j.sysconle.2022.105452_b17 – ident: 10.1016/j.sysconle.2022.105452_b41 – start-page: 8 issue: 1 year: 2020 ident: 10.1016/j.sysconle.2022.105452_b47 article-title: Optimal approximation with sparsely connected deep neural networks publication-title: SIAM J. Math. Data Sci. doi: 10.1137/18M118709X – start-page: 17 year: 2021 ident: 10.1016/j.sysconle.2022.105452_b19 article-title: The finite-time turnpike phenomenon for optimal control problems: Stabilization by non-smooth tracking terms – year: 2016 ident: 10.1016/j.sysconle.2022.105452_b12 – start-page: 1 year: 2021 ident: 10.1016/j.sysconle.2022.105452_b22 article-title: Control on the manifolds of mappings with a view to the deep learning publication-title: J. Dyn. Control Syst. – start-page: 1 year: 2020 ident: 10.1016/j.sysconle.2022.105452_b18 article-title: Variational networks: An optimal control approach to early stopping variational methods for image restoration publication-title: J. Math. Imaging Vision – volume: 27 start-page: 301 issue: 6 year: 2006 ident: 10.1016/j.sysconle.2022.105452_b33 article-title: On L1-minimization in optimal control and applications to robotics publication-title: Optim. Control Appl. Methods doi: 10.1002/oca.781 – ident: 10.1016/j.sysconle.2022.105452_b7 doi: 10.1109/CVPR.2016.90 – year: 2022 ident: 10.1016/j.sysconle.2022.105452_b26 article-title: Universal approximation power of deep residual neural networks through the lens of control publication-title: IEEE Trans. Automat. Control doi: 10.1109/TAC.2022.3190051 – volume: 23 start-page: 367 year: 2022 ident: 10.1016/j.sysconle.2022.105452_b15 article-title: Turnpike properties in optimal control publication-title: Numer. Control: A – year: 2021 ident: 10.1016/j.sysconle.2022.105452_b27 – volume: 13 start-page: 85 issue: 1 year: 2010 ident: 10.1016/j.sysconle.2022.105452_b28 article-title: Switching control publication-title: J. Eur. Math. Soc. doi: 10.4171/jems/245 – year: 2022 ident: 10.1016/j.sysconle.2022.105452_b42 article-title: Learning polymorphic Neural ODEs with time-evolving mixture publication-title: IEEE Trans. Pattern Anal. Mach. Intell. – volume: 64 start-page: 107 issue: 3 year: 2021 ident: 10.1016/j.sysconle.2022.105452_b3 article-title: Understanding deep learning (still) requires rethinking generalization publication-title: Commun. ACM doi: 10.1145/3446776 – volume: 35 start-page: 1652 issue: 4 year: 2022 ident: 10.1016/j.sysconle.2022.105452_b16 article-title: Turnpike in Lipschitz—nonlinear optimal control publication-title: Nonlinearity doi: 10.1088/1361-6544/ac4e61 – year: 2018 ident: 10.1016/j.sysconle.2022.105452_b43 – year: 2006 ident: 10.1016/j.sysconle.2022.105452_b48 – year: 2022 ident: 10.1016/j.sysconle.2022.105452_b30 article-title: Optimal actuator design via Brunovsky’s normal form publication-title: IEEE Trans. Automat. Control doi: 10.1109/TAC.2022.3181222 – volume: 7 start-page: 1307 year: 1986 ident: 10.1016/j.sysconle.2022.105452_b1 article-title: Linear inversion of band-limited reflection seismograms publication-title: SIAM J. Sci. Stat. Comput. doi: 10.1137/0907087 – volume: 5 start-page: 1 issue: 1 year: 2017 ident: 10.1016/j.sysconle.2022.105452_b4 article-title: A proposal on machine learning via dynamical systems publication-title: Commun. Math. Stat. doi: 10.1007/s40304-017-0103-z – volume: 374 issue: 2065 year: 2016 ident: 10.1016/j.sysconle.2022.105452_b46 article-title: Understanding deep convolutional networks publication-title: Phil. Trans. R. Soc. A doi: 10.1098/rsta.2015.0203 – volume: 36 start-page: 512 issue: 4 year: 2015 ident: 10.1016/j.sysconle.2022.105452_b29 article-title: Linear-quadratic control problems with L1-control cost publication-title: Optimal Control Appl. Methods doi: 10.1002/oca.2126 |
| SSID | ssj0002033 |
| Score | 2.4524567 |
| Snippet | We consider the neural ODE and optimal control perspective of supervised learning, with ℓ1-control penalties, where rather than only minimizing a final cost... |
| SourceID | crossref elsevier |
| SourceType | Enrichment Source Index Database Publisher |
| StartPage | 105452 |
| SubjectTerms | Learning Neural ODEs Optimal Control Sparsity |
| Title | Sparsity in long-time control of neural ODEs |
| URI | https://dx.doi.org/10.1016/j.sysconle.2022.105452 |
| Volume | 172 |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV1LS8NAEF5KvehBfGJ9lBw8mmazmc1mj6W2VIsVrMXewuax0hLSYuvBi7_dnSbRCkIPngJhJySzuzOz4ZvvI-TaTGmsJTc7TVGwIdCRrVwlbVN5J2AOY5Rp_KH_MPT7Y7if8EmNdKpeGIRVlrG_iOnraF3ecUpvOovp1BkhgN43e5UhzoBJbPgFELjKW58_MA9GCzl55PfG0RtdwrPW8mNpTp0Z0mUyhpK3wNnfCWoj6fQOyH5ZLVrt4oUOSS3Nj8jeBofgMbkZLdQaWGFNcyub5682ysVbJQTdmmsLKSvNQx5vu8sTMu51nzt9u5RAsGNPwsp2Y5oGlCuuRCQgNvVAxIDrVFBfe9JDNnVIYhWoNODKc32eyMR4l0smFBXK9U5JPZ_n6RmxlAQlEppAYNKykGag9mOpwIQZD1INDcKr7w7jkh8cZSqysAKCzcLKXyH6Kyz81SDOt92iYMjYaiErt4a_5jo0YXyL7fk_bC_ILorFF5jrS1Jfvb2nV6akWEXN9Zppkp323aA_xOvg6WXwBb7AyuY |
| linkProvider | Elsevier |
| linkToHtml | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV09T8MwED2VMgAD4lOUzwyMpHEcO4lHVFoVaMvQVupmOR9Graq0omVg4bdzbhIoElIH1sgXJc-58zl6fg_gFqc01oJjpinCbBbqyFauEjZ23gnDzRih2vzQ7_b89pA9jfioAo3yLIyhVRa1P6_pq2pdXHEKNJ35eOz0DYHex1ylhmdARbgF2wzT19gY1D9_eB6U5H7yRuDbDF87JjypLz4WuO2cGr1MSo3nLeP07xVqbdVpHcB-0S5a9_kTHUIlzY5gb01E8Bju-nO1YlZY48yazrJX2_jFWwUH3Zppy2hW4k1eHpqLExi2moNG2y48EOzYE2xpuzFJQ8IVV0EUsBgbgogyrtOA-NoTnpFTZ0msQpWGXHmuzxORILxc0ECRQLneKVSzWZaegaUEU0FCEhbiuhwIHKj9WCiGdcZjqWY14OV7y7gQCDc-FVNZMsEmssRLGrxkjlcNnO-4eS6RsTFClLDKX5MtsY5viD3_R-wN7LQH3Y7sPPaeL2DXOMfnBOxLqC7f3tMr7C-W0fXq-_kCPxvK0Q |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Sparsity+in+long-time+control+of+neural+ODEs&rft.jtitle=Systems+%26+control+letters&rft.au=Esteve-Yag%C3%BCe%2C+Carlos&rft.au=Geshkovski%2C+Borjan&rft.date=2023-02-01&rft.issn=0167-6911&rft.volume=172&rft.spage=105452&rft_id=info:doi/10.1016%2Fj.sysconle.2022.105452&rft.externalDBID=n%2Fa&rft.externalDocID=10_1016_j_sysconle_2022_105452 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0167-6911&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0167-6911&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0167-6911&client=summon |