No-Regret Constrained Bayesian Optimization of Noisy and Expensive Hybrid Models using Differentiable Quantile Function Approximations
This paper investigates the problem of efficient constrained global optimization of hybrid models that are a composition of a known white-box function and an expensive multi-output black-box function subject to noisy observations, which often arises in real-world science and engineering applications...
Saved in:
Main Authors | , |
---|---|
Format | Journal Article |
Language | English |
Published |
05.05.2023
|
Subjects | |
Online Access | Get full text |
Cover
Loading…
Abstract | This paper investigates the problem of efficient constrained global
optimization of hybrid models that are a composition of a known white-box
function and an expensive multi-output black-box function subject to noisy
observations, which often arises in real-world science and engineering
applications. We propose a novel method, Constrained Upper Quantile Bound
(CUQB), to solve such problems that directly exploits the composite structure
of the objective and constraint functions that we show leads substantially
improved sampling efficiency. CUQB is a conceptually simple, deterministic
approach that avoid constraint approximations used by previous methods.
Although the CUQB acquisition function is not available in closed form, we
propose a novel differentiable sample average approximation that enables it to
be efficiently maximized. We further derive bounds on the cumulative regret and
constraint violation under a non-parametric Bayesian representation of the
black-box function. Since these bounds depend sublinearly on the number of
iterations under some regularity assumptions, we establis bounds on the
convergence rate to the optimal solution of the original constrained problem.
In contrast to most existing methods, CUQB further incorporates a simple
infeasibility detection scheme, which we prove triggers in a finite number of
iterations when the original problem is infeasible (with high probability given
the Bayesian model). Numerical experiments on several test problems, including
environmental model calibration and real-time optimization of a reactor system,
show that CUQB significantly outperforms traditional Bayesian optimization in
both constrained and unconstrained cases. Furthermore, compared to other
state-of-the-art methods that exploit composite structure, CUQB achieves
competitive empirical performance while also providing substantially improved
theoretical guarantees. |
---|---|
AbstractList | This paper investigates the problem of efficient constrained global
optimization of hybrid models that are a composition of a known white-box
function and an expensive multi-output black-box function subject to noisy
observations, which often arises in real-world science and engineering
applications. We propose a novel method, Constrained Upper Quantile Bound
(CUQB), to solve such problems that directly exploits the composite structure
of the objective and constraint functions that we show leads substantially
improved sampling efficiency. CUQB is a conceptually simple, deterministic
approach that avoid constraint approximations used by previous methods.
Although the CUQB acquisition function is not available in closed form, we
propose a novel differentiable sample average approximation that enables it to
be efficiently maximized. We further derive bounds on the cumulative regret and
constraint violation under a non-parametric Bayesian representation of the
black-box function. Since these bounds depend sublinearly on the number of
iterations under some regularity assumptions, we establis bounds on the
convergence rate to the optimal solution of the original constrained problem.
In contrast to most existing methods, CUQB further incorporates a simple
infeasibility detection scheme, which we prove triggers in a finite number of
iterations when the original problem is infeasible (with high probability given
the Bayesian model). Numerical experiments on several test problems, including
environmental model calibration and real-time optimization of a reactor system,
show that CUQB significantly outperforms traditional Bayesian optimization in
both constrained and unconstrained cases. Furthermore, compared to other
state-of-the-art methods that exploit composite structure, CUQB achieves
competitive empirical performance while also providing substantially improved
theoretical guarantees. |
Author | Paulson, Joel A Lu, Congwen |
Author_xml | – sequence: 1 givenname: Congwen surname: Lu fullname: Lu, Congwen – sequence: 2 givenname: Joel A surname: Paulson fullname: Paulson, Joel A |
BackLink | https://doi.org/10.48550/arXiv.2305.03824$$DView paper in arXiv |
BookMark | eNotkMFOAjEURbvQhaIf4Mr-wGBn2k7bJSKICUI07Eln5pW8BNpJOxDGD_C7xdHVPatzk3NLrnzwQMhDzsZCS8mebDzjaVxwJseM60LckO9VyD5hF6Gj0-BTFy16aOiz7SGh9XTddnjAL9th8DQ4ugqYemp9Q2fnFnzCE9BFX0Vs6HtoYJ_oMaHf0Rd0DiL4Dm21B_pxtBe8wPzo68E1adsYzngYzOmOXDu7T3D_vyOymc8200W2XL--TSfLzJZKZIVUWufM5FyrupaFaUqZK1E6BqUz0jihq1JqKwUYzblplFKmYkIbAap2BR-Rxz_tEGLbxst_7Le_QbZDEP4DC9Ve0Q |
ContentType | Journal Article |
Copyright | http://creativecommons.org/licenses/by/4.0 |
Copyright_xml | – notice: http://creativecommons.org/licenses/by/4.0 |
DBID | AKY EPD GOX |
DOI | 10.48550/arxiv.2305.03824 |
DatabaseName | arXiv Computer Science arXiv Statistics arXiv.org |
DatabaseTitleList | |
Database_xml | – sequence: 1 dbid: GOX name: arXiv.org url: http://arxiv.org/find sourceTypes: Open Access Repository |
DeliveryMethod | fulltext_linktorsrc |
ExternalDocumentID | 2305_03824 |
GroupedDBID | AKY EPD GOX |
ID | FETCH-LOGICAL-a674-257881091387cc529d651746f0e6f959f48b658a54e98339d7779b04894e7cf23 |
IEDL.DBID | GOX |
IngestDate | Mon Jan 08 05:41:29 EST 2024 |
IsDoiOpenAccess | true |
IsOpenAccess | true |
IsPeerReviewed | false |
IsScholarly | false |
Language | English |
LinkModel | DirectLink |
MergedId | FETCHMERGED-LOGICAL-a674-257881091387cc529d651746f0e6f959f48b658a54e98339d7779b04894e7cf23 |
OpenAccessLink | https://arxiv.org/abs/2305.03824 |
ParticipantIDs | arxiv_primary_2305_03824 |
PublicationCentury | 2000 |
PublicationDate | 2023-05-05 |
PublicationDateYYYYMMDD | 2023-05-05 |
PublicationDate_xml | – month: 05 year: 2023 text: 2023-05-05 day: 05 |
PublicationDecade | 2020 |
PublicationYear | 2023 |
Score | 1.8852385 |
SecondaryResourceType | preprint |
Snippet | This paper investigates the problem of efficient constrained global
optimization of hybrid models that are a composition of a known white-box
function and an... |
SourceID | arxiv |
SourceType | Open Access Repository |
SubjectTerms | Computer Science - Learning Statistics - Machine Learning |
Title | No-Regret Constrained Bayesian Optimization of Noisy and Expensive Hybrid Models using Differentiable Quantile Function Approximations |
URI | https://arxiv.org/abs/2305.03824 |
hasFullText | 1 |
inHoldings | 1 |
isFullTextHit | |
isPrint | |
link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwdV09T8MwELXaTiwIBKh86gbWQJPYTjyWj1Ih0QpUpG6VndioUmlQk6L2D_C7uXOCYGG1LQ9nvdw5vvceY5dSkyVOlAcW81vAZZjid1BzRDy3qc56Rlj63_E0ksNX_jgV0xaDHy6MXm3mn7U-sCmvsT4WV704jXibtaOIWrYextP6cdJLcTXrf9dhjemH_iSJwR7bbao76NfHsc9adnnAvkZF8GLxXlsB2WN6Uwabw43eWmIwwhhR-97QIaFwMCrm5Rbwhg-kQ-z7y2G4JWYVkHPZogRqVn-Du8bbBDFqFhae1xgkxDgMMFX5vfqkF76Z1-TE8pBNBveT22HQ2B8EWiY8ICylXrYzTbJMRCqXpCotXc9Kp4RyPDVYPmjBrUrjWOVJkiiDgFTcJpmL4iPWWRZL22VgpLOJ1iY0jgT-QmNwG5UZwV2Y4eQx6_qgzT5qhYsZxXPm43ny_9Qp2yHvdd_9J85Yp1qt7Tlm6Mpc-GP6BrXgk4Y |
link.rule.ids | 228,230,786,891 |
linkProvider | Cornell University |
openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=No-Regret+Constrained+Bayesian+Optimization+of+Noisy+and+Expensive+Hybrid+Models+using+Differentiable+Quantile+Function+Approximations&rft.au=Lu%2C+Congwen&rft.au=Paulson%2C+Joel+A&rft.date=2023-05-05&rft_id=info:doi/10.48550%2Farxiv.2305.03824&rft.externalDocID=2305_03824 |