Stochastic Learning and Optimization A Sensitivity-Based Approach
"Performance optimization is vital in the design and operation of modern engineering systems, including communications, manufacturing, robotics, and logistics. Most engineering systems are too complicated to model, or the system parameters cannot be easily identified, so learning techniques hav...
Saved in:
| Main Author | |
|---|---|
| Format | eBook Book |
| Language | English |
| Published |
New York
Springer-Verlag
2007
Springer Springer US |
| Edition | 1. Aufl. |
| Subjects | |
| Online Access | Get full text |
Cover
Loading…
| Abstract | "Performance optimization is vital in the design and operation of modern engineering systems, including communications, manufacturing, robotics, and logistics. Most engineering systems are too complicated to model, or the system parameters cannot be easily identified, so learning techniques have to be applied. This is a multi-disciplinary area which has been attracting wide attention across many disciplines. Areas such as perturbation analysis (PA) in discrete event dynamic systems (DEDSs), Markov decision processes (MDPs) in operations research, reinforcement learning (RL) or neuro-dynamic programming (NDP) in computer science, identification and adaptive control (IAC) in control systems, share the common goal: to make the ""best decision"" to optimize system performance. This book provides a unified framework based on a sensitivity point of view. It also introduces new approaches and proposes new research topics within this sensitivity-based framework." |
|---|---|
| AbstractList | Performance optimization is vital in the design and operation of modern engineering systems. This book provides a unified framework based on a sensitivity point of view. It introduces new approaches and proposes new research topics. "Performance optimization is vital in the design and operation of modern engineering systems, including communications, manufacturing, robotics, and logistics. Most engineering systems are too complicated to model, or the system parameters cannot be easily identified, so learning techniques have to be applied. This is a multi-disciplinary area which has been attracting wide attention across many disciplines. Areas such as perturbation analysis (PA) in discrete event dynamic systems (DEDSs), Markov decision processes (MDPs) in operations research, reinforcement learning (RL) or neuro-dynamic programming (NDP) in computer science, identification and adaptive control (IAC) in control systems, share the common goal: to make the ""best decision"" to optimize system performance. This book provides a unified framework based on a sensitivity point of view. It also introduces new approaches and proposes new research topics within this sensitivity-based framework." |
| Author | Cao, Xi-Ren |
| Author_xml | – sequence: 1 fullname: Cao, Xi-Ren |
| BackLink | https://cir.nii.ac.jp/crid/1130282269408728064$$DView record in CiNii |
| BookMark | eNotUE1PAjEQrVGMgPwAb8QQbyvTmXbbHpXgR0LCQWO8Ne1StAq7uLse9NdbWOcwk5e8N3nvDdhJWZWBsQsO1xxATY3SGWSkVZYb0JipIzaABA9IHHeAcqXVa48NMEkMalJ4yvoIxmiUyM_YqGk-IA1KLiX02eSprYp317SxGC-Cq8tYvo1duRovd23cxl_Xxqo8Z7212zRh9H-H7OVu_jx7yBbL-8fZzSJzQhDKLMj0F2WO3hOElRdBCiGkk16Qc94J9JKbFSi1NorI65XzwRSSgzaS1pqGbNo9bnZ1MhJq66vqs7Ec7L4DmzqwYFNOe0htVVJcdYpdXX19h6a1YS8pQtnWbmPntzOiVIlMxElHLGO0RdxvzglQI-ZGgFaoIReJdtnRiphaqGzysXX1j-0gUk5o6A-z6G1B |
| ContentType | eBook Book |
| Copyright | Springer-Verlag US 2007 |
| Copyright_xml | – notice: Springer-Verlag US 2007 |
| DBID | 08O RYH |
| DEWEY | 519.23 |
| DOI | 10.1007/978-0-387-69082-7 |
| DatabaseName | ciando eBooks CiNii Complete |
| DatabaseTitleList | |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Mathematics Computer Science Applied Sciences |
| EISBN | 0387690824 9780387690827 |
| Edition | 1. Aufl. 1 |
| ExternalDocumentID | 139397 EBC337875 BA83735788 ciando236329 |
| GroupedDBID | 08O 0D6 0DA 38. 7M 7P A4I AABBV AABFA AAHDE AAUKK ACFGI ADQVG ADVHH AETDV AEZAY AGNDD AHMWK ALMA_UNASSIGNED_HOLDINGS AZZ BBABE BG CZZ IEZ JJU LZA MYL NUC NUP SAO SBO Z7X Z83 Z88 Z8R Z8W Z92 -T. 0E8 AAJYQ AATVQ ABBUY ABCYT ABMNI ACAMX ACBPT ACDTA ACDUY AEHEY AEJLV AEKFX AEVYL AHNNE ATJMZ E6I RYH TPJZQ |
| ID | FETCH-LOGICAL-a44325-e50252562bb30edb4e54445a5b43aaba42b519d077f9733b8dabe9c5108953f83 |
| ISBN | 038736787X 9780387367873 |
| IngestDate | Tue Jul 29 20:32:04 EDT 2025 Fri May 30 21:22:49 EDT 2025 Thu Jun 26 22:59:47 EDT 2025 Mon Feb 07 09:33:34 EST 2022 |
| IsPeerReviewed | false |
| IsScholarly | false |
| Keywords | Computer Informatik |
| LCCN | 2007928372 |
| LCCallNum_Ident | QA76.9.M35 |
| Language | English |
| LinkModel | OpenURL |
| MergedId | FETCHMERGED-LOGICAL-a44325-e50252562bb30edb4e54445a5b43aaba42b519d077f9733b8dabe9c5108953f83 |
| Notes | With 119 figures, 27 tables, and 212 problems Includes bibliographical references and index |
| OCLC | 209982521 |
| PQID | EBC337875 |
| PageCount | 575 |
| ParticipantIDs | springer_books_10_1007_978_0_387_69082_7 proquest_ebookcentral_EBC337875 nii_cinii_1130282269408728064 ciando_primary_ciando236329 |
| PublicationCentury | 2000 |
| PublicationDate | 2007 c2007 |
| PublicationDateYYYYMMDD | 2007-01-01 |
| PublicationDate_xml | – year: 2007 text: 2007 |
| PublicationDecade | 2000 |
| PublicationPlace | New York |
| PublicationPlace_xml | – name: New York – name: New York, NY – name: Boston, MA |
| PublicationYear | 2007 |
| Publisher | Springer-Verlag Springer Springer US |
| Publisher_xml | – name: Springer-Verlag – name: Springer – name: Springer US |
| SSID | ssj0000251550 |
| Score | 2.3905337 |
| Snippet | "Performance optimization is vital in the design and operation of modern engineering systems, including communications, manufacturing, robotics, and logistics.... Performance optimization is vital in the design and operation of modern engineering systems. This book provides a unified framework based on a sensitivity... |
| SourceID | springer proquest nii ciando |
| SourceType | Publisher |
| SubjectTerms | Artificial Intelligence Calculus of Variations and Optimal Control; Optimization Computer Science Control and Systems Theory Datenverarbeitung Discrete Mathematics in Computer Science Engineering Design Learning models (Stochastic processes) Lernendes System Mathematical optimization Modèles stochastiques d'apprentissage Optimierung Optimisation mathématique Performanz (Linguistik) Probability Theory and Stochastic Processes Stochastisches System Technisches System |
| Subtitle | A Sensitivity-Based Approach |
| TableOfContents | Performance Difference Formulas -- Performance Derivative Formulas -- Optimization -- Learning: Estimating Aggregated Potentials -- Aggregated Potentials -- Aggregated Potentials in the Event-Based Optimization -- Applications and Examples -- Manufacturing -- Service Rate Control -- General Applications -- Problems -- Constructing Sensitivity Formulas -- Motivation -- Markov Chains on the Same State Space -- Event-Based Systems -- Sample-Path Construction* -- Parameterized Systems: An Example -- Markov Chains with Different State Spaces* -- One Is a Subspace of the Other* -- A More General Case -- Summary -- Problems -- Part III Appendices: Mathematical Background -- Probability and Markov Processes -- Probability -- Markov Processes -- Problems -- Stochastic Matrices -- Canonical Form -- Eigenvalues -- The Limiting Matrix -- Problems -- Queueing Theory -- Single-Server Queues -- Queueing Networks -- Some Useful Techniques -- Problems -- Notation and Abbreviations -- References -- Index MDPs with Discounted Rewards -- The nth-Bias Optimization* -- nth-Bias Difference Formulas* -- Optimality Equations* -- Policy Iteration* -- nth-Bias Optimal Policy Spaces* -- Problems -- Sample-Path-Based Policy Iteration -- Motivation -- Convergence Properties -- Convergence of Potential Estimates -- Sample Paths with a Fixed Number of Regenerative Periods -- Sample Paths with Increasing Lengths -- ``Fast" Algorithms* -- The Algorithm That Stops in a Finite Number of Periods* -- With Stochastic Approximation* -- Problems -- Reinforcement Learning -- Stochastic Approximation -- Finding the Zeros of a Function Recursively -- Estimating Mean Values -- Temporal Difference Methods -- TD Methods for Potentials -- Q-Factors and Other Extensions -- TD Methods for Performance Derivatives -- TD Methods and Performance Optimization -- PA-Based Optimization -- Q-Learning -- Optimistic On-Line Policy Iteration -- Value Iteration -- Summary of the Learning and Optimization Methods -- Problems -- Adaptive Control Problems as MDPs -- Control Problems and MDPs -- Control Systems Modelled as MDPs -- A Comparison of the Two Approaches -- MDPs with Continuous State Spaces -- Operators on Continuous Spaces -- Potentials and Policy Iteration -- Linear Control Systems and the Riccati Equation -- The LQ Problem -- The JLQ Problem* -- On-Line Optimization and Adaptive Control -- Discretization and Estimation -- Discussion -- Problems -- Part II The Event-Based Optimization - A New Approach -- Event-Based Optimization of Markov Systems -- An Overview -- Summary of Previous Chapters -- An Overview of the Event-Based Approach -- Events Associated with Markov Chains -- The Event and Event Space -- The Probabilities of Events -- The Basic Ideas Illustrated by Examples -- Classification of Three Types of Events -- Event-Based Optimization -- The Problem Formulation Intro -- Preface -- Contents -- Introduction -- An Overview of Learning and Optimization -- Problem Description -- Optimal Policies -- Fundamental Limitations of Learning and Optimization -- A Sensitivity-Based View of Learning and Optimization -- Problem Formulations in Different Disciplines -- Perturbation Analysis (PA) -- Markov Decision Processes (MDPs) -- Reinforcement Learning (RL) -- Identification and Adaptive Control (I& -- AC) -- Event-Based Optimization and Potential Aggregation -- A Map of the Learning and Optimization World -- Terminology and Notation -- Problems -- Part I Four Disciplines in Learning and Optimization -- Perturbation Analysis -- Perturbation Analysis of Markov Chains -- Constructing a Perturbed Sample Path -- Perturbation Realization Factors and Performance Potentials -- Performance Derivative Formulas -- Gradients with Discounted Reward Criteria -- Higher-Order Derivatives and the MacLaurin Series -- Performance Sensitivities of Markov Processes -- Performance Sensitivities of Semi-Markov Processes* -- Fundamentals for Semi-Markov Processes* -- Performance Sensitivity Formulas* -- Perturbation Analysis of Queueing Systems -- Constructing a Perturbed Sample Path -- Perturbation Realization -- Performance Derivatives -- Remarks on Theoretical Issues* -- Other Methods* -- Problems -- Learning and Optimization with Perturbation Analysis -- The Potentials -- Numerical Methods -- Learning Potentials from Sample Paths -- Coupling* -- Performance Derivatives -- Estimating through Potentials -- Learning Directly -- Optimization with PA -- Gradient Methods and Stochastic Approximation -- Optimization with Long Sample Paths -- Applications -- Problems -- Markov Decision Processes -- Ergodic Chains -- Policy Iteration -- Bias Optimality -- MDPs with Discounted Rewards -- Multi-Chains -- Policy Iteration -- Bias Optimality |
| Title | Stochastic Learning and Optimization |
| URI | http://ebooks.ciando.com/book/index.cfm/bok_id/236329 https://cir.nii.ac.jp/crid/1130282269408728064 https://ebookcentral.proquest.com/lib/[SITE_ID]/detail.action?docID=337875 http://link.springer.com/10.1007/978-0-387-69082-7 |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwnV1BT9swFH4a5cIusMK0Dhg-7IA0eUpiO052g6oIVbDDxFBvlh07Wg-009pd-PV7TuyElklou1iJFcWJv-S95_f8vQfwMXGJdjazNJX4N_G0qmkhdEa5qExV1zqVTY2l26_59Xc-nYlZXzmmYZeszefq8a-8kv9BFfsQV8-S_Qdku5tiBx4jvtgiwthuGb_daeBvrJfVD-1TLMeyDy3TcIkC4CEwKz-1POaV36HeloigXmXZLo14H35o3KWzOf0WeGHRCyC3vADRC7ixOvSRaYbKqK0V8kxW9tsj2gy7ua9-TmWvGLrtepcXuIz1aXGKHdiREoXH7sVkenPfObP8QgWXOiEa3owZsxt1zxBDyhtZfcOYqP69D8cuUcMv5vMNa38rQN3o_bsDGHguyBt45RZD2I8VMEgQiEN4fdtlvV0dwriHhURYCI5HnsJCvhBNnoFCIihHcH81uRtf01CkgmrOWSaoE_j2aDhmxrDEWcOd4JwLLQxnWhvNM4NWsk2krEvJmCmsNq6sUBYWpWB1wd7CYLFcuHdA0LStrBZa1lXKjacQo_no3RelqY21bATH7TSpn20qEtWeZixnWTmCU5w77PJt6gPSaPzlJU8KX4Is5yM4i7Oqmkh82P6rJpdjxhAiMYLzONnKX7BSMak1IqYShYipBjEl378w2DHs9Z_pCQzWv367UzTf1uZD-Hj-AKmVOI8 |
| linkProvider | Library Specific Holdings |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=book&rft.title=Stochastic+learning+and+optimization+%3A+a+sensitivity-based+approach&rft.au=Cao%2C+Xi-Ren&rft.date=2007-01-01&rft.pub=Springer&rft.isbn=9780387367873&rft_id=info:doi/10.1007%2F978-0-387-69082-7&rft.externalDocID=BA83735788 |
| thumbnail_s | http://utb.summon.serialssolutions.com/2.0.0/image/custom?url=https%3A%2F%2Fmedia.springernature.com%2Fw306%2Fspringer-static%2Fcover-hires%2Fbook%2F978-0-387-69082-7 |