Approximate Dynamic Programming: Solving the Curses of Dimensionality (Hardcover)

A complete and accessible introduction to the real-world applications of approximate dynamic programming

With the growing levels of sophistication in modern-day operations, it is vital for practitioners to understand how to approach, model, and solve complex industrial problems. Approximate Dynamic Programming is a result of the author's decades of experience working in large industrial settings to develop practical and high-quality solutions to problems that involve making decisions in the presence of uncertainty. This groundbreaking book uniquely integrates four distinct disciplines—Markov design processes, mathematical programming, simulation, and statistics—to demonstrate how to successfully model and solve a wide range of real-life problems using the techniques of approximate dynamic programming (ADP). The reader is introduced to the three curses of dimensionality that impact complex problems and is also shown how the post-decision state variable allows for the use of classical algorithmic strategies from operations research to treat complex stochastic optimization problems.

Designed as an introduction and assuming no prior training in dynamic programming of any form, Approximate Dynamic Programming contains dozens of algorithms that are intended to serve as a starting point in the design of practical solutions for real problems. The book provides detailed coverage of implementation challenges including: modeling complex sequential decision processes under uncertainty, identifying robust policies, designing and estimating value function approximations, choosing effective stepsize rules, and resolving convergence issues.

With a focus on modeling and algorithms in conjunction with the language of mainstream operations research, artificial intelligence, and control theory, Approximate Dynamic Programming:

• Models complex, high-dimensional problems in a natural and practical way, which draws on years of industrial projects

• Introduces and emphasizes the power of estimating a value function around the post-decision state, allowing solution algorithms to be broken down into three fundamental steps: classical simulation, classical optimization, and classical statistics

• Presents a thorough discussion of recursive estimation, including fundamental theory and a number of issues that arise in the development of practical algorithms

• Offers a variety of methods for approximating dynamic programs that have appeared in previous literature, but that have never been presented in the coherent format of a book

Motivated by examples from modern-day operations research, Approximate Dynamic Programming is an accessible introduction to dynamic modeling and is also a valuable guide for the development of high-quality solutions to problems that exist in operations research and engineering. The clear and precise presentation of the material makes this an appropriate text for advanced undergraduate and beginning graduate courses, while also serving as a reference for researchers and practitioners. A companion Web site is available for readers, which includes additional exercises, solutions to exercises, and data sets to reinforce the book's main concepts.

一本完整且易於理解的近似動態規劃實際應用介紹

隨著現代操作水平的不斷提高，從業人員了解如何處理、建模和解決複雜的工業問題變得至關重要。《近似動態規劃》是作者在大型工業環境中工作數十年的經驗的結晶，旨在開發出在不確定性情況下做出決策的實際高質量解決方案。這本開創性的書籍獨特地整合了四個不同的學科——馬爾可夫設計過程、數學規劃、模擬和統計學——以展示如何使用近似動態規劃（ADP）的技術成功地建模和解決各種現實問題。讀者將了解到影響複雜問題的三個維度的困境，並且還將展示後決策狀態變量如何允許使用運營研究中的傳統算法策略來處理複雜的隨機優化問題。

《近似動態規劃》旨在作為一本介紹，不需要任何形式的動態規劃的先前培訓，其中包含數十種算法，旨在作為設計實際問題解決方案的起點。本書詳細介紹了實施挑戰，包括：在不確定性下建模複雜的序列決策過程，確定強健策略，設計和估計值函數近似，選擇有效的步長規則，以及解決收斂問題。

《近似動態規劃》著重於建模和算法，並結合主流運營研究、人工智能和控制理論的語言，具有以下特點：

- 以自然且實用的方式建模複雜的高維問題，借鑒了多年的工業項目經驗。
- 引入並強調在後決策狀態周圍估計值函數的能力，使解決算法可以分解為三個基本步驟：傳統模擬、傳統優化和傳統統計。
- 提供了對遞歸估計的詳細討論，包括基本理論和在實際算法開發中出現的一些問題。
- 提供了多種在先前文獻中出現的近似動態規劃方法，但從未以書籍的連貫格式呈現。

《近似動態規劃》以現代運營研究的實例為基礎，是動態建模的易於理解的入門書，也是開發運營研究和工程領域中高質量解決方案的寶貴指南。本書的清晰而精確的呈現使其成為高年級本科生和初級研究生課程的適當教材，同時也是研究人員和從業人員的參考書。讀者還可以訪問附帶的網站，該網站包含額外的練習題、練習題解答和數據集，以加強本書的主要概念。