Approximate Dynamic Programming / Wiley Series in Probability and Statistics (PDF)

Praise for the First Edition

"Finally, a book devoted to dynamic programming and written
using the language of operations research (OR)! This beautiful book
fills a gap in the libraries of OR specialists and
practitioners."

--Computing Reviews

This new edition showcases a focus on modeling and
computation for complex classes of approximate dynamic programming
problems

Understanding approximate dynamic programming (ADP) is vital in
order to develop practical and high-quality solutions to complex
industrial problems, particularly when those problems involve
making decisions in the presence of uncertainty. Approximate
Dynamic Programming, Second Edition uniquely integrates four
distinct disciplines--Markov decision processes, mathematical
programming, simulation, and statistics--to demonstrate how to
successfully approach, model, and solve a wide range of real-life
problems using ADP.

The book continues to bridge the gap between computer science,
simulation, and operations research and now adopts the notation and
vocabulary of reinforcement learning as well as stochastic search
and simulation optimization. The author outlines the essential
algorithms that serve as a starting point in the design of
practical solutions for real problems. The three curses of
dimensionality that impact complex problems are introduced and
detailed coverage of implementation challenges is provided. The
Second Edition also features:

* A new chapter describing four fundamental classes of policies
for working with diverse stochastic optimization problems: myopic
policies, look-ahead policies, policy function approximations, and
policies based on value function approximations

* A new chapter on policy search that brings together stochastic
search and simulation optimization concepts and introduces a new
class of optimal learning strategies

* Updated coverage of the exploration exploitation problem in ADP,
now including a recently developed method for doing active learning
in the presence of a physical state, using the concept of the
knowledge gradient

* A new sequence of chapters describing statistical methods for
approximating value functions, estimating the value of a fixed
policy, and value function approximation while searching for
optimal policies

The presented coverage of ADP emphasizes models and algorithms,
focusing on related applications and computation while also
discussing the theoretical side of the topic that explores proofs
of convergence and rate of convergence. A related website features
an ongoing discussion of the evolving fields of approximation
dynamic programming and reinforcement learning, along with
additional readings, software, and datasets.

Requiring only a basic understanding of statistics and
probability, Approximate Dynamic Programming, Second Edition
is an excellent book for industrial engineering and operations
research courses at the upper-undergraduate and graduate levels. It
also serves as a valuable reference for researchers and
professionals who utilize dynamic programming, stochastic
programming, and control theory to solve problems in their everyday
work.