Basics of Optimization Theory / Synthesis Lectures on Mathematics & Statistics (PDF)

This book presents a short introduction to the main tools of optimization methodology including linear programming, steepest descent, conjugate gradients, and the Karush-Kuhn-Tucker-John conditions. Each topic is developed in terms of a specific physical model, so that the strategy behind every step is motivated by a logical, concrete, easily visualized objective. A quick perusal of the Fibonacci search algorithm provides a simple and tantalizing first encounter with optimization theory, and a review of the max-min exposition of one-dimensional calculus prepares readers for the more sophisticated topics found later in the book. Notable features are the innovative perspectives on the simplex algorithm and Karush-Kuhn-Tucker-John conditions as well as a wealth of helpful diagrams. The author provides pointers to references for readers who would like to learn more about rigorous definitions, proofs, elegant reformulations and extensions, and case studies. However, the book is sufficiently self-contained to serve as a reliable resource for readers who wish to exploit commercially available optimization software without investing the time to develop expertise in its aspects.