Allied Topics in Control Systems - Stochastic Control copy 1

Objectives:

(1) Develop understanding of stochastic modeling in control systems.
(2) Develop familiarity with stochastic calculus, filtering techniques and optimal feedback control.



Course Contents:

1. Recap of probability and linear system theory.
2. Stochastic processes -- Poisson counters and Brownian motion.
3. Stochastic differential equations -- Ito and Stratonovich integrals, Fokker-Plank equation, Langevin equation, Ornstein–Uhlenbeck process.
4. Filtering – Riccati equations, Kalman filter in continuous and discrete time, Extended Kalman filter
5. Stochastic optimal control -- Dynamic programming, Hamilton–Jacobi–Bellman (HJB) equation, Linear quadratic Gaussian control, Linear exponential Gaussian control.



Text Books:

1. Introduction to Stochastic Control Theory by Karl Johan Astrom (Dover edition, 2006) 
2. Notes on Stochastic Control by Roger W. Brockett (2009) (available online)
3. J. L. Speyer and W. H. Chung, Stochastic Processes, Estimation and Control, SIAM, 2008.
4. Dynamic Programming and Optimal Control vol. I by Dimitri Bertsekas (Athena 2017)



Reference Books:

1. P. R. Kumar and P. Varaiya, Stochastic Systems: Estimation, Identification, and Adaptive Control, Prentice Hall, 1986.
2. D. Roy and G.V.Rao, Stochastic Dynamics, Filtering and Optimization, Cambridge University Press, 2016.
3. B. Oeksendal, Stochastic differential equations: an introduction with applications, Springer, 2003.
4. W. H. Fleming and R.W. Rishel, Deterministic and Stochastic Optimal Control, Springer, 1975.