Adaptive Control & Reinforcement Learning
Term: Fall 2020
Lectures: TBD Office Hours: ECOT 248, Times: TBD
Grading: The grade will be based on the following criteria:
60 % Homework: 5 HW, each counts for 12%.
40 % Final Project: Project Proposal 5% + Report 20% + Presentation 15%
Course Ad: [pdf] Syllabus: [pdf]
08/20/2020: Welcome to the course website for ECEN 5008 - Adaptive Control and Reinforcement Learning.
Course contents: Dynamic programming, Policy Evaluation, Policy Improvement, Policy Iteration, Maximum Principle for optimal control, the Hamilton-Jacobi-Bellman equation, the linear quadratic regulator, neural networks, system identification, learning dynamics, conditions of persistence of excitation, approximate dynamic programming, excitation-based online approximate optimal control, Lyapunov-based stability theory. Examples and applications in the areas of robotics, cyber-physical systems, autonomous vehicles and transportation systems. Students will develop: (a) A solid understanding of the principles behind adaptive control and neuro-adaptive reinforcement learning in continuous-time, discrete-time, continuous spaces and discrete spaces. (b) A solid understanding of the most common reinforcement learning algorithms, as well as their theoretical and practical limitations. Students will be endowed with a fundamental background that will allow them to pursue systems-centered PhD paths or explore future opportunities in the automation, optimization, and control systems marketplace.
The class has no official textbook. However, some useful references that we will use, include:
"Adaptive Control: Stability, Convergence and Robustness" by M. Bodson and S. S. Sastry.
"Adaptive Control with Applications", by Astrom and Wittenmark.
"Reinforcement Learning and Optimal Control", by Bertsekas.
"Optimal Adaptive Control and Differential Games by Reinforcement Learning Principles", by Vrabie, Vamvoudakis, Lewis.
"Nonlinear and Adaptive Control with Applications", by Alessandro Astolfi, Dimitrios Karagiannis, Romeo Ortega.