|Lecture (CSI 2107):
|TuTh 5:00pm - 6:15pm P. S. Krishnaprasad
|Discussion: location TBD
|PSK Office Hours (AVW 2233):
|M 5:00pm - 8:00pm.
Special Announcements - most recent at the top
(2) Based on discussions in class today (2018-08-28), there will be 6 or 7 homework assignments (that will be collected and returned with feedback), 1 mid-term in late October and a course project. The grades will be based on mid-term performance (40%) and semester project (60%). There will be some discussions sessions - to be scheduled for thinking about problem sets.
(1) This is a course on stochastic models, problems, and methods. The focus will be on filtering and (feedback) control. Continuous time models built on differential equations
subject to noise processes (fluctuations) with continuous sample paths will be central to the course. Classic examples of this setting, such as the Kalman-Bucy filter,
and linear stochastic optimal control with quadratic cost functionals will be worked out. In the setting of discrete time stationary processes applications to prediction and
minimum variance control will be presented. This will be followed by discussion of processes with jumps (e.g. processes driven by Poisson counters).
Models with counting measurement processes will also be discussed, with motivation from biology (neuroscience) and physics.
Examples such as the Wonham filter and later developments in nonlinear filtering will be of interest.
Making sense of the continuous time models and their solutions will require understanding stochastic integrals.
For linear equations this is not a large step, but for nonlinear problems we use methods of Ito and Stratonovich leading to stochastic calculus.
We will give a self-contained treatment of stochastic calculus with a rich collection of examples.
Models extending state machines, leading to classic solvable problems of Markov Decision Processes (MDP), and
Partially Observable Markov Decision Processes (POMDP) will also be discussed. Applications of these models and solutions to planning and learning problems in
autonomous systems (robotics) will be explored. FOR MORE INFORMATION about textbooks pre-requisites etc. - see course outline.
Lecture Notes by P. S. KrishnaprasadAs supporting material for instruction and NOT as a substitute for the textbooks
Weekly Homework AssignmentsGroup effort in working out homework problems is acceptable. However everyone should submit individual homework solutions. Precise credit for any sources used (colleagues, teachers, journal articles, books, web resources etc.) should be given.
Some interesting resources on the web
Wiki on Bayesian Probability
Stanford Encyclopedia of Philosophy on Bayes' Theorem