Math 466/564/STAT 555: Applied Random Processes (Spring 2024)

This is an advanced undergraduate/graduate course on applied stochastic processes, designed for those students who are going to need to use stochastic processes in their research but do not have the measure-theoretic background to take the Math 561-562 sequence. Measure theory is not a prerequisite for this course. However, a basic knowledge of probability theory (Math 461 or its equivalent) is expected, as well as some knowledge of linear algebra and analysis. The goal of this course is a good understanding of basic stochastic processes, in particular discrete-time and continuous-time Markov chains, and their applications. The materials covered in this course include the following:

  • Fundamentals: background on probability, linear algebra, and set theory.
  • Discrete-time Markov chains: classes, hitting times, absorption probabilities, recurrence and transience, invariant distribution, limiting distribution, reversibility, ergodic theorem, mixing times;
  • Continuous-time Markov chains: same topics as above, holding times, explosion, forward/backward Kolmogorov equations;
  • Related topics: Discrete-time martingales, potential theory, Brownian motion;
  • Applications: Queueing theory, population biology, MCMC.
This course can be tailored to the interests of the audience.


Coursego.illinois.edu/math466
Official SiteCanvas
Student hours TBA + by Appointment.

Instructor Partha Dey
Office35 CAB
ContactEmail with subject "Math 466:" (Use your official @illinois.edu address).
Class TR 9:30am-10:50am in 163 Noyes Lab.
Textbook 1. Norris: Markov Chains, Cambridge University Press, 1998;
Other Refs 2. Levin, Peres, and Wilmer: Markov Chains and Mixing Times, AMS, 2009;
3. Grimmett and Stirzaker: Probability and Random Processes, 4th Ed., OUP, 2020.
Prerequisite Math 461 (Undergraduate Probability) and MATH 447/448 (Undergraduate Analysis).
A basic knowledge of probability theory, linear algebra and analysis is expected. Measure theory is not a prerequisite for this course.
DRESTo obtain disability-related academic adjustments and/or auxiliary aids, students should contact both the instructor and the Disability Resources and Educational Services (DRES) as soon as possible. You can contact DRES at 1207 S. Oak Street, Champaign, (217) 333-1970, or via e-mail at disability@illinois.edu.
Grading Policy Homework: 40% of the course grade. Homework will be assigned weekly on Thursdays on Canvas, to be submitted before the start of next Thursday lecture in Canvas.

Solving a lot of problems is an extremely important part of learning probability. You are encouraged to work together on the homework, but I ask that you write up your own solutions and turn them in separately. Late homework will not be graded. If for some reason you've done a homework but can't turn it in online, send it via email before class. Because of this strict policy on late homework, I will drop your lowest score. Please talk to the instructor in cases of emergency.

Midterm : 25% will depend on an in-class midterm exam on Tuesday, Feb 27, 2024 in class.

Final : 35% will depend on the final exam on Tuesday, May 7, 2024 at 7-10pm.

Attendance & Participation: 3% of the course grade.
Exam PolicyMake-up exams will be given only for medical or other serious reasons. If you discover that you cannot be at an exam, please let me know as soon as possible, so that we can make other arrangements. You must work completely on your own during exams. I make my exams fair and similar to homework, so as long as you use the resources provided, you should do fine. If you have difficulties of any kind or fall behind in the course, please come talk to me as soon as possible.
Grading scaleFinal scores will be converted to letter grades beginning with the following scale:
AA-B+BB-C+CD
9390878380777060
As for a curve, these cutoffs might be adjusted, but only in the downward direction (to make letter grades higher).

Week Date Due Content






1 T Jan 16 Set theory and Probability basics.
R Jan 18 Expectation and Basics of linear algebra.






2 T Jan 23 Definition of Markov chain.
R Jan 25 Homework 0 Properties of Markov chain.






3 T Jan 30 Hitting time and stopping time.
R Feb 01 Homework 1 Strong Markov property.






4 T Feb 06 Class structure.
R Feb 08 Homework 2 Recurrence and Transience.






5 T Feb 13 Invariant distribitions.
R Feb 15 Homework 3 Positive recurrence and aperiodicity.






6 T Feb 20 Convergence to invariant distribution.
R Feb 22 Homework 4 Convergence for periodic MC.






7 T Feb 27 Midterm.
R Feb 29 No HW Time reversal and detailed balance.






8 T Mar 05 Mixing time.
R Mar 07 Homework 5 Ergodic theory and Metropolis-Hastings algorithm.






9 T Mar 12 No classes. Spring break.
R Mar 14






10 T Mar 19 Cutoff Phenomenon.
R Mar 21 Homework 6 Continuous time Markov Chain.






11 T Mar 26 Jump Chain and Holding Times.
R Mar 28 Homework 7 Poisson Process and Birth Processes.






12 T Apr 02 Explosion time and Minimal Chain.
R Apr 04 Homework 8 Class Structure, Hitting Times, Recurrence and Transience.






13 T Apr 09 Invariant measure for CTMC.
R Apr 11 Homework 9 Time reversal and convergence to equilibrium.






14 T Apr 16 Branching processes.
R Apr 18 Homework 10 Epidemics and queueing theory.






15 T Apr 23 Martingales.
R Apr 25 Homework 11 Brownian Motion.






16 T Apr 30 Homework 12 Review.






17 T May 7 Final exam.