Schedule

Week 1

Monday | 2017.8.28
- First day of class! Course overview and syllabus discussion.
- Materials: Syllabus Slides, Full Syllabus
- ISL Videos: Opening Remarks and Examples, Supervised and Unsupervised Learning
Wednesday | 2017.8.30
- Quick probability review. Recapping some R basics.
- Reading: R4SL Chapter 2, R4SL Chapter 3
- Slides: Probability Recap, R Introduction
- Lab: R Basics, R Basics Solutions
Friday | 2017.9.1
- Introduction to rmarkdown.
- Slides: rmarkdown Introduction

Week 2

Monday | 2017.9.4
- No class! Labor Day
Wednesday | 2017.9.6
- More rmarkdown details and practice. What is a model?
- Reading: R4SL Chapter 3
- Lab: rmarkdown, rmarkdown Solutions
Friday | 2017.9.8
- Begin recap of regression basics.
- Reading: ISL 3.1 - 3.4, R4SL Chapter 4
- ISL Slides: Linear Regression
- ISL Videos: Simple Linear Regression, Hypothesis Testing, Interpreting Regression Coefficients, Model Selection and Qualitative Predictors, Interactions and Nonlinearity
- Deadline: Homework 00 Due

Week 3

Monday | 2017.9.11
- Review using lm() for regression models in R.
- Reading: R4SL Chapter 4
Wednesday | 2017.9.13
- Introduce the supervised learning, regression, task. Discuss the test-train split and models that generalize to unseen data.
- Reading: ISL 2.1 - 2.2
- ISL Slides: Statistical Learning
- ISL Videos: Statistical Learning and Regression, Assessing Model Accuracy and Bias-Variance Trade-off
- Lab: Test-Train Split, Test-Train Split Solutions
Friday | 2017.9.15
- Continue discussion of regression in the context of statistical learning.
- Slides: Linear Models for Statistical Learning, Regression
- Deadline: Homework 01 Due

Week 4

Monday | 2017.9.18
- Introduce KNN. Compare non-parametric methods to parametric methods. Discuss tuning parameters versus model parameters.
- Reading: R4SL Chapter 7 (Currently very sparse notes.)
Wednesday | 2017.9.20
- Continue discussion of KNN. Compare KNN to linear models. Some live coding examples.
Friday | 2017.9.22
- Finish discussion of KNN
- Deadline: Homework 02 Due

Week 5

Monday | 2017.9.25
- Bias-Variance Tradeoff
- Reading: R4SL Chapter 8
- Slides: Bias-Variance Tradeoff
Wednesday | 2017.9.27
- Begin classification.
- Reading: ISL 4.1, R4SL Chapter 9
- Slides: Classification Introduction
Friday | 2017.9.29
- More classification. Introduction to logistic regression
- Reading: ISL 4.2 - 4.3, R4SL Chapter 10
- ISL Slides: Classification
- ISL Videos: Introduction to Classification
- Deadline: Homework 03 Due

Week 6

Monday | 2017.10.2
- Continued discussion of logistic regression.
- Reading: ISL 4.3, R4SL Chapter 10
- ISL Slides: Classification
- ISL Videos: Logistic Regression, Multiple Logistic Regression
Wednesday | 2017.10.4
- KNN for classification.
- Reading: R4SL Chapter 12
Friday | 2017.10.6
- Generative methods in R.
- Reading: ISL 4.4, R4SL Chapter 11
- Deadline: Homework 04 Due

Week 7

Monday | 2017.10.9
- Continued discussion of generative methods. Details of univariate LDA.
- Reading: ISL 4.4, R4SL Chapter 11
- ISL Slides: Classification
- ISL Videos: Linear Discriminant Analysis and Bayes Theorem, Univariate Linear Discriminant Analysis
Wednesday | 2017.10.11
- Continued discussion of generative methods. Multivariate LDA, QDA, Naive Bayes.
- Reading: ISL 4.4, R4SL Chapter 11
- ISL Videos: Multivariate Linear Discriminant Analysis, Quadratic Discriminant Analysis and Naive Bayes
Friday | 2017.10.13
- Some final thoughts on generative methods. Some recap of classification methods. Some R details.
- Deadline: Homework 05 Due

Week 8

Monday | 2017.10.16
- Review.
Wednesday | 2017.10.18
- Quiz I
Friday | 2017.10.20
- Begin discussing Statistical Learning in practice.
- Deadline: None. No homework during quiz week.

Week 9

Monday | 2017.10.23
- Cross-validation and caret.
- Reading: ISL 5.1, R4SL Chapter 20, R4SL Chapter 21
- ISL Slides: Resampling
- ISL Videos: Validation Set Approach, k-fold Cross-Validation, Cross-Validation: The Right and Wrong Ways
Wednesday | 2017.10.25
- More cross-validation and caret.
Friday | 2017.10.27
- More cross-validation and caret.
- Deadline: Homework 06 Due

Week 10

Monday | 2017.10.30
- Some comments on variable selection.
- Reading: ISL 6.1, R4SL Chapter 22
- ISL Slides: Model Selection
- ISL Videos: Best Subset Selection, Forward Stepwise Selection, Backward Stepwise Selection, Estimating Test Error I, Estimating Test Error II
Wednesday | 2017.11.1
- Entering the modern age. Introducing regularization.
- Reading: ISL 6.2, R4SL Chapter 24 - Regularization
- ISL Slides: Model Selection
- ISL Videos: Shrinkage Methods and Ridge Regression, The Lasso, Tuning Parameter Selection
Friday | 2017.11.3
- More on ridge and lasso. Using ridge and lasso in R.
- Reading: ISL 6.2, R4SL Chapter 24 - Regularization, R4SL Chapter 25 - Elastic Net
- Deadline: Homework 07 Due

Week 11

Monday | 2017.11.6
- Elastic net.
- R4SL: Elastic Net
Wednesday | 2017.11.8
- Overview: Introduction to trees.
- Reading: ISL 8.1
- ISL Slides: Trees
- Additonal Slides: Part II: Tree-based Methods
- ISL Videos: Decision Trees, Pruning a Decision Tree, Classification Trees and Comparison with Linear Models
Friday | 2017.11.10
- Discussed some finer details of R.
- Deadline: Homework 08 Due
- Deadline: Final Project Group Choice

Week 12

Monday | 2017.11.13
- Continuation of tree discussion. Introduction to ensemble methods, mostly random forests.
- Reading: ISL 8.2
- R4SL: Ensemble Methods
- ISL Slides: Trees
- Additonal Slides: Part I: Pruning, Bagging, Boosting
- ISL Videos: Bootstrap Aggregation (Bagging) and Random Forests, Boosting and Variable Importance
Wednesday | 2017.11.15
- Continuation of tree discussion. Introduction to ensemble methods, mostly boosting.
- Reading: ISL 8.2
Friday | 2017.11.17
- Extensions of random forests and boosting, in R. Some summary of supervised learning.
- Additional Slides: Supervised Learning Review
- Reading: Extremely Randomized Trees, Ranger, XGBoost [rmarkdown]
- Reading: Statistical Modeling: The Two Cultures
- Reading: Do we Need Hundreds of Classifiers to Solve Real World Classification Problems?

Week 13

Monday | 2017.11.13
- No class. Fall break.
- Deadline: Homework 09 Due
Wednesday | 2017.11.15
- No class. Fall break.
Friday | 2017.11.17
- No class. Fall break.

Week 14

Monday | 2017.11.27
- No class. Consider a group meeting.
Wednesday | 2017.11.29
- Unsupervised learning. Clustering.
- Reading: ISL 10.1 - 10.3
- R4SL: Unsupervised Learning
- ISL Slides: Unsupervised Learning
- Additional Slides: Unsupervised Learning, Part I, Clustering
- ISL Videos: Unsupervised Learning and Principal Components Analysis, Exploring Principal Components Analysis and Proportion of Variance Explained, K-means Clustering, Hierarchical Clustering
Friday | 2017.12.01
- Unsupervised learning. Clustering in R.
- Reading: ISL 10.1 - 10.3
- Deadline: Project proposals. No homework is due.

Week 15

Monday | 2017.12.4
- Unsupervised learning. PCA. Clustering again.
- Reading: ISL 10.1 - 10.3
- Additional Slides: Unsupervised Learning, Part II, PCA
Wednesday | 2017.12.6
- Quiz II
Friday | 2017.12.01
- No class. Office hours 8 - 10 at David’s office. Work on projects!

Week 16

Monday | 2017.12.11
- Discussion of graduate student project results. Thoughts on keeping up to date with data science and machine learning.
- Reading: Some Machine Learning and Data Science Resources
Wednesday | 2017.12.13
- No class. Office hours 8 - 10 at David’s office. Work on projects!
- Deadline: Homework 10 Due
Friday | 2017.12.15
- No class. Finals!

Week 17

T H U R S D A Y | 2017.12.21
- Deadline: Final Project Report
- Deadline: Final Project Peer Review

STAT 430: Basics of Statistical Learning

University of Illinois at Urbana-Champaign

Fall 2017, Dalpiaz

Schedule - Homework - Quizzes - Projects

Syllabus - Compass - R4SL

Schedule

Week 1

Week 2

Week 3

Week 4

Week 5

Week 6

Week 7

Week 8

Week 9

Week 10

Week 11

Week 12

Week 13

Week 14

Week 15

Week 16

Week 17

Homework

Homework 00

Homework 01

Homework 02

Homework 03

Homework 04

Homework 05

Homework 06

Homework 07

Homework 08

Homework 09

Homework 10

Quizzes

Quiz 1

Quiz 2

Projects

Group Final Project

Graduate Student Project