src.md

% Machine Learning % Mort Yao % 2017-01-19

Reading:

• Christopher Bishop. Pattern Recognition and Machine Learning. (PRML)
• Yaser S. Abu-Mostafa, Malik Magdon-Ismail and Hsuan-Tien Lin. Learning from Data: A Short Course.
• Yevgeny Seldin. Machine Learning Lecture Notes.
• Su-Yun Huang, Kuang-Yao Lee and Horng-Shing Lu. Lecture Notes: Statistical and Machine Learning.

---

Learning Theory

Preliminaries: Basic probability theory, statistics, information theory, theory of computation (NP-hard problems).

Generalization Bounds

Occam Learning

PAC Learning

Vapnik-Chervonenkis (VC) Theory

Supervised Learning

Classification

Perceptron

Naive Bayes Classifier

Regression

Simple Linear Regression: Linear Least Squares

Bayesian Linear Regression

Logistic Regression

Generalized Linear Model (GLM) and LASSO

k-Nearest Neighbors ($k$-NN)

Kernel Methods

Support Vector Machine (SVM)

Decision Trees and Ensembles

Bagging

Boosting

Random Forests

Unsupervised Learning

Clustering

$k$-means and $k$-means++

Mean Shift

Expectation–Maximization (EM)

Density Estimation

Kernel Density Estimation (KDE)

Online Learning Models

Statistical Learning Model

Adversarial Model

Decomposition and Dimensionality Reduction Methods

Singular Value Decomposition (SVD)

Principal Component Analysis (PCA)

Factor Analysis

Independent Component Analysis (ICA)

Nonlinear Dimensionality Reduction (NLDR) and Manifold Learning

Anomaly Detection Methods

Structured Prediction Methods

Bayesian Network

Hidden Markov Model (HMM)

Conditional Random Field (CRF)

Hierarchical Temporal Memory (HTM)

Neural Networks and Deep Learning

Reinforcement Learning