Introduction to Machine Learning

The first part of the course will cover supervised training algorithms, establishing a general foundation through a series of extensions to linear prediction, including: nonlinear input transformations (features), L2 regularization (kernels), prediction uncertainty (Gaussian processes), L1 regularization (sparsity), nonlinear output transformations (matching losses), surrogate losses (classification), multivariate prediction, and structured prediction. Relevant optimization concepts will be acquired along the way. The second part of the course will then demonstrate how unsupervised and semi-supervised formulations follow from a relationship between forward and reverse prediction problems. This connection allows dimensionality reduction and sparse coding to be unified with regression, and clustering and vector quantization to be unified with classification - even in the context of other extensions. Current convex relaxations of such training problems will be discussed. The last part of the course covers partially-supervised learning - the problem of learning an input representation concurrently with a predictor. A brief overview of current research will be presented, including recent work on boosting and convex relaxations.