Created by:

Profile Photo

Last updated:

September 25, 2023


Unlimited Duration


This course includes:

Unlimited Duration

Badge on Completion

Certificate of completion

Unlimited Duration


Large-Scale Machine Learning and Stochastic Algorithms by Leon Bottou - Machine Learning Summer School at Purdue, 2011. During the last decade, data sizes have outgrown processor speed. We are now frequently facing statistical machine learning problems for which datasets are virtually infinite. Computing time is then the bottleneck.

The first part of the lecture centers on the qualitative difference between small-scale and large-scale learning problem. Whereas small-scale learning problems are subject to the usual approximation - estimation tradeoff, large-scale learning problems are subject to a qualitatively different tradeoff involving the computational complexity of the underlying optimization algorithms in non-trivial ways. Unlikely optimization algorithm such as stochastic gradient show amazing performance for large-scale machine learning problems. The second part makes a detailed overview of stochastic learning algorithms applied to both linear and nonlinear models. In particular I would like to spend time on the use of stochastic gradient for structured learning problems and on the subtle connection between nonconvex stochastic gradient and active learning.

Course Curriculum

  • Lecture 1 – Learning with Stochastic Gradient Descent Unlimited
  • Lecture 2 – The Tradeoffs of Large Scale Learning Unlimited
  • Lecture 3 – Experiments with SGD Unlimited
  • Lecture 4 – Analysis for a Simple Case, Learning with a Single Epoch Unlimited
  • Lecture 5 – General Convergence Results Unlimited
  • Lecture 6 – SGD for Neyman-Pearson Classification Unlimited

About the instructor

5 5

Instructor Rating







Profile Photo
We are an educational and skills marketplace to accommodate the needs of skills enhancement and free equal education across the globe to the millions. We are bringing courses and trainings every single day for our users. We welcome everyone woth all ages, all background to learn. There is so much available to learn and deliver to the people.