0
(
ratings
)
students
Created by:
Last updated:
Duration:
This course includes:
Unlimited Duration
Badge on Completion
Certificate of completion
Unlimited Duration
Description
Information Theory. Instructor: Prof. Himanshu Tyagi, Department of Electrical Engineering, IISc Bangalore.
This is a graduate level introductory course in Information Theory where we will introduce the mathematical notion of information and justify it by various operational meanings. This basic theory builds on probability theory and allows us to quantitatively measure the uncertainty and randomness in a random variable as well as information revealed on observing its value. We will encounter quantities such as entropy, mutual information, total variation distance, and KL divergence and explain how they play a role in important problems in communication, statistics, and computer science. Information theory was originally invented as a mathematical theory of communication, but has since found applications in many areas ranging from physics to biology. In fact, any field where people want to evaluate how much information about an unknown is revealed by a particular experiment, information theory can help. In this course, we will lay down the foundations of this fundamental field. (from nptel.ac.in)
Course Curriculum

 Lecture 01 – What is Information? Unlimited
 Lecture 02 – How to Model Uncertainty? Unlimited
 Lecture 03 – Basic Concepts of Probability Unlimited
 Lecture 04 – Estimates of Random Variables Unlimited
 Lecture 05 – Limit Theorems Unlimited

 Lecture 06A – Unit 1 Review Unlimited
 Lecture 06B – Source Model Unlimited
 Lecture 07 – Motivating Examples Unlimited
 Lecture 08 – A Compression Problem Unlimited
 Lecture 09 – Shannon Entropy Unlimited
 Lecture 10 – Random Hash Unlimited

 Lecture 11A – Unit 2 Review Unlimited
 Lecture 11B – Uncertainty and Randomness Unlimited
 Lecture 12 – Total Variation Distance Unlimited
 Lecture 13 – Generating almost Random Bits Unlimited
 Lecture 14 – Generating Samples from a Distribution using Uniform Randomness Unlimited
 Lecture 15 – Typical Sets and Entropy Unlimited

 Lecture 16A – Unit 3 Review Unlimited
 Lecture 16B – Hypothesis Testing and Estimation Unlimited
 Lecture 17 – Examples Unlimited
 Lecture 18 – The LogLikelihood Ratio Test Unlimited
 Lecture 19 – KullbackLeibler Divergence and Stein’s Lemma Unlimited
 Lecture 20 – Properties of KL Divergence Unlimited

 Lecture 21A – Unit 4 Review Unlimited
 Lecture 21B – Information per CoinToss Unlimited
 Lecture 22 – Multiple Hypothesis Testing Unlimited
 Lecture 23 – Error Analysis of Multiple Hypothesis Testing Unlimited
 Lecture 24 – Mutual Information Unlimited
 Lecture 25 – Fano’s Inequality Unlimited

 Lecture 26 – Measures of Information Unlimited
 Lecture 27 – Chain Rules Unlimited
 Lecture 28 – Shape of Measures of Information Unlimited
 Lecture 29 – Data Processing Inequality Unlimited

 Lecture 30A – Review So Far Unlimited
 Lecture 30B – Proof of Fano’s Inequality Unlimited
 Lecture 31 – Variational Formulae Unlimited
 Lecture 32 – Capacity as Information Radius Unlimited
 Lecture 33 – Proof of Pinsker’s Inequality Unlimited
 Lecture 34 – Continuity of Entropy Unlimited

 Lecture 35 – Lower Bound for Compression Unlimited
 Lecture 36 – Lower Bound for Hypothesis Testing Unlimited
 Lecture 37 – Review Unlimited
 Lecture 38 – Lower Bound for Random Number Generation Unlimited
 Lecture 39 – Strong Converse Unlimited
 Lecture 40 – Lower Bound for Minmax Statistical Estimation Unlimited

 Lecture 41 – Variable Length Source Codes Unlimited
 Lecture 42A – Review Unlimited
 Lecture 42B – Kraft’s Inequality Unlimited
 Lecture 43 – Shannon Code Unlimited
 Lecture 44 – Huffman Code Unlimited

 Lecture 45 – Minmax Redundancy Unlimited
 Lecture 46 – Type based Universal Compression Unlimited
 Lecture 47A – Review Unlimited
 Lecture 47B – Arithmetic Code Unlimited
 Lecture 48 – Online Probability Assignment Unlimited

 Lecture 49 – Compression of Databases: A Scheme Unlimited
 Lecture 50 – Compression of Databases: A Lower Bound Unlimited

 Lecture 51 – Repetition Code Unlimited
 Lecture 52 – Channel Capacity Unlimited

 Lecture 53 – Sphere Packing Bound for BSC Unlimited
 Lecture 54 – Random Coding Bound for BSC Unlimited
 Lecture 55 – Random Coding Bound for General Channel Unlimited
 Lecture 56 – Review Unlimited
 Lecture 57 – Converse Proof for Channel Coding Theorem Unlimited

 Lecture 58 – Additive Gaussian Noise Channel Unlimited
 Lecture 59 – Mutual Information and Differential Entropy Unlimited
 Lecture 60 – Channel Coding Theorem for Gaussian Channel Unlimited
 Lecture 61 – Parallel Channels and WaterFilling Unlimited
About the instructor
Instructor Rating
Reviews
Courses
Students