0

(

ratings

)

students

Created by:

Profile Photo

Last updated:

September 25, 2023

Duration:

Unlimited Duration

FREE

This course includes:

Unlimited Duration

Badge on Completion

Certificate of completion

Unlimited Duration

Description

Information Theory. Instructor: Prof. Himanshu Tyagi, Department of Electrical Engineering, IISc Bangalore.

This is a graduate level introductory course in Information Theory where we will introduce the mathematical notion of information and justify it by various operational meanings. This basic theory builds on probability theory and allows us to quantitatively measure the uncertainty and randomness in a random variable as well as information revealed on observing its value. We will encounter quantities such as entropy, mutual information, total variation distance, and KL divergence and explain how they play a role in important problems in communication, statistics, and computer science. Information theory was originally invented as a mathematical theory of communication, but has since found applications in many areas ranging from physics to biology. In fact, any field where people want to evaluate how much information about an unknown is revealed by a particular experiment, information theory can help. In this course, we will lay down the foundations of this fundamental field. (from nptel.ac.in)

Course Curriculum

    • Lecture 01 – What is Information? Unlimited
    • Lecture 02 – How to Model Uncertainty? Unlimited
    • Lecture 03 – Basic Concepts of Probability Unlimited
    • Lecture 04 – Estimates of Random Variables Unlimited
    • Lecture 05 – Limit Theorems Unlimited
    • Lecture 06A – Unit 1 Review Unlimited
    • Lecture 06B – Source Model Unlimited
    • Lecture 07 – Motivating Examples Unlimited
    • Lecture 08 – A Compression Problem Unlimited
    • Lecture 09 – Shannon Entropy Unlimited
    • Lecture 10 – Random Hash Unlimited
    • Lecture 11A – Unit 2 Review Unlimited
    • Lecture 11B – Uncertainty and Randomness Unlimited
    • Lecture 12 – Total Variation Distance Unlimited
    • Lecture 13 – Generating almost Random Bits Unlimited
    • Lecture 14 – Generating Samples from a Distribution using Uniform Randomness Unlimited
    • Lecture 15 – Typical Sets and Entropy Unlimited
    • Lecture 16A – Unit 3 Review Unlimited
    • Lecture 16B – Hypothesis Testing and Estimation Unlimited
    • Lecture 17 – Examples Unlimited
    • Lecture 18 – The Log-Likelihood Ratio Test Unlimited
    • Lecture 19 – Kullback-Leibler Divergence and Stein’s Lemma Unlimited
    • Lecture 20 – Properties of KL Divergence Unlimited
    • Lecture 21A – Unit 4 Review Unlimited
    • Lecture 21B – Information per Coin-Toss Unlimited
    • Lecture 22 – Multiple Hypothesis Testing Unlimited
    • Lecture 23 – Error Analysis of Multiple Hypothesis Testing Unlimited
    • Lecture 24 – Mutual Information Unlimited
    • Lecture 25 – Fano’s Inequality Unlimited
    • Lecture 26 – Measures of Information Unlimited
    • Lecture 27 – Chain Rules Unlimited
    • Lecture 28 – Shape of Measures of Information Unlimited
    • Lecture 29 – Data Processing Inequality Unlimited
    • Lecture 30A – Review So Far Unlimited
    • Lecture 30B – Proof of Fano’s Inequality Unlimited
    • Lecture 31 – Variational Formulae Unlimited
    • Lecture 32 – Capacity as Information Radius Unlimited
    • Lecture 33 – Proof of Pinsker’s Inequality Unlimited
    • Lecture 34 – Continuity of Entropy Unlimited
    • Lecture 35 – Lower Bound for Compression Unlimited
    • Lecture 36 – Lower Bound for Hypothesis Testing Unlimited
    • Lecture 37 – Review Unlimited
    • Lecture 38 – Lower Bound for Random Number Generation Unlimited
    • Lecture 39 – Strong Converse Unlimited
    • Lecture 40 – Lower Bound for Minmax Statistical Estimation Unlimited
    • Lecture 41 – Variable Length Source Codes Unlimited
    • Lecture 42A – Review Unlimited
    • Lecture 42B – Kraft’s Inequality Unlimited
    • Lecture 43 – Shannon Code Unlimited
    • Lecture 44 – Huffman Code Unlimited
    • Lecture 45 – Minmax Redundancy Unlimited
    • Lecture 46 – Type based Universal Compression Unlimited
    • Lecture 47A – Review Unlimited
    • Lecture 47B – Arithmetic Code Unlimited
    • Lecture 48 – Online Probability Assignment Unlimited
    • Lecture 49 – Compression of Databases: A Scheme Unlimited
    • Lecture 50 – Compression of Databases: A Lower Bound Unlimited
    • Lecture 51 – Repetition Code Unlimited
    • Lecture 52 – Channel Capacity Unlimited
    • Lecture 53 – Sphere Packing Bound for BSC Unlimited
    • Lecture 54 – Random Coding Bound for BSC Unlimited
    • Lecture 55 – Random Coding Bound for General Channel Unlimited
    • Lecture 56 – Review Unlimited
    • Lecture 57 – Converse Proof for Channel Coding Theorem Unlimited
    • Lecture 58 – Additive Gaussian Noise Channel Unlimited
    • Lecture 59 – Mutual Information and Differential Entropy Unlimited
    • Lecture 60 – Channel Coding Theorem for Gaussian Channel Unlimited
    • Lecture 61 – Parallel Channels and Water-Filling Unlimited

About the instructor

5 5

Instructor Rating

6

Reviews

4637

Courses

24154

Students

Profile Photo
OpenCoursa
We are an educational and skills marketplace to accommodate the needs of skills enhancement and free equal education across the globe to the millions. We are bringing courses and trainings every single day for our users. We welcome everyone woth all ages, all background to learn. There is so much available to learn and deliver to the people.