Home » Course Layouts » Free Course Layout Udemy
Information Theory. Instructor: Prof. Himanshu Tyagi, Department of Electrical Engineering, IISc Bangalore.
0
English
English [CC]
- Learn basic syntax that can apply to any language.
- Learn what is a programming language and the basic concepts for beginners.
- Understand what is Javascript in it's truest form.
- Know the basic syntax of Javascript.
- Know some hidden quirks in Javascript.
Description
This is a graduate level introductory course in Information Theory where we will introduce the mathematical notion of information and justify it by various operational meanings. This basic theory builds on probability theory and allows us to quantitatively measure the uncertainty and randomness in a random variable as well as information revealed on observing its value. We will encounter quantities such as entropy, mutual information, total variation distance, and KL divergence and explain how they play a role in important problems in communication, statistics, and computer science. Information theory was originally invented as a mathematical theory of communication, but has since found applications in many areas ranging from physics to biology. In fact, any field where people want to evaluate how much information about an unknown is revealed by a particular experiment, information theory can help. In this course, we will lay down the foundations of this fundamental field. (from nptel.ac.in)
Course content
-
- Lecture 01 – What is Information? Unlimited
- Lecture 02 – How to Model Uncertainty? Unlimited
- Lecture 03 – Basic Concepts of Probability Unlimited
- Lecture 04 – Estimates of Random Variables Unlimited
- Lecture 05 – Limit Theorems Unlimited
-
- Lecture 06A – Unit 1 Review Unlimited
- Lecture 06B – Source Model Unlimited
- Lecture 07 – Motivating Examples Unlimited
- Lecture 08 – A Compression Problem Unlimited
- Lecture 09 – Shannon Entropy Unlimited
- Lecture 10 – Random Hash Unlimited
- Lecture 11A – Unit 2 Review Unlimited
- Lecture 11B – Uncertainty and Randomness Unlimited
- Lecture 12 – Total Variation Distance Unlimited
- Lecture 13 – Generating almost Random Bits Unlimited
- Lecture 14 – Generating Samples from a Distribution using Uniform Randomness Unlimited
- Lecture 15 – Typical Sets and Entropy Unlimited
- Lecture 21A – Unit 4 Review Unlimited
- Lecture 21B – Information per Coin-Toss Unlimited
- Lecture 22 – Multiple Hypothesis Testing Unlimited
- Lecture 23 – Error Analysis of Multiple Hypothesis Testing Unlimited
- Lecture 24 – Mutual Information Unlimited
- Lecture 25 – Fano’s Inequality Unlimited
- Lecture 30A – Review So Far Unlimited
- Lecture 30B – Proof of Fano’s Inequality Unlimited
- Lecture 31 – Variational Formulae Unlimited
- Lecture 32 – Capacity as Information Radius Unlimited
- Lecture 33 – Proof of Pinsker’s Inequality Unlimited
- Lecture 34 – Continuity of Entropy Unlimited
- Lecture 41 – Variable Length Source Codes Unlimited
- Lecture 42A – Review Unlimited
- Lecture 42B – Kraft’s Inequality Unlimited
- Lecture 43 – Shannon Code Unlimited
- Lecture 44 – Huffman Code Unlimited
- Lecture 49 – Compression of Databases: A Scheme Unlimited
- Lecture 50 – Compression of Databases: A Lower Bound Unlimited
- Lecture 53 – Sphere Packing Bound for BSC Unlimited
- Lecture 54 – Random Coding Bound for BSC Unlimited
- Lecture 55 – Random Coding Bound for General Channel Unlimited
- Lecture 56 – Review Unlimited
- Lecture 57 – Converse Proof for Channel Coding Theorem Unlimited
Instructor
