1
6.441 offers an introduction to the quantitative theory of information and its applications to reliable, efficient communication systems.
FREE
This course includes
Hours of videos
638 years, 9 months
Units & Quizzes
23
Unlimited Lifetime access
Access on mobile app
Certificate of Completion
Topics include mathematical definition and properties of information, source coding theorem, lossless compression of data, optimal lossless coding, noisy communication channels, channel coding theorem, the source channel separation theorem, multiple access channels, broadcast channels, Gaussian noise, and time-varying channels.
Course Currilcum
- Introduction, entropy Unlimited
- Jensen’s inequality, data processing theorem, Fanos’s inequality Unlimited
- Different types of convergence, asymptotic equipartition property (AEP), typical set, joint typicality Unlimited
- Entropies of stochastic processes Unlimited
- Data compression, Kraft inequality, optimal codes Unlimited
- Huffman codes Unlimited
- Shannon-Fano-Elias codes, Slepian-Wolf Unlimited
- Channel capacity, binary symmetric and erasure channels Unlimited
- Maximizing capacity, Blahut-Arimoto Unlimited
- The channel coding theorem Unlimited
- Strong coding theorem, types of errors Unlimited
- Strong coding theorem, error exponents Unlimited
- Fano’s inequality and the converse to the coding theorem Unlimited
- Feedback capacity Unlimited
- Joint source channel coding Unlimited
- Differential entropy, maximizing entropy Unlimited
- Additive Gaussian noise channel Unlimited
- Gaussian channels: parallel, colored noise, inter-symbol interference Unlimited
- Gaussian channels with feedback Unlimited
- Multiple access channels Unlimited
- Broadcast channels Unlimited
- Finite state Markov channels Unlimited
- Channel side information, wide-band channels Unlimited
