Information Theory

Cod y Modiwl
MA35810
Teitl y Modiwl
Information Theory
Blwyddyn Academaidd
2026/2027
Semester
Semester 1
Cyd-gysylltydd y Modiwl
Dr Rolf Gohm
Rhestr Ddarllen
Gweld ar Aspire
Rhagofynion
MA26010
Anghymharus (Unrhyw Flwyddyn Acad)
MAM5820
Staff Eraill sy'n Cyfrannu
Dr Rolf Gohm

Dulliau Asesu

Math o Asesiad

Manylion Asesiad

Hyd Asesiad

Cyfran
Arholiad Semester Exam: 2 Awr 100%
Arholiad Ailsefyll Exam: 2 Awr 100%

Canlyniadau Dysgu

Wedi cwblhau'r modiwl dylai'r myfyrwyr fedru:

  1. State various concepts of information and entropy and explain the relationships between them.
  2. Achieve efficient data compression by coding procedures, guided by theoretical limits.
  3. Explain the notion of a channel as a model of information transmission.
  4. State Shannon’s main theorems about channel capacity and coding.
  5. Reproduce the main assumptions and arguments leading to these theorems
  6. Apply the theoretical results to construct and to analyse a variety of important channels.

Disgrifiad cryno

C. Shannon’s seminal paper ‘A mathematical theory of communication’ (1948) created information theory as a part of mathematics. It provides the tools for a rigorous understanding of information processing and communication. In this module we carefully develop the main concepts like entropy, data compression and coding, channels and their capacity. We explain the main theorems and results and apply them to various classes of examples.

Cynnwys

• Introduction, history, what is information, examples
• Entropy, joint and conditional entropy, relative entropy and mutual information, rules and inequalities
• Asymptotic equipartition property, typical sets and source coding
• Data compression, Kraft inequality and optimal codes, Huffman codes
• Channels and channel capacity, examples
• Shannon’s channel coding theorem, zero error codes, Hamming codes
• Source-channel coding theorem, binary case
• Information transmission guided by theory, detailed discussion of examples

Sgiliau Modiwl

Math o Sgiliau

Manylion Sgiliau
Addasrwydd a gwydnwch Good understanding of the contents requires considerable intellectual effort over an extended period of time.
Cydlynu ag erail Discussing the theory and solving problems together during the module is encouraged.
Cyfathrebu proffesiynol Discussing the theory and solving problems together during the module is encouraged.
Datrys Problemau Creadigol Problem sessions based on problem sheets to be solved independently. This is crucial to prepare for the problems in the exam.
Gallu digidol Insights are provided into the mathematical principles of digital information processing.
Meddwl beirniadol a dadansoddol Theory is developed rigorously and compared with real world situations
Myfyrdod Intuitive ideas need to be translated into mathematical reasoning.
Sgiliau Pwnc-benodol The ability to solve concrete problems in information theory is checked in the exam.
Synnwyr byd go iawn Theory is compared with real world applications.

Nodau

Mae'r modiwl hwn yn cydymffurfio a FfCChC Lefel 6