17EC54 Information Theory & Coding syllabus for TE

Information Theory:

Introduction, Measure of information, Information content of message, Average Information content of symbols in Long Independent sequences, Average Information content of symbols in Long dependent sequences, Markov Statistical Model of Information Sources, Entropy and Information rate of Markoff Sources (Section 4.1, 4.2 of Text 1). L1, L2, L3

Source Coding:

Source coding theorem, Prefix Codes, Kraft McMillan Inequality property – KMI (Section 2.2 of Text 2). Encoding of the Source Output, Shannon‘s Encoding Algorithm (Sections 4.3, 4.3.1 of Text 1).

Shannon Fano Encoding Algorithm, Huffman codes, Extended Huffman coding, Arithmetic Coding, Lempel – Ziv Algorithm (Sections 3.6, 3.7, 3.8, 3.10 of Text 3). L1, L2, L3

Information Channels:

Communication Channels ( Section 4.4 of Text 1). Channel Models, Channel Matrix, Joint probabilty Matrix, Binary Symmetric Channel, System Entropies, Mutual Information, Channel Capacity, Channel Capacity of : Binary Symmetric Channel, Binary Erasure Channel, Muroga,s Theorem, Contineuos Channels (Sections 4.2, 4.3, 4.4, 4.6, 4.7 of Text 3). L1, L2, L3

Error Control Coding:

Introduction, Examples of Error control coding, methods of Controlling Errors, Types of Errors, types of Codes, Linear Block Codes: matrix description of Linear Block Codes, Error Detection and Error Correction Capabilities of Linear Block Codes, Single Error Correcting hamming Codes, Table lookup Decoding using Standard Array.

Binary Cyclic Codes:

Algebraic Structure of Cyclic Codes, Encoding using an (n-k) Bit Shift register, Syndrome Calculation, Error Detection and Correction (Sections 9.1, 9.2, 9.3, 9.3.1, 9.3.2, 9.3.3 of Text 1). L1, L2, L3

Some Important Cyclic Codes:

Golay Codes, BCH Codes( Section 8.4 – Article 5 of Text 2).

Convolution Codes:

Convolution Encoder, Time domain approach, Transform domain approach, Code Tree, Trellis and State Diagram, The Viterbi Algorithm) (Section 8.5 – Articles 1,2 and 3, 8.6- Article 1 of Text 2). L1, L2, L3

Course Outcomes

At the end of the course the students will be able to:

• Explain concept of Dependent & Independent Source, measure of information, Entropy, Rate of Information and Order of a source
• Represent the information using Shannon Encoding, Shannon Fano, Prefix and Huffman Encoding Algorithms
• Model the continuous and discrete communication channels using input, output and joint probabilities
• Determine a codeword comprising of the check bits computed using Linear Block codes, cyclic codes & convolutional codes
• Design the encoding and decoding circuits for Linear Block codes, cyclic codes, convolutional codes, BCH and Golay codes.

Text Books:

1. Digital and analog communication systems, K. Sam Shanmugam, John Wiley India Pvt. Ltd, 1996.

2. Digital communication, Simon Haykin, John Wiley India Pvt. Ltd, 2008.

3. Information Theory and Coding, Muralidhar Kulkarni, K.S. Shivaprakasha, Wiley India Pvt. Ltd, 2015, ISBN:978-81-265-5305-1.

Reference Books:

1. ITC and Cryptography, Ranjan Bose, TMH, II edition, 2007

2. Principles of digital communication, J. Das, S. K. Mullick, P. K. Chatterjee, Wiley, 1986 - Technology & Engineering

3. Digital Communications – Fundamentals and Applications, Bernard Sklar, Second Edition, Pearson Education, 2016, ISBN: 9780134724058.

4. Information Theory and Coding, K.N.Haribhat, D.Ganesh Rao, Cengage Learning, 2017.