Shannon theory of ergodic sources and channels

Many interesting source and channel models have memory. When the temporal dependence fades away sufficiently fast (or more precisely, if the source or noise process is ergodic), the standard coding techniques developed in classical Shannon theory can be extended beyond i.i.d. memoryless cases, resulting in limiting expressions for rate--distortion and capacity. The key idea behind this extension is the ergodic decomposition of stationary processes, which was utilized earlier by Gallager for rate--distortion theory of ergodic sources and by Kim for capacity of ergodic channels with or without capacity. Such interplay between information theory and ergodic theory is expected to play an important role in problems other than point-to-point source and channel coding. Some technical background can be found in