Central to many rate-distortion theoretic results is the concept of type covering, i.e., covering of the set of vectors with the same type P by identical "distortion balls." For example, the achievability of the rate-distortion function RP (D) can be shown using the fact that it suffices to use ≈ 2 nR P (D) balls with radius D to cover a type class T n P of length-n sequences [2] . Another example is guessing [1], where X, drawn from a DMS, is guessed using a fixed sequence y(1), y(2), . . .

more »

... il d(X, y(i)) ≤ D. The optimal guessing list (minimizing expected i) is formed by sorting P in increasing RP (D), and concatenating centers of balls that cover each T n P . In the hierarchical extension introduced in [3], T n P is first covered using D1-balls, and then the portion of each D1-ball (parent) that lies within T n P is covered using D2-balls (children). We call this strong hierarchical type covering to distinguish it from the weak version introduced in [5], where the D2-balls are chosen so that for all x ∈ T n P , there exists a pair of parent D1-and child D2-balls both covering x. The main use of hierarchical type covering in both [3] and [5] was the determination of achievable error exponents in scalable source coding. In fact, weak covering is sufficient for that purpose. Strong covering, on the other hand, is necessary for hierarchical guessing [4]: In the first stage, X is guessed using a fixed y1(1), y1(2), . . . until d1(X, y1(i)) ≤ D1, and in the second stage, new fixed guesses y2(1|i), y2(2|i), . . . are used until d2(X, y2(j|i)) ≤ D2. If the type PX of X is known beforehand, the optimal strategy is to find a strong hierarchical covering of T n P X achieving a balance point in the tradeoff of the required number of D1-and D2-balls. The distinction between the two covering strategies motivated us to re-derive single-letter characterizations of the rates of D1-and D2-balls necessary and sufficient to cover T n P both weakly and strongly. Somewhat surprisingly, the two characterizations lead to different rate regions. In fact, the claimed rate region for strong covering that appeared in [3] is precisely the achievability region for weak covering 2 . Denote by I(Q1, V ) the mutual information between Y1 and X induced by as the region of all (R1, R2) such that I(Q1, V ) ≤ R1 and 2 Private communication with Tamas Linder and Prakash Narayan revealed that if covering with D1-balls is replaced by covering with Vshells, which are subsets of D1-balls, then a variant of strong covering can achieve the weak covering rates. However, this variant is not useful in hierarchical guessing as defined in [4] . Q1, D1) , and W ∈ W(V, Q 2|1 , Q1, D2). Also define TP (D1, D2) as the rate pairs (R1, R2) such that there exists Q1(y1) satisfying and for all V ∈ V(P, Q1, D1), there exists Q 2|1 (y2|y1) with The next lemma shows that TP (D1, D2) completely characterizes achievable rates for strong covering. Lemma 1: For any {y1(i)} M 1 i=1 and {y2(j|i)} M 2 j=1 that strongly covers T n P , there exists Q1(y1) such that 1 n log M1 + (n) ≥ Im(P ||Q1, D1)