The Renyi redundancy of generalized Huffman codes

A.C. Blumer, R.J. McEliece
<span title="">1988</span> <i title="Institute of Electrical and Electronics Engineers (IEEE)"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/niovmjummbcwdg4qshgzykkpfu" style="color: black;">IEEE Transactions on Information Theory</a> </i> &nbsp;
If optimality is measured by average codeword length, Huffman's algorithm gives optimal codes, and the redundancy can be measured as the difference between the average codeword length and Shannon's entropy. If the objective function is replaced by an exponentially weighted average, then a simple modification of Huffman's algorithm gives optimal codes. The redundancy can now be measured as the difference between this new average and Renyi's generalization of Shannon's entropy. By decreasing some
more &raquo; ... of the codeword lengths in a Shannon code, the upper bound on the redundancy given in the standard proof of the noiseless source coding theorem is improved. The lower bound is improved by randomizing between codeword lengths, allowing linear programming techniques to be used on an integer programming problem. These bounds are shown to be asymptotically equal, providing a new proof of Kricevski's results on the redundancy of Huffman codes. These results are generalized to the Renyi case and are related to Gallager's bound on the redundancy of Huffman codes. N 1961, Renyi [12] proposed that the Shannon entropy could be generalized to PREVIOUS WORK I which approaches the Shannon entropy ass~ o+. In 1965, Campbell [1] showed that just as the Shannon entropy is a lower bound on the average codeword length of a uniquely decodable code, the Renyi entropy is a lower bound on the exponentially weighted average codeword length ~log ( i~l P;2sl;)' s > 0. Also, lim -log L p) 51 ' = L P;l;· Manuscript received March 18, 1986.
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1109/18.21251">doi:10.1109/18.21251</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/dc52iecranguvm5c23yjqr3kbi">fatcat:dc52iecranguvm5c23yjqr3kbi</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20170706025733/http://authors.library.caltech.edu/31759/1/BLUieeetit88.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/44/80/4480c1952c1f8b7b6b6c34ed873b4fea10c60b3e.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1109/18.21251"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="external alternate icon"></i> ieee.com </button> </a>