Shannon's Entropy, Number of Arrangements and Conservation of Information

Francesco R. Ruggeri
2020 Zenodo  
In a previous note (1), we argued that information= ln(Pi), where Pi is the probability for the ith letter in the 26 letter English alphabet to appear in a long message, may be obtained by considering "interactions" which leave Pi unchanged as the initial message undergoes interactions until it reaches a destination. ln(Pi) then naturally appears as a conserved quantity in the interactions which ensure that Pi does not change as one moves from source to destination. Thus, each letter contains a
more » ... h letter contains a substance ln(Pi), called information, which is similar to the energy of a particle in a gas. An interaction may change one letter into another, but the overall information of a long message cannot change because otherwise Pi would change. In the literature (2), (3), there are descriptions of ln(Pi) as being associated with different arrangements of the same letters in a long message. In this note, we try to show how the conserved quantity ln(Pi) is linked to ideas of arrangements of letters.
doi:10.5281/zenodo.3745560 fatcat:52wlk6kbe5cxnceo3boy2ga3pu