我最近在大学开设了一门数据压缩类(class)。然而，我发现“熵”一词在计算机科学中的使用相当模糊。据我所知，它大致可以翻译为系统或结构的“随机性”。
计算机科学“熵”的正确定义是什么？
最佳答案
熵可以表示不同的含义:
In computing, entropy is the randomness collected by an operating system or application for use in cryptography or other uses that require random data. This randomness is often collected from hardware sources, either pre-existing ones such as mouse movements or specially provided randomness generators.
In information theory, entropy is a measure of the uncertainty associated with a random variable. The term by itself in this context usually refers to the Shannon entropy, which quantifies, in the sense of an expected value, the information contained in a message, usually in units such as bits. Equivalently, the Shannon entropy is a measure of the average information content one is missing when one does not know the value of the random variable
数据压缩中的熵
数据压缩中的熵可能表示您输入压缩算法的数据的随机性。熵越大，压缩比越小。这意味着文本越随机，可以压缩的程度就越小。
Shannon's entropy represents an absolute limit on the best possible lossless compression of any communication: treating messages to be encoded as a sequence of independent and identically-distributed random variables, Shannon's source coding theorem shows that, in the limit, the average length of the shortest possible representation to encode the messages in a given alphabet is their entropy divided by the logarithm of the number of symbols in the target alphabet.
关于computer-science - 熵的计算机科学定义是什么？，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/510412/