在物理学中，“熵”一词具有重要的物理意义，表示系统的“无序”程度。在数学中，使用更抽象的定义。变量 
的（香农）熵定义为
![H(X)=-sum_(x)P(x)log_2[P(x)]](/images/equations/Entropy/NumberedEquation1.svg) |
比特，其中 
是 X 处于状态 
的概率，并且如果 
，则 
定义为 0。变量 
, ..., 
的联合熵则定义为
![H(X_1,...,X_n)=-sum_(x_1)...sum_(x_n)P(x_1,...,x_n)log_2[P(x_1,...,x_n)].](/images/equations/Entropy/NumberedEquation2.svg) |
熵
另请参阅
微分熵, 信息论, 柯尔莫哥洛夫熵, 最大熵方法, 度量熵, 互信息, 纳特, 奥恩斯坦定理, 冗余, 相对熵, 拓扑熵使用 Wolfram|Alpha 探索
参考文献
Ellis, R. S. Entropy, Large Deviations, and Statistical Mechanics. New York: Springer-Verlag, 1985.Havil, J. "A Measure of Uncertainty." §14.1 in Gamma: Exploring Euler's Constant. Princeton, NJ: Princeton University Press, pp. 139-145, 2003.Khinchin, A. I. Mathematical Foundations of Information Theory. New York: Dover, 1957.Lasota, A. and Mackey, M. C. Chaos, Fractals, and Noise: Stochastic Aspects of Dynamics, 2nd ed. New York: Springer-Verlag, 1994.Ott, E. "Entropies." §4.5 in Chaos in Dynamical Systems. New York: Cambridge University Press, pp. 138-144, 1993.Rothstein, J. "Information, Measurement, and Quantum Mechanics." Science 114, 171-175, 1951.Schnakenberg, J. "Network Theory of Microscopic and Macroscopic Behavior of Master Equation Systems." Rev. Mod. Phys. 48, 571-585, 1976.Shannon, C. E. "A Mathematical Theory of Communication." The Bell System Technical J. 27, 379-423 and 623-656, July and Oct. 1948. http://cm.bell-labs.com/cm/ms/what/shannonday/shannon1948.pdf.Shannon, C. E. and Weaver, W. Mathematical Theory of Communication. Urbana, IL: University of Illinois Press, 1963.在 Wolfram|Alpha 中被引用
熵请引用为
Eric W. Weisstein "熵。" 来自 MathWorld——Wolfram Web 资源。 https://mathworld.net.cn/Entropy.html