Quantities of information

A simple information diagram showing additive and subtractive relationships among Shannon's basic quantities of information for correlated variables $X$ and $Y$. The area contained by both circles is the joint entropy $\mathrm {H} (X,Y)$. The circle on the left (red and violet) is the individual entropy $\mathrm {H} (X)$, with the red being the conditional entropy $\mathrm {H} (X|Y)$. The circle on the right (blue and violet) is $\mathrm {H} (Y)$, with the blue being $\mathrm {H} (Y|X)$. The violet is the mutual information $\operatorname {I} (X;Y)$. Positive areas may represent negative quantities.

The mathematical theory of information is based on probability theory and statistics, and measures information with several quantities of information, where quantity is a property that can be quantified by measurement.