Shannon Entropy Calculator

How to Use the Calculator

1. Enter your values in Category probabilities or counts — one per category, separated by spaces, semicolons or new lines. You can type probabilities (e.g. 0.5; 0.25; 0.25) or raw counts (e.g. 40; 30; 20; 10); decimal commas are accepted.
2. Set Input type to "Auto-detect" (values summing to 1 are treated as probabilities), or force "Probabilities (must sum to 1)" or "Counts / frequencies (normalized)" if the automatic choice would be ambiguous.
3. Choose the Logarithm base: 2 for bits, \(e\) for nats, or 10 for dits (hartleys).
4. Read the results: the Shannon entropy H, the maximum possible entropy \(H_{max}\) for your number of categories, the efficiency ratio \(H/H_{max}\), and a per-category breakdown showing each probability, its information content and its contribution to H.

What Is Shannon Entropy?

Shannon entropy, introduced by Claude Shannon in 1948, quantifies the uncertainty of a probability distribution: how hard it is, on average, to predict which category will occur. It is the expected value of the surprisal \(\log_b(1/p_i)\). Rare events are surprising (large \(\log_b(1/p_i)\)) but happen seldom; common events happen often but are barely surprising. Entropy is the balance of the two — the probability-weighted average surprise.

The value of H ranges from 0 to \(\log_b k\). It is 0 when one outcome is certain (a single \(p_i = 1\)), and maximal when the distribution is uniform, since then no guess is better than any other. With base 2 the units are bits, and H can be read literally as the expected number of yes/no questions needed to pin down the outcome — the idea behind data compression and coding theory. With base \(e\) the units are nats, common in machine learning and physics; base 10 gives dits. Changing the base only rescales H: rankings of distributions and the ratio \(H/H_{max}\) stay the same.

The efficiency \(H/H_{max}\) (also called normalized entropy or relative entropy) tells you how close your distribution is to the most uncertain one possible. In the example above, 92.32% means the four brands split the market fairly evenly. A value near 0 means the outcome is nearly predictable. Because it is dimensionless, this ratio is the most convenient way to compare uncertainty across distributions with different numbers of categories — something raw H cannot do, since \(H_{max}\) grows with \(k\).

Assumptions, Limitations and Tips

• Complete distribution: the values must cover all categories. Probabilities must sum to 1 (tiny rounding drift is fixed automatically); counts are normalized so that they do.
• Zero probabilities: by convention \(0 \cdot \log 0 = 0\), so empty categories contribute nothing — but note that a zero-probability category still raises \(H_{max}\) by increasing \(k\), which lowers your efficiency.
• Categories, not magnitudes: entropy measures uncertainty about which category occurs, not the size of the values. It ignores category order, spacing and labels, so it is not a measure of variance — for that, see the Skewness & Kurtosis calculator.
• Sample data: with small samples, entropy estimated from counts is biased downward. The plugin-style bias correction (Miller–Madow) adds \((k-1)/(2n\ln b)\) when \(n\) is the total count — worth applying if your counts are small.
• Related measures: the same \(p \log p\) terms appear in cross-entropy and the Kullback–Leibler divergence used to compare a distribution against a reference. Entropy of a uniform spread relates directly to the Uniform Distribution, and it is routinely applied to a confusion matrix to assess classifier uncertainty. For comparing an observed distribution against an expected one, the Kruskal–Wallis test is another option.

Quick Reference: Units and Maximum Entropy

To convert between units, use \(H_{\text{bits}} = H_{\text{nats}} / \ln 2\) or \(H_{\text{bits}} = H_{\text{dits}} \cdot \log_2 10\). If you are unsure which base to pick, base 2 is the safest default — bits are the most widely reported unit.