{"id":67003,"date":"2026-10-13T11:23:03","date_gmt":"2026-10-13T08:23:03","guid":{"rendered":"https:\/\/wpcalc.com\/en\/?p=67003"},"modified":"2026-10-13T11:23:03","modified_gmt":"2026-10-13T08:23:03","slug":"shannon-entropy-calculator","status":"publish","type":"post","link":"https:\/\/wpcalc.com\/en\/statistics\/shannon-entropy-calculator\/","title":{"rendered":"Shannon Entropy Calculator"},"content":{"rendered":"<section class=\"not-prose calculator-box\" aria-labelledby=\"calculator-title\"><div class=\"flex items-baseline sm:items-center justify-between gap-2 sm:gap-3\"><div class=\"flex flex-col sm:flex-row sm:items-center gap-2\"><span class=\"icon icon-calculator text-primary-700! text-base! dark:text-primary-300!\" aria-hidden=\"true\"><\/span><h2 id=\"calculator-title\" class=\"text-lg font-display font-bold\">Measure the Uncertainty of a Distribution<\/h2><\/div><div class=\"calcdash-favorite group\">\n  <button type=\"button\" class=\"favorite\" id=\"favorite\" data-favorite-id=\"67003\" data-favorite-title=\"Shannon Entropy\" data-favorite-url=\"https:\/\/wpcalc.com\/en\/statistics\/shannon-entropy-calculator\/\" data-favorite-excerpt=\"Calculate the Shannon entropy of any set of probabilities or counts. Enter values per category, choose a logarithm base (bits, nats or dits), and get H, the maximum entropy and the efficiency ratio.\" aria-label=\"Add to Favorites\" data-favorite-icon=\"icon icon-wpcalc-dice\">\n    <span class=\"icon icon-shape-star-empty\" aria-hidden=\"true\"><\/span>\n  <\/button>\n  <div class=\"calcdash-favorite__tip\" aria-hidden=\"true\">\n   <span class=\"favorite-tooltip\" id=\"favorite-tooltip\"><\/span>\n  <\/div>\n  <span class=\"sr-only\" role=\"status\" id=\"favorite-status\"><\/span>\n<\/div><\/div><form action=\"https:\/\/wpcalc.com\/en\/statistics\/shannon-entropy-calculator\/\" method=\"POST\" class=\"calculator\" id=\"calculator-67003\" data-post=\"67003\"><fieldset class=\"fieldset-input is-1\"><legend class=\"sr-only\">Input Fields<\/legend><div class=\"field\" id=\"input-1\"><label for=\"field-1\">Category probabilities or counts<\/label><textarea id=\"field-1\" name=\"category_probabilities_or_counts\">40; 30; 20; 10<\/textarea><\/div><div class=\"field\" id=\"input-2\"><label for=\"field-2\">Input type<\/label><div class=\"grid grid-cols-1\"><select id=\"field-2\" name=\"input_type\"><option value=\"auto\" selected>Auto-detect<\/option><option value=\"probs\">Probabilities (must sum to 1)<\/option><option value=\"counts\">Counts \/ frequencies (normalized)<\/option><\/select><svg class=\"pointer-events-none col-start-1 row-start-1 mr-2 size-5 self-center justify-self-end text-gray-500 sm:size-4\" viewBox=\"0 0 16 16\" fill=\"currentColor\" aria-hidden=\"true\" data-slot=\"icon\">\n              <path fill-rule=\"evenodd\" d=\"M4.22 6.22a.75.75 0 0 1 1.06 0L8 8.94l2.72-2.72a.75.75 0 1 1 1.06 1.06l-3.25 3.25a.75.75 0 0 1-1.06 0L4.22 7.28a.75.75 0 0 1 0-1.06Z\" clip-rule=\"evenodd\" \/>\n            <\/svg><\/div><\/div><div class=\"field\" id=\"input-3\"><label for=\"field-3\">Logarithm base<\/label><div class=\"grid grid-cols-1\"><select id=\"field-3\" name=\"logarithm_base\"><option value=\"2\" selected>2 \u2014 bits<\/option><option value=\"10\">10 \u2014 dits (hartleys)<\/option><option value=\"e\">e \u2248 2.718 \u2014 nats<\/option><\/select><svg class=\"pointer-events-none col-start-1 row-start-1 mr-2 size-5 self-center justify-self-end text-gray-500 sm:size-4\" viewBox=\"0 0 16 16\" fill=\"currentColor\" aria-hidden=\"true\" data-slot=\"icon\">\n              <path fill-rule=\"evenodd\" d=\"M4.22 6.22a.75.75 0 0 1 1.06 0L8 8.94l2.72-2.72a.75.75 0 1 1 1.06 1.06l-3.25 3.25a.75.75 0 0 1-1.06 0L4.22 7.28a.75.75 0 0 1 0-1.06Z\" clip-rule=\"evenodd\" \/>\n            <\/svg><\/div><\/div><\/fieldset><div class=\"buttons\"><button type=\"submit\" data-text=\"Re-Calculate\" id=\"calculate-button\" data-post=\"67003\">Calculate<\/button><button type=\"reset\">Reset<\/button><\/div><div class=\"field is-checkbox hidden!\" id=\"field-auto-calc\"><input type=\"checkbox\" id=\"auto-calc\"><label for=\"auto-calc\">Calculate automatically<\/label><small>If enabled, the result will update automatically when you change any value.<\/small><\/div><div class=\"fieldset-result is-hidden\" aria-labelledby=\"result-title\" aria-live=\"polite\" role=\"region\"> <h3 class=\"result-title bg-gradient-to-r from-primary-50 to-gray-50 dark:from-primary-900 dark:to-gray-800\" id=\"result-title\"><span class=\"icon icon-s-pulse\" aria-hidden=\"true\"><\/span> Shannon Entropy Results<\/h3><div class=\"result-box\"><div class=\"field-result\" id=\"output-1\"><span class=\"field-title\"><span>Shannon entropy<\/span><span class=\"result-term\">H<\/span><\/span><span class=\"field-value\" id=\"result-1\"><\/span><button type=\"button\" class=\"copy-result\" data-tooltip=\"Copy Result\" aria-label=\"Copy Result\"><span class=\"copy-icon icon icon-document-copy\" aria-hidden=\"true\"><\/span><\/button><\/div><div class=\"field-result\" id=\"output-2\"><span class=\"field-title\"><span>Maximum possible entropy (uniform over k categories)<\/span><span class=\"result-term\">H_max<\/span><\/span><span class=\"field-value\" id=\"result-2\"><\/span><button type=\"button\" class=\"copy-result\" data-tooltip=\"Copy Result\" aria-label=\"Copy Result\"><span class=\"copy-icon icon icon-document-copy\" aria-hidden=\"true\"><\/span><\/button><\/div><div class=\"field-result\" id=\"output-3\"><span class=\"field-title\"><span>Entropy as a share of the maximum (efficiency)<\/span><span class=\"result-term\">H \/ H_max<\/span><\/span><span class=\"field-value\" id=\"result-3\"><\/span><button type=\"button\" class=\"copy-result\" data-tooltip=\"Copy Result\" aria-label=\"Copy Result\"><span class=\"copy-icon icon icon-document-copy\" aria-hidden=\"true\"><\/span><\/button><\/div><div class=\"field-result is-column\" id=\"output-4\"><span class=\"field-title\"><span>Per-category contribution<\/span><\/span><span class=\"field-value\" id=\"result-4\"><\/span><button type=\"button\" class=\"copy-result\" data-tooltip=\"Copy Result\" aria-label=\"Copy Result\"><span class=\"copy-icon icon icon-document-copy\" aria-hidden=\"true\"><\/span><\/button><\/div><div class=\"field-result is-column\" id=\"output-5\"><span class=\"field-title\"><span>How this entropy was computed<\/span><\/span><span class=\"field-value\" id=\"result-5\"><\/span><button type=\"button\" class=\"copy-result\" data-tooltip=\"Copy Result\" aria-label=\"Copy Result\"><span class=\"copy-icon icon icon-document-copy\" aria-hidden=\"true\"><\/span><\/button><\/div><\/div><div class=\"result-actions\"><button type=\"button\" class=\"copy-link\"><svg viewBox=\"0 0 20 20\" fill=\"currentColor\" aria-hidden=\"true\"><path d=\"M12.232 4.232a2.5 2.5 0 0 1 3.536 3.536l-1.225 1.224a.75.75 0 0 0 1.061 1.06l1.224-1.224a4 4 0 0 0-5.656-5.656l-3 3a4 4 0 0 0 .225 5.865.75.75 0 0 0 .977-1.138 2.5 2.5 0 0 1-.142-3.667l3-3Z\"\/><path d=\"M11.603 7.963a.75.75 0 0 0-.977 1.138 2.5 2.5 0 0 1 .142 3.667l-3 3a2.5 2.5 0 0 1-3.536-3.536l1.225-1.224a.75.75 0 0 0-1.061-1.06l-1.224 1.224a4 4 0 1 0 5.656 5.656l3-3a4 4 0 0 0-.225-5.865Z\"\/><\/svg><span class=\"copy-link__text\">Copy link to this calculation<\/span><\/button><span class=\"sr-only\" role=\"status\" id=\"copy-link-status\"><\/span><\/div><\/div><a href=\"#respond\" class=\"hidden transition-opacity duration-300 opacity-0 w-50 text-sm justify-center items-center gap-2 px-4 py-2 rounded bg-gray-200 text-gray-700 hover:bg-gray-300\" id=\"leave-comment\"><span class=\"icon icon-comments\" aria-hidden=\"true\"><\/span>Leave a Comment<\/a><\/form><\/section><section id=\"calc-reactions\" class=\"not-prose hidden my-12 bg-gradient-to-r from-primary-50 to-gray-50 border border-indigo-100 rounded-xl px-6 py-4 shadow-sm\" data-post=\"67003\" aria-live=\"polite\" aria-labelledby=\"calc-reactions-title\"><p id=\"calc-reactions-title\" class=\"text-sm text-gray-500 text-center mb-2\">How did this result make you feel?<\/p><div role=\"group\" aria-labelledby=\"calc-reactions-title\" class=\"grid grid-cols-3 sm:flex sm:flex-row sm:justify-around gap-4 sm:items-center sm:flex-wrap\"><button type=\"button\" aria-pressed=\"false\" class=\"reaction-btn flex flex-col items-center gap-1 text-sm text-gray-700 hover:text-primary-600 cursor-pointer transition-transform hover:scale-105\" data-reaction=\"like\"><span class=\"reaction-count\" data-reaction-count=\"like\">0<\/span><span class=\"reaction text-3xl\" aria-hidden=\"true\">\ud83d\ude00 <\/span><span class=\"reaction-description font-medium\">Like <\/span><\/button><button type=\"button\" aria-pressed=\"false\" class=\"reaction-btn flex flex-col items-center gap-1 text-sm text-gray-700 hover:text-primary-600 cursor-pointer transition-transform hover:scale-105\" data-reaction=\"helpful\"><span class=\"reaction-count\" data-reaction-count=\"helpful\">0<\/span><span class=\"reaction text-3xl\" aria-hidden=\"true\">\ud83d\udca1 <\/span><span class=\"reaction-description font-medium\">Helpful <\/span><\/button><button type=\"button\" aria-pressed=\"false\" class=\"reaction-btn flex flex-col items-center gap-1 text-sm text-gray-700 hover:text-primary-600 cursor-pointer transition-transform hover:scale-105\" data-reaction=\"confused\"><span class=\"reaction-count\" data-reaction-count=\"confused\">0<\/span><span class=\"reaction text-3xl\" aria-hidden=\"true\">\ud83d\ude15 <\/span><span class=\"reaction-description font-medium\">Confused <\/span><\/button><button type=\"button\" aria-pressed=\"false\" class=\"reaction-btn flex flex-col items-center gap-1 text-sm text-gray-700 hover:text-primary-600 cursor-pointer transition-transform hover:scale-105\" data-reaction=\"disappointed\"><span class=\"reaction-count\" data-reaction-count=\"disappointed\">0<\/span><span class=\"reaction text-3xl\" aria-hidden=\"true\">\ud83d\ude1e <\/span><span class=\"reaction-description font-medium\">Disappointed <\/span><\/button><button type=\"button\" aria-pressed=\"false\" class=\"reaction-btn flex flex-col items-center gap-1 text-sm text-gray-700 hover:text-primary-600 cursor-pointer transition-transform hover:scale-105\" data-reaction=\"inaccurate\"><span class=\"reaction-count\" data-reaction-count=\"inaccurate\">0<\/span><span class=\"reaction text-3xl\" aria-hidden=\"true\">\u274c <\/span><span class=\"reaction-description font-medium\">Inaccurate <\/span><\/button><\/div><div id=\"reaction-message\" class=\"hidden mt-8 rounded-md border border-primary-100 bg-white\/50 backdrop-blur-sm px-4 py-2 text-sm text-center shadow-sm\"><\/div><\/section><div class=\"calcdash-ad not-prose calcdash-ad--calc_result calcdash-ad--in-article\"><ins class=\"adsbygoogle\" style=\"display:block\" data-ad-client=\"ca-pub-1721569815777345\" data-ad-slot=\"6317458308\" data-ad-layout=\"in-article\" data-ad-format=\"fluid\"><\/ins><script>(adsbygoogle = window.adsbygoogle || []).push({});<\/script><\/div>\n<style id=\"wpcoderpro-inline-1\">.formula-box {\r\n\tborder-bottom: none !important;\r\n}<\/style><section class=\"formula-box\" aria-labelledby=\"formula-title\">\r\n\t<div id=\"formula-title\" class=\"not-prose font-display mb-4\"><span class=\"icon icon-formula\" aria-hidden=\"true\"><\/span><h2>Shannon Entropy Formula<\/h2><\/div>\r\n\t<figure class=\"not-prose formula\">\r\n\t\t<figcaption class=\"formula-title\">Formula<\/figcaption>\r\n\t\t<div class=\"text-base text-gray-800\" id=\"formula\">\n\\[ H = -\\sum_{i=1}^{k} p_i \\log_b p_i = \\sum_{i=1}^{k} p_i \\log_b \\frac{1}{p_i} \\]\n<\/div>\r\n\t<\/figure>\r\n\t<\/p>\n<p><strong>Where:<\/strong><\/p>\n<ul>\n<li>\\(p_i\\) \u2014 probability of category \\(i\\) (counts are normalized: \\(p_i = n_i \/ n\\))<\/li>\n<li>\\(k\\) \u2014 number of categories<\/li>\n<li>\\(b\\) \u2014 logarithm base: 2 gives bits, \\(e\\) gives nats, 10 gives dits (hartleys)<\/li>\n<\/ul>\n<p>The formula expresses the average surprise, or information content, of an outcome drawn from the distribution: each term weights the information \\(\\log_b(1\/p_i)\\) of a category by how often it occurs.<\/p>\n<p>\r\n<\/section>\n<style id=\"wpcoderpro-inline-459\">.calculation-box {\r\n\tborder-bottom: none !important;\r\n}<\/style><section id=\"calculation\" class=\"calculation-box\" aria-labelledby=\"calculation-title\">\r\n\t<div id=\"calculation-title\" class=\"not-prose font-display\"><span class=\"icon icon-unordered-list\" aria-hidden=\"true\"><\/span><h2>Shannon Entropy \u2013 Calculation Example<\/h2><\/div>\r\n\r\n<\/p>\n<p><strong>Example:<\/strong> a survey of 100 people asked for their favorite of four brands, with counts 40, 30, 20 and 10. The counts are normalized to probabilities \\(p = (0.4,\\ 0.3,\\ 0.2,\\ 0.1)\\), and the base is 2 (bits).<\/p>\n<p>\\[ H = -(0.4\\log_2 0.4 + 0.3\\log_2 0.3 + 0.2\\log_2 0.2 + 0.1\\log_2 0.1) \\]<\/p>\n<p>\\[ H = 0.5288 + 0.5211 + 0.4644 + 0.3322 = 1.8464 \\text{ bits} \\]<\/p>\n<p>\\[ H_{max} = \\log_2 4 = 2 \\text{ bits}, \\quad \\frac{H}{H_{max}} = \\frac{1.8464}{2} = 0.9232 \\]<\/p>\n<p><strong>Result:<\/strong> \\(H \\approx 1.8464\\) bits, which is 92.32% of the maximum \u2014 a distribution close to uniform over its four categories.<\/p>\n<p>\n\r\n<\/section>\n<h2>How to Use the Calculator<\/h2>\n<ol>\n<li>Enter your values in <strong>Category probabilities or counts<\/strong> \u2014 one per category, separated by spaces, semicolons or new lines. You can type probabilities (e.g. 0.5; 0.25; 0.25) or raw counts (e.g. 40; 30; 20; 10); decimal commas are accepted.<\/li>\n<li>Set <strong>Input type<\/strong> to \"Auto-detect\" (values summing to 1 are treated as probabilities), or force \"Probabilities (must sum to 1)\" or \"Counts \/ frequencies (normalized)\" if the automatic choice would be ambiguous.<\/li>\n<li>Choose the <strong>Logarithm base<\/strong>: 2 for bits, \\(e\\) for nats, or 10 for dits (hartleys).<\/li>\n<li>Read the results: the <strong>Shannon entropy<\/strong> H, the <strong>maximum possible entropy<\/strong> \\(H_{max}\\) for your number of categories, the <strong>efficiency<\/strong> ratio \\(H\/H_{max}\\), and a per-category breakdown showing each probability, its information content and its contribution to H.<\/li>\n<\/ol>\n<h2>What Is Shannon Entropy?<\/h2>\n<p>Shannon entropy, introduced by Claude Shannon in 1948, quantifies the uncertainty of a probability distribution: how hard it is, on average, to predict which category will occur. It is the expected value of the <em>surprisal<\/em> \\(\\log_b(1\/p_i)\\). Rare events are surprising (large \\(\\log_b(1\/p_i)\\)) but happen seldom; common events happen often but are barely surprising. Entropy is the balance of the two \u2014 the probability-weighted average surprise.<\/p>\n<p>The value of H ranges from 0 to \\(\\log_b k\\). It is 0 when one outcome is certain (a single \\(p_i = 1\\)), and maximal when the distribution is uniform, since then no guess is better than any other. With base 2 the units are bits, and H can be read literally as the expected number of yes\/no questions needed to pin down the outcome \u2014 the idea behind data compression and coding theory. With base \\(e\\) the units are nats, common in machine learning and physics; base 10 gives dits. Changing the base only rescales H: rankings of distributions and the ratio \\(H\/H_{max}\\) stay the same.<\/p>\n<p>The <strong>efficiency<\/strong> \\(H\/H_{max}\\) (also called normalized entropy or relative entropy) tells you how close your distribution is to the most uncertain one possible. In the example above, 92.32% means the four brands split the market fairly evenly. A value near 0 means the outcome is nearly predictable. Because it is dimensionless, this ratio is the most convenient way to compare uncertainty across distributions with different numbers of categories \u2014 something raw H cannot do, since \\(H_{max}\\) grows with \\(k\\).<\/p>\n<h2>Assumptions, Limitations and Tips<\/h2>\n<ul>\n<li><strong>Complete distribution:<\/strong> the values must cover all categories. Probabilities must sum to 1 (tiny rounding drift is fixed automatically); counts are normalized so that they do.<\/li>\n<li><strong>Zero probabilities:<\/strong> by convention \\(0 \\cdot \\log 0 = 0\\), so empty categories contribute nothing \u2014 but note that a zero-probability category still raises \\(H_{max}\\) by increasing \\(k\\), which lowers your efficiency.<\/li>\n<li><strong>Categories, not magnitudes:<\/strong> entropy measures uncertainty about <em>which<\/em> category occurs, not the size of the values. It ignores category order, spacing and labels, so it is not a measure of variance \u2014 for that, see the <a href=\"https:\/\/wpcalc.com\/en\/statistics\/skewness-and-kurtosis-calculator\/\">Skewness &amp; Kurtosis calculator<\/a>.<\/li>\n<li><strong>Sample data:<\/strong> with small samples, entropy estimated from counts is biased downward. The plugin-style bias correction (Miller\u2013Madow) adds \\((k-1)\/(2n\\ln b)\\) when \\(n\\) is the total count \u2014 worth applying if your counts are small.<\/li>\n<li><strong>Related measures:<\/strong> the same \\(p \\log p\\) terms appear in cross-entropy and the Kullback\u2013Leibler divergence used to compare a distribution against a reference. Entropy of a uniform spread relates directly to the <a href=\"https:\/\/wpcalc.com\/en\/statistics\/uniform-distribution-calculator\/\">Uniform Distribution<\/a>, and it is routinely applied to a <a href=\"https:\/\/wpcalc.com\/en\/statistics\/confusion-matrix-calculator\/\">confusion matrix<\/a> to assess classifier uncertainty. For comparing an observed distribution against an expected one, the <a href=\"https:\/\/wpcalc.com\/en\/statistics\/kruskal-wallis-test-calculator\/\">Kruskal\u2013Wallis test<\/a> is another option.<\/li>\n<\/ul>\n<h2>Quick Reference: Units and Maximum Entropy<\/h2>\n<table>\n<thead>\n<tr>\n<th>Base<\/th>\n<th>Unit<\/th>\n<th>\\(H_{max}\\) for \\(k\\) categories<\/th>\n<th>Typical use<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>2<\/td>\n<td>bits<\/td>\n<td>\\(\\log_2 k\\)<\/td>\n<td>Information theory, coding, computer science<\/td>\n<\/tr>\n<tr>\n<td>\\(e \\approx 2.718\\)<\/td>\n<td>nats<\/td>\n<td>\\(\\ln k\\)<\/td>\n<td>Machine learning, statistical mechanics<\/td>\n<\/tr>\n<tr>\n<td>10<\/td>\n<td>dits (hartleys)<\/td>\n<td>\\(\\log_{10} k\\)<\/td>\n<td>Classic information measures<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>To convert between units, use \\(H_{\\text{bits}} = H_{\\text{nats}} \/ \\ln 2\\) or \\(H_{\\text{bits}} = H_{\\text{dits}} \\cdot \\log_2 10\\). If you are unsure which base to pick, base 2 is the safest default \u2014 bits are the most widely reported unit.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Calculate the Shannon entropy of any set of probabilities or counts. Enter values per category, choose a logarithm base (bits, nats or dits), and get H, the maximum entropy and the efficiency ratio.<\/p>\n","protected":false},"author":3168,"featured_media":0,"comment_status":"open","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[74],"tags":[76],"class_list":["post-67003","post","type-post","status-publish","format-standard","hentry","category-statistics","tag-probability"],"acf":[],"_links":{"self":[{"href":"https:\/\/wpcalc.com\/en\/wp-json\/wp\/v2\/posts\/67003","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/wpcalc.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/wpcalc.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/wpcalc.com\/en\/wp-json\/wp\/v2\/users\/3168"}],"replies":[{"embeddable":true,"href":"https:\/\/wpcalc.com\/en\/wp-json\/wp\/v2\/comments?post=67003"}],"version-history":[{"count":1,"href":"https:\/\/wpcalc.com\/en\/wp-json\/wp\/v2\/posts\/67003\/revisions"}],"predecessor-version":[{"id":67004,"href":"https:\/\/wpcalc.com\/en\/wp-json\/wp\/v2\/posts\/67003\/revisions\/67004"}],"wp:attachment":[{"href":"https:\/\/wpcalc.com\/en\/wp-json\/wp\/v2\/media?parent=67003"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/wpcalc.com\/en\/wp-json\/wp\/v2\/categories?post=67003"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/wpcalc.com\/en\/wp-json\/wp\/v2\/tags?post=67003"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}