Two-Sample Kolmogorov–Smirnov Test Calculator
Enter two independent samples and instantly compute the Kolmogorov–Smirnov statistic D and its approximate p-value to test whether both samples come from the same distribution.
Compare Two Empirical Distributions
Two-Sample Kolmogorov–Smirnov Test Formula
Where:
- \(F_1(x)\), \(F_2(x)\) — empirical cumulative distribution functions of the two samples
- \(n_1\), \(n_2\) — sample sizes
- \(n_e\) — effective sample size
- \(D\) — maximum vertical distance between the two empirical CDFs
- \(\lambda\) — scaled statistic used in the asymptotic approximation
The statistic \(D\) measures the largest gap between the two empirical distribution functions. The p-value is obtained from the asymptotic Kolmogorov distribution using Pelz–Good's finite-sample correction: for \(\lambda < 1.18\) a series in odd harmonics is used, otherwise the classical alternating series \(p = 2\sum_{k=1}^{\infty} (-1)^{k+1} e^{-2k^2\lambda^2}\).
Two-Sample Kolmogorov–Smirnov Test – Calculation Example
Example: Sample A contains the values 1 to 10 and Sample B the values 7 to 16 (the calculator's default data).
\[ F_A(x) = \frac{\#\{a_i \le x\}}{10}, \quad F_B(x) = \frac{\#\{b_i \le x\}}{10} \]
Just below 7 we have \(F_A = 6/10 = 0.6\) and \(F_B = 0\), a gap of 0.6. Checking every other point shows no larger gap, so:
\[ D = 0.6, \qquad n_e = \frac{10 \cdot 10}{10 + 10} = 5 \]
\[ \lambda = (\sqrt{5} + 0.12 + 0.11/\sqrt{5}) \cdot 0.6 \approx 1.4432 \quad \Rightarrow \quad p \approx 0.0310 \]
Result: \(D = 0.6000\), p-value \(\approx 0.0310\). Since \(p < 0.05\), the samples provide evidence against the hypothesis that they come from the same distribution.
How to use the calculator
- Paste or type your first dataset into the Sample A field. Separate the observations with spaces, semicolons or new lines; decimal commas (e.g.
4,2) are accepted. - Enter the second, independent dataset into Sample B using the same format. Each sample must contain between 2 and 5,000 numbers.
- Read the Maximum empirical distribution difference (the D statistic) and the Approximate p-value, together with the Sample A size and Sample B size as a sanity check that all values were parsed.
- Check the Interpretation block for the conclusion at the 5% significance level and any warnings about ties or small samples.
What the two-sample Kolmogorov–Smirnov test does
The two-sample Kolmogorov–Smirnov (KS) test is a nonparametric test of the null hypothesis that two independent samples were drawn from the same underlying distribution. It works by building the empirical cumulative distribution function (ECDF) of each sample — the fraction of observations at or below each value — and finding the largest vertical distance \(D\) between the two curves. If the samples really come from the same distribution, this gap should be small; a large gap suggests the distributions differ.
Unlike many classical tests, the KS test is sensitive to any kind of difference between distributions: a shift in location (like a change in the mean or median), a difference in spread, skewness, or entirely different distribution shapes. This makes it more general than, say, a Z-test for a mean, which only compares averages, or rank-based tests such as the Wilcoxon signed-rank test and the Kruskal–Wallis test, which mainly detect location shifts. The price of this generality is somewhat lower power against any one specific alternative.
The test is exact in principle, but this calculator uses an asymptotic approximation with a finite-sample correction, which is accurate to roughly three decimal places when both samples have at least 10 observations and improves as the samples grow. The D statistic itself is always computed exactly from your data.
Assumptions, limitations and practical tips
- Independence: observations must be independent both within and between samples. For paired or repeated-measures data, use a paired test such as the Wilcoxon signed-rank test instead.
- Continuous data: the p-value assumes a continuous underlying distribution. With many tied values (e.g. ratings on a 1–5 scale or rounded counts), the approximation becomes conservative or unstable; the calculator flags this. For heavily tied ordinal data, consider the Kruskal–Wallis or Friedman test.
- Sample size: with fewer than about 10 observations per sample the asymptotic p-value is only a rough guide. The calculator adds a warning when either sample is small; for tiny samples, consult exact KS tables.
- Sensitivity: the KS statistic depends on the largest single gap between the ECDFs, which typically occurs near the median. It is comparatively less sensitive to differences in the distribution tails.
- Interpreting non-significance: a p-value above 0.05 does not prove the distributions are identical — it only means the data do not provide enough evidence to distinguish them, especially in small samples. If you are planning a study and want to know whether your sample sizes can realistically detect a given difference, the statistical power calculator is a useful companion tool.
- Effect size: \(D\) itself is a descriptive measure of distributional overlap (0 = identical ECDFs, 1 = fully separated), so report it alongside the p-value rather than relying on significance alone.
Frequently Asked Questions
How do I calculate the KS statistic D by hand?
Sort both samples and combine all distinct values. At each value, compute the fraction of Sample A observations ≤ that value and the same fraction for Sample B. Subtract the two fractions, take the absolute difference, and keep the largest one you find across all values — that maximum is D. For example, if at some point 6 of 10 values in Sample A are at or below a number while only 1 of 10 in Sample B is, the gap there is \(6/10 - 1/10 = 0.5\). Check every distinct value in both samples to make sure no larger gap exists.
What is a good or typical value of D?
D always lies between 0 and 1. A value of 0 means the two empirical CDFs coincide exactly; values near 1 mean the samples barely overlap. There is no universal "good" threshold — whether D = 0.2 is meaningful depends on sample sizes. With 50 observations per sample, D ≈ 0.27 is already significant at the 5% level, while with 1,000 per sample even D = 0.06 can be. This is why D should always be reported together with the p-value, and ideally the sample sizes.
Can I use this test for one-sided hypotheses?
This calculator performs the two-sided test: it detects any difference between the distributions. If you have a directional hypothesis — for example, that values in Sample B tend to be stochastically larger than in Sample A — you need a one-sided KS test, which uses the maximum of \(F_1(x) - F_2(x)\) (or its negative) rather than the absolute value, and a different p-value formula. SciPy's ks_2samp with alternative='less' or 'greater' implements this.
When should I choose the KS test over a t-test or Mann–Whitney test?
Use the KS test when you care about any distributional difference — shape, spread, skewness, multiple modes — not just a shift in location or mean. A t-test only detects mean differences; Mann–Whitney mainly detects location shifts. If your research question is specifically "do these groups differ in average value?", those dedicated tests are usually more powerful. If the question is "do these samples come from the same distribution at all?", the KS test is the more appropriate choice.
What happens if my data contain ties or are discrete?
The classical KS p-value assumes a continuous distribution with no ties. With rounded measurements, integer counts, or Likert-scale ratings, tied values make the approximation conservative or unstable, and the calculator displays a warning. The D statistic itself remains a valid descriptive measure of the gap between the two empirical CDFs, but for formal inference on heavily tied ordinal data, rank-based alternatives such as the Kruskal–Wallis test (for 2+ groups) are typically safer.
How many observations do I need for the p-value to be reliable?
The asymptotic approximation used here is accurate to about three decimal places once both samples have at least 10 observations, and it improves as the samples grow. With fewer than 10 observations in either sample the calculator warns you, and the p-value should be treated as a rough indication only. For very small samples, exact critical values from published KS tables or software that computes the exact permutation distribution (e.g., R's ks.test default) are preferable.
Does a large p-value mean the two samples come from the same distribution?
No. A non-significant result only means the data do not provide enough evidence to distinguish the distributions — absence of evidence is not evidence of identical distributions. Small samples often fail to detect real differences, so with, say, 8 observations per group even a sizable D may yield p > 0.05. If you need to know whether your sample sizes could realistically detect a given difference before collecting data, do a power analysis first.