Bootstrapping: Confidence Without a Formula

You have 200 people's salaries. You want a 95 percent confidence interval for the median. The math for median confidence intervals is ugly. Bradley Efron's 1979 insight: just pretend your sample is the population. Sample with replacement 10,000 times, compute the median each time, and use the 2.5th and 97.5th percentiles as your interval. That is the bootstrap.

A minimal bootstrap

import numpy as np

data = np.array([45, 52, 60, 61, 67, 72, 78, 85, 92, 120])

def bootstrap_median(data, n_boot=10000):
    n = len(data)
    medians = np.zeros(n_boot)
    for i in range(n_boot):
        sample = np.random.choice(data, size=n, replace=True)
        medians[i] = np.median(sample)
    return medians

boots = bootstrap_median(data)
ci_low, ci_high = np.percentile(boots, [2.5, 97.5])
print(f'Median: {np.median(data):.1f}')
print(f'95% CI: [{ci_low:.1f}, {ci_high:.1f}]')

What you can bootstrap

• Any statistic: mean, median, standard deviation, IQR, correlation, regression coefficient
• Model performance metrics: accuracy, AUC, RMSE
• The difference between two groups' statistics
• Predictions from a model (by bootstrapping the training data)

When bootstrap fails

• Extreme statistics (maximum, minimum): bootstrap systematically under-estimates
• Very skewed distributions: may need BCa correction
• Time-series: plain bootstrap destroys temporal structure (use block bootstrap)

Permutation tests: a cousin of bootstrap

A permutation test answers: how likely is the observed difference between groups if the labels were random? Shuffle the labels many times, compute the statistic each time, and see where your real statistic falls. It needs no parametric assumptions.

The big idea: when the math gets hard, simulation gets easy. Bootstrap lets you compute uncertainty for almost any statistic with a few lines of code. It is one of the quiet superpowers of modern data analysis.