Outliers: Keep Them, Remove Them, or Investigate?

You are analyzing student test scores. 98 percent of scores are between 60 and 100. One score is 3,400,000. Obviously a data-entry error. Remove it. But sometimes outliers are the real story: the one billionaire in an income dataset, the one fraudulent transaction, the one patient whose recovery changed medicine.

Three kinds of outliers

Detection methods

• IQR rule: flag anything below Q1 - 1.5*IQR or above Q3 + 1.5*IQR
• Z-score: flag anything with |z| > 3
• Isolation Forest: ML-based anomaly detection, works in high dimensions
• Domain knowledge: the best detector, often ignored

import pandas as pd
import numpy as np

df = pd.read_csv('data.csv')

# IQR method
Q1 = df['value'].quantile(0.25)
Q3 = df['value'].quantile(0.75)
IQR = Q3 - Q1
low = Q1 - 1.5 * IQR
high = Q3 + 1.5 * IQR

outliers = df[(df['value'] < low) | (df['value'] > high)]
print(f'Found {len(outliers)} outliers out of {len(df)} rows')

# Inspect before removing
print(outliers.head(20))

Decision framework

1. 1Plot your data first — outliers often jump out visually
2. 2Check whether the outlier is likely an error (units, typos)
3. 3If it is an error, either fix or remove it and document why
4. 4If it is real, ask whether your analysis goal can tolerate extreme values
5. 5Consider using robust statistics (median, MAD) instead of removing
6. 6Report sensitivity: how much do results change with/without outliers?

Robust alternatives

Instead of removing outliers, use statistics that are less sensitive to them. The median is more robust than the mean. Median Absolute Deviation (MAD) is more robust than standard deviation. Robust regression methods like Huber loss can accept outliers without being distorted by them.

The big idea: outliers are questions, not answers. Investigate each one and decide deliberately, rather than scrubbing them reflexively. Sometimes the anomaly is the discovery.