Missing Data and How to Spot It

Real datasets have holes. Blank cells, NaN, NULL, -999, and the dreaded empty string. Learning to see them is a core skill.

25 min · Reviewed 2026

Data Has Holes

In a perfect world, every row would have every column filled in. In reality, datasets are full of gaps. A survey respondent skipped a question. A sensor cut out for three seconds. A database migration dropped a field. All of this creates missing data.

Three flavors of missingness

MCAR — Missing Completely At Random: a sensor glitched. The gap has nothing to do with the value.
MAR — Missing At Random: men are less likely to answer a health survey question. The missingness depends on another column (gender) but not on the answer itself.
MNAR — Missing Not At Random: people with very high incomes refuse to report their income. The value itself causes the missingness. This is the dangerous one.

Common ways to handle missing data

Drop rows with missing values (simple but throws away data)
Fill with the mean or median of the column (imputation)
Fill with a predicted value from other columns
Flag missingness as its own feature (is_missing = true)
Leave it for the model to handle (some models tolerate NaN)

import pandas as pd
import numpy as np

df = pd.read_csv('survey.csv', na_values=['-999', 'N/A', 'unknown'])

# How much is missing in each column?
print(df.isna().sum())
print(df.isna().mean())  # as a fraction

# Fill age with the median
df['age'] = df['age'].fillna(df['age'].median())

# Flag missingness before filling
df['income_was_missing'] = df['income'].isna()
df['income'] = df['income'].fillna(df['income'].median())Detecting and handling missing data in pandas

The big idea: missing data is not just absence, it is often information. Treat every gap as a question, not an error.

End-of-lesson check

15 questions · take it digitally for instant feedback at tendril.neural-forge.io/learn/quiz/end-data-missing-data

What is the core idea behind "Missing Data and How to Spot It"?
1. Real datasets have holes. Blank cells, NaN, NULL, -999, and the dreaded empty string. Learning to see them is a core skill.
2. Write a one-paragraph README with the question, method, finding
3. Violating a website's Terms of Service and violating copyright are different leg…
4. Post-hoc fairness metrics without root-cause analysis
Which term best describes a foundational idea in "Missing Data and How to Spot It"?
1. NaN
2. missing data
3. imputation
4. MCAR
A learner studying Missing Data and How to Spot It would need to understand which concept?
1. missing data
2. imputation
3. NaN
4. MCAR
Which of these is directly relevant to Missing Data and How to Spot It?
1. missing data
2. NaN
3. MCAR
4. imputation
Which of the following is a key point about Missing Data and How to Spot It?
1. MCAR — Missing Completely At Random: a sensor glitched. The gap has nothing to do with the value.
2. MAR — Missing At Random: men are less likely to answer a health survey question.
3. MNAR — Missing Not At Random: people with very high incomes refuse to report their income.
4. Write a one-paragraph README with the question, method, finding
What is one important takeaway from studying Missing Data and How to Spot It?
1. Fill with the mean or median of the column (imputation)
2. Drop rows with missing values (simple but throws away data)
3. Fill with a predicted value from other columns
4. Flag missingness as its own feature (is_missing = true)
Which of these does NOT belong in a discussion of Missing Data and How to Spot It?
1. Write a one-paragraph README with the question, method, finding
2. Drop rows with missing values (simple but throws away data)
3. Fill with a predicted value from other columns
4. Fill with the mean or median of the column (imputation)
What is the key insight about "The many faces of missing" in the context of Missing Data and How to Spot It?
1. Write a one-paragraph README with the question, method, finding
2. Violating a website's Terms of Service and violating copyright are different leg…
3. Post-hoc fairness metrics without root-cause analysis
4. Missing data hides under many names: NaN, NULL, NA, None, empty string, -1, -999, N/A, and sometimes just a blank cell.
What is the key insight about "MNAR is sneaky" in the context of Missing Data and How to Spot It?
1. If you just fill in the average income for the missing folks, you underestimate inequality.
2. Write a one-paragraph README with the question, method, finding
3. Violating a website's Terms of Service and violating copyright are different leg…
4. Post-hoc fairness metrics without root-cause analysis
Which statement accurately describes an aspect of Missing Data and How to Spot It?
1. Write a one-paragraph README with the question, method, finding
2. In a perfect world, every row would have every column filled in. In reality, datasets are full of gaps.
3. Violating a website's Terms of Service and violating copyright are different leg…
4. Post-hoc fairness metrics without root-cause analysis
What does working with Missing Data and How to Spot It typically involve?
1. Write a one-paragraph README with the question, method, finding
2. Violating a website's Terms of Service and violating copyright are different leg…
3. The big idea: missing data is not just absence, it is often information. Treat every gap as a question, not an error.
4. Post-hoc fairness metrics without root-cause analysis
Which best describes the scope of "Missing Data and How to Spot It"?
1. It is unrelated to foundations workflows
2. It applies only to the opposite beginner tier
3. It was deprecated in 2024 and no longer relevant
4. It focuses on Real datasets have holes. Blank cells, NaN, NULL, -999, and the dreaded empty string. Learning to se
Which section heading best belongs in a lesson about Missing Data and How to Spot It?
1. Three flavors of missingness
2. Write a one-paragraph README with the question, method, finding
3. Violating a website's Terms of Service and violating copyright are different leg…
4. Post-hoc fairness metrics without root-cause analysis
Which section heading best belongs in a lesson about Missing Data and How to Spot It?
1. Write a one-paragraph README with the question, method, finding
2. Common ways to handle missing data
3. Violating a website's Terms of Service and violating copyright are different leg…
4. Post-hoc fairness metrics without root-cause analysis
Which of the following is a concept covered in Missing Data and How to Spot It?
1. NaN
2. imputation
3. missing data
4. MCAR

← Back to interactive lesson

Tendril · Builders · AI Foundations

Missing Data and How to Spot It

Real datasets have holes. Blank cells, NaN, NULL, -999, and the dreaded empty string. Learning to see them is a core skill.

25 min · Reviewed 2026

Data Has Holes

Three flavors of missingness

MCAR — Missing Completely At Random: a sensor glitched. The gap has nothing to do with the value.
MAR — Missing At Random: men are less likely to answer a health survey question. The missingness depends on another column (gender) but not on the answer itself.
MNAR — Missing Not At Random: people with very high incomes refuse to report their income. The value itself causes the missingness. This is the dangerous one.

Common ways to handle missing data

Drop rows with missing values (simple but throws away data)
Fill with the mean or median of the column (imputation)
Fill with a predicted value from other columns
Flag missingness as its own feature (is_missing = true)
Leave it for the model to handle (some models tolerate NaN)

import pandas as pd
import numpy as np

df = pd.read_csv('survey.csv', na_values=['-999', 'N/A', 'unknown'])

# How much is missing in each column?
print(df.isna().sum())
print(df.isna().mean())  # as a fraction

# Fill age with the median
df['age'] = df['age'].fillna(df['age'].median())

# Flag missingness before filling
df['income_was_missing'] = df['income'].isna()
df['income'] = df['income'].fillna(df['income'].median())Detecting and handling missing data in pandas

The big idea: missing data is not just absence, it is often information. Treat every gap as a question, not an error.

End-of-lesson check

15 questions · take it digitally for instant feedback at tendril.neural-forge.io/learn/quiz/end-data-missing-data

What is the core idea behind "Missing Data and How to Spot It"?
1. Real datasets have holes. Blank cells, NaN, NULL, -999, and the dreaded empty string. Learning to see them is a core skill.
2. Write a one-paragraph README with the question, method, finding
3. Violating a website's Terms of Service and violating copyright are different leg…
4. Post-hoc fairness metrics without root-cause analysis
Which term best describes a foundational idea in "Missing Data and How to Spot It"?
1. NaN
2. missing data
3. imputation
4. MCAR
A learner studying Missing Data and How to Spot It would need to understand which concept?
1. missing data
2. imputation
3. NaN
4. MCAR
Which of these is directly relevant to Missing Data and How to Spot It?
1. missing data
2. NaN
3. MCAR
4. imputation
Which of the following is a key point about Missing Data and How to Spot It?
1. MCAR — Missing Completely At Random: a sensor glitched. The gap has nothing to do with the value.
2. MAR — Missing At Random: men are less likely to answer a health survey question.
3. MNAR — Missing Not At Random: people with very high incomes refuse to report their income.
4. Write a one-paragraph README with the question, method, finding
What is one important takeaway from studying Missing Data and How to Spot It?
1. Fill with the mean or median of the column (imputation)
2. Drop rows with missing values (simple but throws away data)
3. Fill with a predicted value from other columns
4. Flag missingness as its own feature (is_missing = true)
Which of these does NOT belong in a discussion of Missing Data and How to Spot It?
1. Write a one-paragraph README with the question, method, finding
2. Drop rows with missing values (simple but throws away data)
3. Fill with a predicted value from other columns
4. Fill with the mean or median of the column (imputation)
What is the key insight about "The many faces of missing" in the context of Missing Data and How to Spot It?
1. Write a one-paragraph README with the question, method, finding
2. Violating a website's Terms of Service and violating copyright are different leg…
3. Post-hoc fairness metrics without root-cause analysis
4. Missing data hides under many names: NaN, NULL, NA, None, empty string, -1, -999, N/A, and sometimes just a blank cell.
What is the key insight about "MNAR is sneaky" in the context of Missing Data and How to Spot It?
1. If you just fill in the average income for the missing folks, you underestimate inequality.
2. Write a one-paragraph README with the question, method, finding
3. Violating a website's Terms of Service and violating copyright are different leg…
4. Post-hoc fairness metrics without root-cause analysis
Which statement accurately describes an aspect of Missing Data and How to Spot It?
1. Write a one-paragraph README with the question, method, finding
2. In a perfect world, every row would have every column filled in. In reality, datasets are full of gaps.
3. Violating a website's Terms of Service and violating copyright are different leg…
4. Post-hoc fairness metrics without root-cause analysis
What does working with Missing Data and How to Spot It typically involve?
1. Write a one-paragraph README with the question, method, finding
2. Violating a website's Terms of Service and violating copyright are different leg…
3. The big idea: missing data is not just absence, it is often information. Treat every gap as a question, not an error.
4. Post-hoc fairness metrics without root-cause analysis
Which best describes the scope of "Missing Data and How to Spot It"?
1. It is unrelated to foundations workflows
2. It applies only to the opposite beginner tier
3. It was deprecated in 2024 and no longer relevant
4. It focuses on Real datasets have holes. Blank cells, NaN, NULL, -999, and the dreaded empty string. Learning to se
Which section heading best belongs in a lesson about Missing Data and How to Spot It?
1. Three flavors of missingness
2. Write a one-paragraph README with the question, method, finding
3. Violating a website's Terms of Service and violating copyright are different leg…
4. Post-hoc fairness metrics without root-cause analysis
Which section heading best belongs in a lesson about Missing Data and How to Spot It?
1. Write a one-paragraph README with the question, method, finding
2. Common ways to handle missing data
3. Violating a website's Terms of Service and violating copyright are different leg…
4. Post-hoc fairness metrics without root-cause analysis
Which of the following is a concept covered in Missing Data and How to Spot It?
1. NaN
2. imputation
3. missing data
4. MCAR

← Back to interactive lesson