Class 9 Modelling and Evaluation

A model turns inputs into predictions: by a decision tree, by nearest neighbours, or by learning. Evaluation counts its right and wrong predictions.

Key ideas

A decision tree asks its questions from the top; the first answer that leads to a leaf decides.

Nearest neighbours compares squared distances; with k = 3, the new point takes the label most of its 3 nearest neighbours have.

Supervised learning uses labelled examples, unsupervised learning finds groups with no labels, and reinforcement learning learns from rewards.

Each prediction is True or False (was it right?) and Positive or Negative (what did it predict?).

A false negative misses a real positive; a false positive raises a false alarm.

Accuracy is the share of all predictions that are right.

Remember

Squared distance

(x₁ − x₂)² + (y₁ − y₂)²: (2, 7) to (5, 3) is 9 + 16 = 25

True positive (TP)

actually positive, predicted positive

False positive (FP)

actually negative, predicted positive

True negative (TN); false negative (FN)

actually negative, predicted negative; actually positive, predicted negative

Accuracy

(TP + TN) ÷ total × 100%

Wrong predictions

FP + FN

Worked example

A model predicts whether an email is spam (positive = spam). Its results:
1. actual spam, predicted spam
2. actual not spam, predicted spam
3. actual spam, predicted spam
4. actual spam, predicted spam
5. actual not spam, predicted not spam
6. actual not spam, predicted not spam
7. actual spam, predicted not spam
8. actual spam, predicted not spam
9. actual spam, predicted spam
10. actual not spam, predicted not spam
11. actual not spam, predicted spam
12. actual spam, predicted spam
Count the (a) true positives (b) false positives (c) true negatives (d) false negatives.

Working: right and positive = TP; wrong and predicted positive = FP; right and negative = TN; wrong and predicted negative = FN

Answer: (a) 5; (b) 2; (c) 3; (d) 2

Modelling and Evaluation

Key ideas

Remember

Watch out

Worked example