Model Evaluation Metrics

Relax. We'll talk through this in plain words — no textbook voice.

What you'll walk away with

Understand Model Evaluation Metrics without being intimidated by them
Be able to run scikit-learn code yourself
Apply this concept in a real project right away

Let's Think About This for a Second

A Confusion Matrix is a table that breaks classification results into 4 categories (True Positive, True Negative, False Positive, False Negative) — Accuracy (correct predictions / total) alone can be misleading on a dataset with class imbalance (like the spam example from Intermediate lesson 2). There's a trade-off between Precision (of all the positive predictions, how many are really positive — prioritize this when you want fewer false alarms) and Recall (of all the actual positives, how many did you catch — prioritize this when false negatives are costly, e.g. disease detection). F1 Score is a single metric that balances both Precision and Recall.

Connecting to a Real Scenario

For a cancer detection model, you should prioritize Recall (misclassifying a patient as 'no disease' — a False Negative — can cause serious harm), while for a spam filter you might prioritize Precision (misclassifying an important email as spam — a False Positive — annoys the user) — running `from sklearn.metrics import classification_report; print(classification_report(y_test, predictions))` lets you see all the metrics at once in table format.

Confusion Matrix = a classification result table broken down into TP/TN/FP/FN
Precision = prioritize when you want fewer false alarms; Recall = prioritize when false negatives are costly
F1 Score = a single metric that balances Precision and Recall

Let's Look at It Together

python

from sklearn.metrics import confusion_matrix, classification_report

predictions = model.predict(X_test)

print(confusion_matrix(y_test, predictions))
print(classification_report(y_test, predictions))

You should see

[[85  5]
 [10 100]]

              precision    recall  f1-score
           0       0.89      0.94      0.92
           1       0.95      0.91      0.93

Try It in 5 Minutes

Evaluate a classification model (the spam example from Intermediate lesson 2) using `confusion_matrix()`/`classification_report()`, and read through the Precision/Recall/F1 values — write down which metric you should prioritize for this use case.

A Quick Word of Caution

Making a final decision based on a single look at metrics from just one test set can be statistically unreliable — using Cross-Validation (next lesson) gives you a more stable estimate.

Easy traps

Evaluating a dataset with class imbalance using Accuracy alone — with 95% not-spam data, even an 'always predict not-spam' model would get 95% accuracy, yet be useless
Following 'always maximize F1 score' as a blanket rule for the Precision/Recall trade-off without considering the use case — you need to factor in business context (the cost of false positives vs. false negatives)

Now Try It Yourself

You'll know it worked when: [[85 5] [10 100]] precision recall f1-score 0 0.89 0.94 0.92 1 0.95 0.91 0.93