Logistic Regression (Classification)

Let's think about it this way for a second

Regression outputs a continuous number (house price $250,000) — Classification, on the other hand, outputs a category/class (spam or not-spam, has a disease or doesn't). Despite the word 'Regression' in its name (a historical naming quirk), Logistic Regression is a classification algorithm — it uses the Sigmoid function to turn the output into a probability between 0 and 1 (you can then set a threshold, like: probability > 0.5 means Class 1, ≤ 0.5 means Class 0).

Let's connect it to a real scenario

To build the email spam classification example (from Basic lesson 1) with Logistic Regression — training with `from sklearn.linear_model import LogisticRegression; model = LogisticRegression(); model.fit(X_train, y_train)`, then `model.predict(X_test)` returns the class (0=not-spam, 1=spam), while `model.predict_proba(X_test)` returns the probability (e.g. 87% chance of being spam).

Regression = predicts a continuous value; Classification = predicts a category/class
Logistic Regression = a classification algorithm (despite the word 'regression' in its name)
predict() = returns a class label; predict_proba() = returns a probability

Let's look at it together

python

from sklearn.linear_model import LogisticRegression

model = LogisticRegression()
model.fit(X_train, y_train)

# Predicted class (0 or 1)
predictions = model.predict(X_test)

# Predicted probability for each class
probabilities = model.predict_proba(X_test)
print(probabilities[:3])
# [[0.13, 0.87], [0.92, 0.08], [0.45, 0.55]]
# column 0 = P(not spam), column 1 = P(spam)

You should see

[[0.13 0.87]
 [0.92 0.08]
 [0.45 0.55]]

5-minute try-it

Train a Logistic Regression model on a binary classification sample dataset (simulated spam/not-spam data), then compare the results of `predict()` and `predict_proba()`.

A quick word of caution

If you train Logistic Regression with default settings on an imbalanced dataset (5% spam, 95% not-spam), the model could get 95% accuracy just by predicting 'not-spam' every single time — while catching zero actual spam. Don't trust accuracy alone here (keep going with the metrics lesson in the Intermediate chapter).