K-Nearest Neighbors (KNN)

Relax. We'll talk through this in plain words — no textbook voice.

What you'll walk away with

Understand K-Nearest Neighbors (KNN), without any of the intimidation
Be able to run scikit-learn code yourself
Be able to apply this concept immediately in a real project

Let's think about it this way for a second

To classify a new data point, KNN (K-Nearest Neighbors) finds the 'K nearest neighbors' in the training data and gives the majority-vote result as the prediction (with K=5, it finds the 5 nearest points, and if most of them are Class A, the prediction is Class A too) — it's called 'Lazy Learning' because the training phase just 'memorizes' the data, and the distance calculations only happen at prediction time. The choice of K value (a small K is sensitive to noise, a large K blurs the decision boundary) affects model performance.

Let's connect it to a real scenario

Writing `from sklearn.neighbors import KNeighborsClassifier; model = KNeighborsClassifier(n_neighbors=5); model.fit(X_train, y_train)` — to predict a new data point, it finds the 5 closest points (by distance) in the training data and returns the majority class — you can try different K values (K=1, K=5, K=15) and compare how the test accuracy changes.

KNN = predicts via the majority vote of the 'K nearest neighbors'
Lazy Learning = the training phase just memorizes the data; the calculation happens at prediction time
Choosing K = a small K (sensitive to noise) vs. a large K (blurs the boundary)

Let's look at it together

python

from sklearn.neighbors import KNeighborsClassifier

for k in [1, 5, 15]:
    model = KNeighborsClassifier(n_neighbors=k)
    model.fit(X_train, y_train)
    accuracy = model.score(X_test, y_test)
    print(f"K={k}: accuracy={accuracy:.2f}")

You should see

K=1: accuracy=0.79
K=5: accuracy=0.86
K=15: accuracy=0.83

5-minute try-it

Train a KNN classifier with several different K values (1, 5, 15), compare the accuracy results, and decide which K value fits this dataset best.

A quick word of caution

KNN's prediction time gets slower as the training data size grows (since it has to calculate the distance to every single point) — for large datasets (millions of rows), using KNN directly in production can cause performance issues.

Easy traps

Using KNN without Feature Scaling (covered in the Basic chapter) — since KNN is a distance-based algorithm, features with mismatched scales can skew the distance calculation (scaling matters especially for KNN)
Setting K to an even number (K=4, K=10) — for binary classification, this risks vote ties, so odd numbers tend to be preferred

Now try it yourself

Train a KNN classifier with several different K values (1, 5, 15), compare the accuracy results, and decide which K value fits this dataset best.

You'll know it worked when: K=1: accuracy=0.79 K=5: accuracy=0.86 K=15: accuracy=0.83