Support Vector Machines (SVM)

Relax. We'll talk through this in plain words — no textbook voice.

What you'll walk away with

Understand Support Vector Machines (SVM) without being intimidated by them
Be able to run scikit-learn code yourself
Apply this concept in a real project right away

Let's Think About This for a Second

SVM's core idea is that when finding the boundary (decision boundary) that separates two classes, it searches for the boundary that keeps both classes 'as far away' from it as possible (maximum margin) — the data points closest to the boundary are called 'Support Vectors' (they get this name because they're the points that determine the boundary). The Kernel Trick is a technique for scenarios where the data can't be separated by a linear boundary (e.g. data scattered in a circular pattern) — it mathematically transforms the data into a higher dimension so a linear boundary can be found there (you can choose options like `kernel='linear'`, `kernel='rbf'`, etc.).

Connecting to a Real Scenario

Writing `from sklearn.svm import SVC; model = SVC(kernel='rbf'); model.fit(X_train, y_train)` uses the 'rbf' (Radial Basis Function) kernel, letting you classify complex patterns that a linear boundary couldn't separate — SVM has traditionally shown strong performance on problems with high-dimensional feature spaces, like image classification and text classification (spam detection).

SVM = finding the boundary that separates two classes with 'maximum margin'
Support Vector = the data points closest to the boundary, which determine the decision boundary
Kernel Trick (linear/rbf) = transforms data that can't be separated linearly into a higher dimension where it can be

Let's Look at It Together

python

from sklearn.svm import SVC

# 'rbf' kernel handles non-linear decision boundaries
model = SVC(kernel='rbf', C=1.0)
model.fit(X_train, y_train)

accuracy = model.score(X_test, y_test)
print(f"SVM accuracy: {accuracy:.2f}")

You should see

SVM accuracy: 0.89

Try It in 5 Minutes

Train an SVM model with both `kernel='linear'` and `kernel='rbf'`, and compare the accuracy results — this lets you gauge whether your dataset's pattern is linear or non-linear based on which kernel performs better.

A Quick Word of Caution

Using SVM's default hyperparameter values (`C`, kernel parameters) without tuning them can lead to suboptimal performance — go on to study Hyperparameter Tuning (next lesson).

Easy traps

Trying to train SVM directly on a large dataset (millions of rows) — SVM's training time grows significantly as dataset size increases, so it's not the best-suited algorithm for large-scale problems
Using SVM without Feature Scaling — since SVM relies on a distance-based concept like KNN, scaling has a significant effect on performance

Now Try It Yourself

You'll know it worked when: SVM accuracy: 0.89