Train/Test Split

Relax. We'll talk through this in plain words — no textbook voice.

What you'll walk away with

Understand Train/Test Split, without any of the intimidation
Be able to run scikit-learn code yourself
Be able to apply this concept immediately in a real project

Let's think about it this way for a second

If you train a model on all your data and then evaluate it on that same data, the model can just 'memorize' the training data, and you have no way of knowing how well it'll actually do on unseen data (overfitting risk). Train/Test Split solves this by dividing the data by a ratio (e.g. 80/20) and showing the model only the training set — the test set is kept completely 'unseen' by the model and used only at evaluation time, like handing out an exam paper the students have never seen before.

Let's connect it to a real scenario

Writing `from sklearn.model_selection import train_test_split; X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)` splits the data into 80% training (`X_train`, `y_train`) and 20% test (`X_test`, `y_test`) — fixing `random_state=42` makes the split reproducible, so you get the same split every run.

Train/Test Split = splitting data into training (shown to the model) vs. test (unseen evaluation)
Keep the test set completely 'unseen' by the model, and use it only at evaluation time
random_state = fixing the seed so the split is reproducible

Let's look at it together

python

from sklearn.model_selection import train_test_split

# X = features, y = target (what we want to predict)
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42
)

print(f"Training samples: {len(X_train)}")
print(f"Test samples: {len(X_test)}")

You should see

Training samples: 800
Test samples: 200

5-minute try-it

Split the sample dataset (from Basic lesson 4) using `train_test_split()` with an 80/20 ratio — print the training/test sample counts to confirm.

A quick word of caution

Random splits aren't suitable for time-series data (stock prices, weather) — if 'future' data ends up in the training set (breaking the time order), you get data leakage; use a chronological split instead.

Easy traps

Setting the test set size too small (5%) — the evaluation results from such a tiny test set may not be statistically reliable, since you're drawing conclusions from very few samples
Not setting a `random_state` — the split changes every run, making results hard to reproduce or debug

Now try it yourself

Split the sample dataset (from Basic lesson 4) using `train_test_split()` with an 80/20 ratio — print the training/test sample counts to confirm.

You'll know it worked when: Training samples: 800 Test samples: 200