Project: Building a Text Sentiment Classifier

Build the mental model

Classifying text sentiment needs a way to turn discrete word indices into something a neural network can do math on, then a way to collapse a variable-length sequence into one fixed-size decision. One-hot vectors would work but waste dimensions and share no information between related words; nn.Embedding instead learns a dense vector per word index during training, so words that behave similarly in context end up with similar vectors. Feeding those embeddings through an RNN one token at a time produces a hidden state that's updated by every word in sequence, so the final hidden state summarizes the whole sentence regardless of its length. A Linear layer then maps that fixed-size summary to two logits (positive/negative). The naive alternative — averaging word embeddings and skipping the RNN entirely — ignores word order ('not good' and 'good not' would look identical), while the RNN's sequential updates let order and context shape the final representation. This embedding-then-RNN-then-classify shape is the backbone of real sentiment models before transformers took over the field.

Connect it to a real scenario

This is a direct prototype for a real Tutorial Platform feature: classifying learner feedback comments as positive or negative so lessons that repeatedly get 'confusing' or 'too fast' feedback get automatically flagged for a content review, while lessons getting 'great explanation' feedback get surfaced as examples of strong teaching style. The same embedding-plus-RNN architecture would scale up from this toy 8-sentence, 20-word vocabulary to real feedback text tokenized with a proper subword tokenizer and a much larger embedding table, but the model shape, training loop, and evaluation logic here would carry over almost unchanged into that production feature.

• nn.Embedding learns dense word representations that share information between related words, unlike one-hot encoding.
• An RNN's final hidden state is a fixed-size summary of a variable-length sequence, built up token by token in order.
• Word order matters for meaning — averaging embeddings loses it, but an RNN's sequential updates preserve it.

Try the working example

import torch
import torch.nn as nn
import torch.optim as optim

vocab_size = 20
embed_dim = 8
hidden_dim = 16

sentences = [
    [1, 2, 3, 4],    # "this lesson was great"
    [5, 6, 7],       # "great explanation thanks"
    [8, 9, 10, 11],  # "this lesson was confusing"
    [12, 13, 14],    # "very confusing explanation"
    [1, 6, 4],        # "this great was"
    [8, 13, 11],       # "this confusing was"
    [5, 2, 3],
    [12, 9, 10],
]
labels = [1, 1, 0, 0, 1, 0, 1, 0]

def pad_sequences(seqs, pad_value=0):
    max_len = max(len(s) for s in seqs)
    return torch.tensor([s + [pad_value] * (max_len - len(s)) for s in seqs])

X = pad_sequences(sentences)
y = torch.tensor(labels)

class SentimentRNN(nn.Module):
    def __init__(self, vocab_size, embed_dim, hidden_dim):
        super().__init__()
        self.embedding = nn.Embedding(vocab_size, embed_dim, padding_idx=0)
        self.rnn = nn.RNN(embed_dim, hidden_dim, batch_first=True)
        self.fc = nn.Linear(hidden_dim, 2)

    def forward(self, x):
        embedded = self.embedding(x)
        _, hidden = self.rnn(embedded)
        return self.fc(hidden.squeeze(0))

model = SentimentRNN(vocab_size, embed_dim, hidden_dim)
criterion = nn.CrossEntropyLoss()
optimizer = optim.Adam(model.parameters(), lr=0.01)

for epoch in range(50):
    optimizer.zero_grad()
    outputs = model(X)
    loss = criterion(outputs, y)
    loss.backward()
    optimizer.step()
    if (epoch + 1) % 10 == 0:
        print(f"Epoch {epoch+1}, Loss: {loss.item():.4f}")

The script prints the loss every 10 epochs, and it drops steadily toward near-zero across the 50 epochs (with only 8 sentences, the model fits them almost perfectly).

PyTorch Docs — Text Classification Tutorial — Deep Learning