Tensors and PyTorch Basics

Build the mental model

A tensor is PyTorch's fundamental data structure: a multi-dimensional array, generalizing the familiar progression from scalar (a single number, 0 dimensions) to vector (a list of numbers, 1 dimension) to matrix (a grid, 2 dimensions) to arbitrary higher dimensions. A batch of 8 RGB images, each 28x28 pixels, is naturally a 4D tensor with shape (8, 3, 28, 28) — batch size, color channels, height, width. On the surface a tensor looks like a NumPy array, and PyTorch deliberately mirrors NumPy's API for creation and indexing, but tensors add two things NumPy arrays lack: they can track the history of operations applied to them so gradients can be computed automatically (covered next lesson), and they can be moved to a GPU with `.to('cuda')` so operations run as massively parallel matrix math instead of one CPU core at a time. Every tensor has a shape (its dimensions) and a dtype (the numeric type of its elements, usually float32 for neural network computations) — mismatches in either are the most common source of runtime errors when building networks.

Connect it to a real scenario

Every feature this course builds toward — sentiment classification of feedback, learned search ranking, lesson recommendations — starts by turning platform data (feedback text, click logs, lesson embeddings) into tensors, since that's the only form PyTorch's models and GPUs can compute on. A batch of 64 learner feedback comments, once converted to numbers, might become a tensor of shape (64, 200) — 64 comments, each represented as 200 numbers. Getting comfortable with shapes, dtypes, and basic tensor math now is what makes every later lesson's code readable instead of mysterious.

A tensor generalizes scalar -> vector -> matrix -> N-dimensional array; shape describes its dimensions.
Tensors track operation history for automatic gradients and can run on a GPU — two things plain NumPy arrays can't do.
`@` or `torch.matmul` performs matrix multiplication; `+`, `*` etc. perform element-wise operations.

Try the working example

python

import torch

# Create tensors
a = torch.tensor([[1.0, 2.0], [3.0, 4.0]])
b = torch.randn(2, 2)

print("a shape:", a.shape, "dtype:", a.dtype)
print("b shape:", b.shape)

# Element-wise operation
c = a + b
print("a + b =\n", c)

# Matrix multiplication
d = a @ b
print("a @ b =\n", d)

# A batch of 8 RGB images, 28x28 pixels -> 4D tensor
batch = torch.zeros(8, 3, 28, 28)
print("batch shape:", batch.shape)

You should see

Prints the shape (torch.Size([2, 2])) and dtype (torch.float32) of tensor a, the result of element-wise addition a + b, the result of matrix multiplication a @ b, and the shape of a 4D batch tensor (torch.Size([8, 3, 28, 28])).

5-minute try-it

Create a 3D tensor of shape (5, 4, 3) filled with random values using torch.randn, print its shape and dtype, then reshape it to (5, 12) using .view() or .reshape() and print the new shape.

One important caution

Mixing tensor dtypes (e.g. adding a float32 tensor to an int64 tensor) raises a RuntimeError in many operations — always check .dtype when combining tensors from different sources.

Using `*` when you meant matrix multiplication — `*` is element-wise and requires matching (or broadcastable) shapes, while `@`/`torch.matmul` follows different shape rules for genuine matrix multiplication; confusing the two either crashes or silently computes the wrong thing.

PyTorch Docs — Tensors — Deep Learning