Exercise — Choose the Right Architecture

Build the mental model

Picking an architecture starts with one question: what shape is the data, and what structure does that shape imply? If the input is a fixed-size grid where nearby values are related — pixels in an image — a CNN wins, because its convolutional filters exploit spatial locality with far fewer parameters than a fully-connected layer would need. If the input is a variable-length sequence where order and long-range context matter — sentences, time series — an RNN/LSTM can work for shorter sequences, but a Transformer's attention mechanism scales better to long-range dependencies and trains faster because it processes positions in parallel rather than step by step. If the input is just a fixed number of independent features with no spatial or sequential relationship — square footage, bedroom count, zip code — a plain feedforward network of stacked Linear and activation layers is not just sufficient but correct: a CNN or RNN would impose structure that doesn't exist in the data, wasting capacity and often hurting generalization. Finally, when labeled data is scarce for a task resembling a well-studied one, transfer learning from a pretrained model — reusing its learned features and fine-tuning only the last layers — consistently beats training an equivalent architecture from scratch.

Connect it to a real scenario

Thuta Learning's roadmap lists three learned features: a sentiment classifier for feedback text, a learned search-ranking model over query and content-metadata features, and a lesson recommender based on each learner's fixed-length engagement-history vector. You're asked to review the design doc before implementation starts and flag any proposed architecture that doesn't match its data's structure — for instance, catching a proposal to run a CNN over the ranking model's tabular features, or to use a plain feedforward network on raw review sentences instead of something that respects word order. Getting this matched correctly up front avoids months of underperforming models that 'sort of work' but never reach the accuracy simpler, correctly-chosen architectures would.

Architecture choice follows data structure, not habit or hype: grid data with locality wants CNNs, ordered sequences want RNN/LSTM or Transformers, and independent feature vectors want plain feedforward networks.
For long sequences, Transformers usually beat RNN/LSTM on both accuracy and training speed because attention processes all positions in parallel instead of one step at a time.
When labeled data is scarce, fine-tuning a pretrained model via transfer learning almost always outperforms training the same architecture from scratch.

Try the working example

python

# Scenario A: classify product photos into 10 categories
# (fixed-size images, 224x224x3, labeled by category)

# Scenario B: predict tomorrow's stock closing price
# from the last 30 days of daily closing prices

# Scenario C: predict a house's sale price from
# square footage, number of bedrooms, and zip code

# Scenario D: translate a full sentence from
# Burmese to English

You should see

This snippet doesn't run or print anything — it's a set of scenario descriptions to reason about. Your job is to write down which architecture family fits each one and why, not to execute code.

5-minute try-it

For each scenario (A–D) above, name the architecture family you'd start with (feedforward network, CNN, RNN/LSTM, or Transformer) and justify it in one sentence by pointing to the specific structural property of the data — grid locality, sequence order, or feature independence — that drove your choice. Then note, for scenario D specifically, why a Transformer is now the more common default over an RNN/LSTM despite both being valid choices.

One important caution

Reaching for a CNN or RNN out of habit even when the data is just independent tabular features — this adds complexity and often hurts accuracy compared to a simple feedforward network.

Assuming more 'advanced' always means better — e.g. using a Transformer on a tiny dataset with short sequences, where its extra parameters overfit and an LSTM (or even a simpler model) would generalize better.

Wikipedia — Neural network (machine learning) — Deep Learning