Model Formats and How to Choose a Model

Build the mental model

A model's architecture is the design - how its layers are wired together - but that design still has to be saved to disk somehow, and there is more than one way to do that.

This is what a model file format is: the specific way a model's weights and structure are packaged into a file, separate from the architecture itself, the way a photograph can be saved as JPEG or PNG without changing what is actually in the picture.

Three formats are common:

• GGUF - built specifically for running quantized models efficiently on consumer hardware; the format most beginner-friendly local tools expect
• Safetensors - focused on safety and speed of loading; deliberately cannot execute arbitrary code when opened, unlike some older formats
• ONNX - designed for portability, letting a model trained in one framework run in many different runtimes and even on non-GPU specialized hardware

Before downloading any model, experienced users read its model card - a documentation page, usually hosted alongside the model file, that describes what the model is, who made it, what license governs its use, and what its limitations are. Treat a model card the way you would treat the label on packaged food: it tells you what is actually inside before you commit resources to it.

Skipping the model card means downloading gigabytes of data before discovering it needs hardware you do not have, or carries a license that forbids the exact use you had in mind.

GGUF

A file format built for running quantized models efficiently on ordinary consumer hardware.

Safetensors

A file format focused on safety and fast loading, designed so opening it cannot execute arbitrary code.

ONNX

A portable format that lets a model trained in one framework run across many different runtimes.

Model Card

A documentation page describing a model's creator, license, and limitations.

FROM REPOSITORY TO RUNNING MODEL
--------------------------------
[Model Repository]
        |
  [Model Card]  <- read this FIRST: license, size, format
        |
  [Download File]  (.gguf / .safetensors / .onnx)
        |
  [Local Runtime]  loads the file and runs inference

Connect it to a real scenario

Imagine you are about to download your first local model and see three files with the same name but different extensions: one .gguf, one .safetensors. Instead of guessing, you check which your chosen local tool actually expects - most beginner tools like simple GGUF-based runtimes want the .gguf file specifically, while Python-based workflows often expect Safetensors.

Before downloading either, you open the model card on its repository page and check: what license governs it, does it fit your VRAM, and is it actually intended for the task you have in mind.

This five-minute check regularly saves hours - it catches, before you spend bandwidth and disk space, a model whose license forbids commercial use when you needed one for a business, or a model that needs 24GB of VRAM when you only have 8. The code below automates a small piece of that check: verifying a model card has the fields you actually need before you trust it.

• A file format is different from architecture - it is just how the file is packaged
• GGUF, Safetensors, and ONNX each serve different purposes
• Always check the model card before downloading

Before Downloading a Model

• model name and version
• creator / organization
• architecture type
• parameter count
• license (does it allow commercial use)
• context window size
• file format (GGUF/Safetensors/ONNX) and quantization level
• hardware needed (RAM/VRAM)
• intended use and known limitations

Try the working example

REQUIRED_FIELDS = ["name", "params", "format", "license", "quantization"]


def check_model_card(card):
    """Check a model card dict for the fields a beginner should
    always verify before downloading a model."""
    present = [f for f in REQUIRED_FIELDS if f in card]
    missing = [f for f in REQUIRED_FIELDS if f not in card]
    return present, missing


cards = {
    "complete-example": {
        "name": "Llama-3-8B-Instruct",
        "params": "8B",
        "format": "GGUF",
        "license": "Llama 3 Community License",
        "quantization": "Q4_K_M",
    },
    "incomplete-example": {
        "name": "MysteryModel-7B",
        "params": "7B",
        "format": "safetensors",
    },
}

for card_name, card in cards.items():
    present, missing = check_model_card(card)
    print(f"{card_name}:")
    print(f"  present: {present}")
    print(f"  missing: {missing}")

complete-example:
  present: ['name', 'params', 'format', 'license', 'quantization']
  missing: []
incomplete-example:
  present: ['name', 'params', 'format']
  missing: ['license', 'quantization']

Hugging Face docs: GGUF — Local AI / Local LLM