The Local AI API

Build the mental model

Once you understand that Ollama, LM Studio's server mode, and llama.cpp's server all expose a local HTTP API, you can talk to a local model from any programming language that can make an HTTP request, not just from a chat window. The pieces of this puzzle are worth naming precisely.

• Localhost — a special hostname, backed by the address 127.0.0.1, that always means “this same machine,” so a request to localhost never leaves your computer and never touches the network.
• Port — a number, like 11434 or 1234, that identifies which particular program on the machine should receive the request, since many programs can be listening on the same machine at once.
• Endpoint — a specific URL path on that server dedicated to one job, such as /api/chat for a conversation or /api/embeddings for turning text into a vector; the same server can expose several endpoints for different tasks.
• Request — the message your application sends: which endpoint, which model name, and the conversation so far, usually encoded as JSON, a plain-text format for structured data that both humans and programs can read.
• Response — the server processes the request through the runtime and model, then sends this back, also JSON, containing the generated text and often some metadata like token counts.

This request-response cycle over HTTP is exactly how a web app talks to a remote API like a weather service; the only difference with a local AI API is that the server lives on your own machine instead of somewhere else on the internet.

localhost

A special hostname (backed by the address 127.0.0.1) that always refers to the same machine the request originates from, so it never travels over an external network.

Endpoint

A specific URL path on a server dedicated to one job — such as /api/chat for a conversation or /api/embeddings for turning text into a vector — with one server able to expose several endpoints.

API

A set of rules and endpoints a server exposes so other programs can send it requests and receive structured responses (usually JSON), letting software talk to software without a human clicking anything.

APP TO MODEL VIA LOCAL API
--------------------------
------------------------------------------------------------
  YOUR APP
     |
     |  HTTP request to
     |  http://localhost:PORT/endpoint
     |  (JSON body: model name + messages)
     v
  LOCAL RUNTIME  (Ollama / llama.cpp server / LM Studio server)
     |
     |  loads and runs
     v
  MODEL
     |
     |  generates text
     v
  JSON RESPONSE  (generated text + metadata)
     |
     |  sent back over the same connection
     v
  YOUR APP  (reads the response, uses the text)

Connect it to a real scenario

Say you're building a small Python script that summarizes meeting notes using a locally running model, and you want it to work entirely offline.

The code example below shows exactly this pattern using Python's `requests` library, written so you can adapt the endpoint URL and message content to any Ollama-compatible local server you're running.

• localhost, port, and endpoint together identify exactly which local program and which job should handle a request.
• A request and its response are both structured as JSON, the same pattern used to call any remote web API.
• Robust code around a local API call handles connection errors, timeouts, and HTTP error statuses, not just the success path.

Try the working example

import requests

OLLAMA_CHAT_URL = "http://localhost:11434/api/chat"


def summarize_notes(model_name, meeting_notes):
    payload = {
        "model": model_name,
        "messages": [
            {
                "role": "system",
                "content": (
                    "You are a concise assistant. Summarize the given "
                    "meeting notes in three bullet points."
                ),
            },
            {"role": "user", "content": meeting_notes},
        ],
        "stream": False,
    }

    try:
        response = requests.post(OLLAMA_CHAT_URL, json=payload, timeout=60)
        response.raise_for_status()
    except requests.exceptions.ConnectionError:
        print("Could not reach the local model server. Is Ollama running?")
        return None
    except requests.exceptions.Timeout:
        print("The request timed out waiting for a reply.")
        return None
    except requests.exceptions.HTTPError as error:
        print(f"The server returned an error: {error}")
        return None

    data = response.json()
    return data["message"]["content"]


if __name__ == "__main__":
    notes = (
        "Team agreed to ship the search feature next sprint. "
        "Backend needs the new /api/search endpoint. "
        "Design will deliver mockups by Thursday."
    )
    summary = summarize_notes("llama3.2", notes)
    if summary is not None:
        print(summary)

This code is not executed here because it requires a running local Ollama server. If Ollama is running with `llama3.2` pulled, `summarize_notes` sends the notes to `/api/chat` and the server responds with a JSON object containing a `message` field holding the assistant's reply text; the function returns that text, and the `print` statement would display whatever three-bullet summary the model generated — the exact wording depends on the model and is not something to hardcode or predict. If Ollama isn't running, the `ConnectionError` branch prints a message instead of crashing.

Ollama API documentation — Local AI / Local LLM