Building a Local AI App (JavaScript/TypeScript)

Build the mental model

Knowing that a local model exposes an HTTP API is only useful once you know where that call belongs in a real application, and the honest answer is: almost never directly in the browser. A typical small local-AI app has three layers.

• Frontend — what the user sees and clicks, built with plain HTML/JS or a framework like React, running inside the user's browser.
• Backend — a small server you control, written in something like Node.js or Express, that receives requests from the frontend and decides what to do with them.
• Local model API — served by Ollama or a similar runtime, this is what actually generates text.

The correct flow is frontend calls backend, and backend calls the local model API, never frontend calls the local model API directly. This matters for three concrete use cases you'll see repeatedly.

• Chat feature — the backend forwards each user message to the model and streams the reply back.
• Summarizer — the backend receives a long document from the frontend, wraps it in a summarization prompt, and returns a shorter version.
• Translator — the backend builds a translation-specific system prompt around whatever text and target language the frontend sent.

Routing everything through your own backend gives you one place to validate input, rate-limit requests, log usage, and add authentication before anything reaches the model. Skipping the backend and calling the local API straight from client-side JavaScript works fine on your own laptop during development, but it quietly assumes the browser and the model server are the same trusted machine, an assumption that breaks the moment you deploy.

SAFE VS INSECURE LOCAL AI ARCHITECTURE
--------------------------------------
------------------------------------------------------------
  SAFE PATTERN
  --------------------------------------------
  BROWSER  -->  YOUR BACKEND  -->  LOCAL MODEL API  -->  MODEL
  (frontend)    (validates,        (Ollama /api/chat)
                 rate-limits,
                 logs, auths)

  INSECURE ANTI-PATTERN -- do not do this
  --------------------------------------------
  BROWSER  ------------------------------->  LOCAL MODEL API
  (frontend)   calls the model API directly,   (exposed, open,
               no backend in between            unauthenticated)

Connect it to a real scenario

Consider building a small internal translator tool: a coworker pastes English text into a web page, picks a target language, and gets a translation back.

The frontend never learns the model's address, its port, or that Ollama is even involved; it only knows how to talk to your backend. This indirection is exactly what lets you add a request limit per user, log which translations were requested, or later swap Ollama for a different local runtime, all without touching the frontend code at all.

• The correct flow is frontend -> your backend -> local model API, never frontend -> local model API directly.
• Chat, summarizer, and translator features all follow the same architecture — only the system prompt and the backend's request shaping differ.
• Routing through your own backend is what makes authentication, rate limiting, and logging possible before a request ever reaches the model.

Try the working example

// backend route handler, e.g. Express: app.post("/api/translate", translateHandler)

interface TranslateRequestBody {
  text: string;
  targetLanguage: string;
}

interface OllamaChatResponse {
  message: { role: string; content: string };
}

const OLLAMA_CHAT_URL = "http://localhost:11434/api/chat";

async function translateHandler(
  req: { body: TranslateRequestBody },
  res: { status: (code: number) => { json: (body: unknown) => void } }
): Promise<void> {
  const { text, targetLanguage } = req.body;

  const systemPrompt =
    `You are a translation engine. Translate the user's text into ` +
    `${targetLanguage}. Reply with only the translated text.`;

  const payload = {
    model: "llama3.2",
    messages: [
      { role: "system", content: systemPrompt },
      { role: "user", content: text },
    ],
    stream: false,
  };

  try {
    const response = await fetch(OLLAMA_CHAT_URL, {
      method: "POST",
      headers: { "Content-Type": "application/json" },
      body: JSON.stringify(payload),
    });

    if (!response.ok) {
      res.status(502).json({ error: "Local model server returned an error." });
      return;
    }

    const data = (await response.json()) as OllamaChatResponse;
    res.status(200).json({ translated: data.message.content });
  } catch (error) {
    res.status(503).json({ error: "Local model server is unreachable." });
  }
}

This code is not executed here because it depends on a locally running Ollama server and an HTTP framework's request/response objects, neither of which exist in this environment. In a real deployment, a successful call returns HTTP 200 with a JSON body like `{ translated: '<translated text>' }`, where the actual translated string comes from the model and is not something to hardcode or predict. If Ollama is unreachable, the handler responds with HTTP 503 and an error message instead of crashing the server process; if Ollama responds with an HTTP error status, the handler responds with HTTP 502.

MDN: Using the Fetch API — Local AI / Local LLM