# It's still client-server

AI for Software Engineers, part 2.1 (4 of 22)

> Every AI app is still client-server: the app or its backend sends an HTTPS POST with the model name, the messages so far and a few parameters, and gets back content, a stop reason and token usage. The API is stateless, so the app does the remembering, and the same request shape works against local runtimes like Ollama.

You type a question into ChatGPT or Claude and press Enter. A second later the answer starts to appear. In Claude Code or GitHub Copilot you ask for a change, and a few seconds later files are being edited.

It feels a bit like magic. So before reading on, think about it the way you would think about any app you've built. What actually happens between you pressing Enter and the text showing up on screen?

The answer is less exciting than the marketing, and that's good news for you. It is still client-server. The chat window or the coding agent is a client. Somewhere it makes an HTTPS call to a model provider's API, gets a response, and shows it to you. You already know every one of those words.

In the last article we split the model from the application. This is where that split becomes concrete. Here's the path, roughly.

![Sequence diagram from a developer application through a provider API to model inference and streamed response chunks.](assets/03-api-inference-streaming.png)
*From the application's point of view, inference is still an API request.*

Your app is whatever you type into: a browser tab, an IDE panel, a terminal. It sends your message to the app's backend. The backend sends an HTTPS POST to the provider's API, with a JSON body and an API key in a header. The provider runs the model and sends JSON back, and the backend passes the answer to your screen.

Sometimes the app and the backend are the same thing. A coding agent running in your terminal can call the provider's API directly with your key. In a web app there's usually a backend in the middle doing a lot of work. It checks who you are, stores your chat history, decides which tools to offer, adds any documents it fetched, and logs the call. The model sees only what the backend chooses to send.

So GitHub Copilot, a startup's chatbot and your own app all have the same shape. What differs is the backend. Sometimes there's one more hop, like a company LLM gateway, or a cloud platform such as Amazon Bedrock or Google's Vertex AI that also serves these models. Same idea as an API gateway in front of any other service.

One practical point while we're here. The API key belongs on the backend, never in the browser or the mobile app. Anyone who has your key can run up your bill in your name.

So what's inside that JSON?

Less than you might expect. Every provider asks for the same few things:

- which **model** you want, by name
- the **messages** so far, each with a **role**: the user said this, the assistant said that
- some **parameters**, like the maximum length of the answer
- optionally, the **tools** the model is allowed to ask for

And the response gives you back:

- the generated **content**
- **why it stopped**: it finished, it hit the length limit, or it wants to call a tool
- **usage**: how many tokens went in and how many came out. This is what you pay for, and we'll come back to it when we talk about context

Every provider does the same job, and each picks its own field names. Here's how a few of them line up.

| What it is | Anthropic Messages | OpenAI Responses | Gemini generateContent |
|---|---|---|---|
| The conversation | `messages` | `input` | `contents` |
| Who said it | `user`, `assistant` | `user`, `assistant` | `user`, `model` |
| Answer length limit | `max_tokens` | `max_output_tokens` | `maxOutputTokens` |
| Why it stopped | `stop_reason` | `status`, `incomplete_details` | `finishReason` |
| Token counts | `usage` | `usage` | `usageMetadata` |

You don't need to memorise this. OpenAI still supports its older Chat Completions API, and Google now recommends its newer Interactions API for new projects, so the names keep moving. When you need them, go to the source: the [Anthropic Messages API](https://platform.claude.com/docs/en/build-with-claude/working-with-messages), the [OpenAI Responses API](https://developers.openai.com/api/docs/guides/migrate-to-responses) and the [Gemini API](https://ai.google.dev/gemini-api/docs/text-generation).

Because the shapes are so close, you'll see "OpenAI-compatible" endpoints everywhere. Google and Anthropic both let you point OpenAI's SDK at their models by changing the base URL and the key. Handy for trying things out, but Anthropic itself says its layer is mainly for testing and comparison, and you lose some features.

Now here's the question I care most about in this article. Say you're five messages into a chat. What does the request need to contain for the model to answer your fifth message well?

All of it. Messages one to four, both sides, plus the new one.

The model remembers nothing between calls. Each request is on its own. Anthropic's docs say it plainly. The Messages API is stateless, so you always send the full conversation history. When ChatGPT seems to follow the conversation, that's the app keeping your history and sending it again, every single time. If you've built a stateless REST service that gets the whole session in each request, you already know how this works.

That one fact explains a lot of what comes later. Long chats cost more because every turn resends everything before it, and eventually the history doesn't fit.

Providers do offer to keep the state for you. OpenAI's Responses API can chain calls with a previous response ID or store a whole Conversation, and Gemini's Interactions API has something similar. It saves you some plumbing, though OpenAI notes you are still billed for the earlier tokens. Treat it like any other external dependency. Your conversation data lives with the provider, under their retention rules, and switching providers means moving that state too. OpenAI gave its older Assistants API, which kept threads on its side, a shutdown date of 26 August 2026. If your product's memory lived in those threads, the migration was your problem. For anything important, I'd keep the history in my own store and send it myself.

So far the model has been one box at the end of the path. What happens inside it?

That box is inference. At the provider, the model's trained weights sit loaded on GPUs. Your messages go in as tokens, and the model works out the next token, adds it to the input, works out the next one, and so on, one token at a time, until it decides to stop or hits your length limit. Training made the weights. Inference is running them. The provider API is a front door to that, with keys, billing and rate limits around it.

You can run the same thing yourself with an open-weight model, like the Gemma models from the last article. [llama.cpp](https://github.com/ggml-org/llama.cpp) is an open-source inference engine in C/C++ that runs on ordinary CPUs, Apple Silicon and most GPUs, with a small HTTP server built in. [Ollama](https://docs.ollama.com/) gives you a simple command line and a local API for open models. [LM Studio](https://lmstudio.ai/docs/app) is a desktop app where you download a model, chat with it and turn on a local server.

Models for these tools usually come as GGUF files, one file holding the weights and the details needed to run them. Most are quantized, which means the weights are stored with fewer bits so the model fits in your machine's memory, at some cost in accuracy.

And this is where the API contract pays off. All three expose an OpenAI-compatible API on localhost, and all three also offer an Anthropic-style Messages endpoint. So a tool built for a hosted API can be pointed at your own machine by changing the base URL. Ollama's docs show this with Claude Code. You set Claude Code's base URL to Ollama's local address, give it a placeholder token and pick a local model. Anthropic says it doesn't support Claude Code with non-Claude models, so treat that as an experiment.

The models you can run on a laptop are smaller and weaker than the frontier models behind the big APIs, and bigger ones need serious hardware. In return your data stays on your machine, it works offline and there's no per-token bill. Hugging Face has a short [guide to GGUF](https://huggingface.co/docs/hub/gguf) if you want to go further.

![Your app or Claude Code sends the same OpenAI-style or Anthropic-style request. With the provider's base URL it goes to the provider API, which handles keys, billing and rate limits, and then to a hosted model on the provider's GPUs. With a localhost base URL it reaches a local runtime such as Ollama, llama.cpp or LM Studio, which runs an open-weight GGUF model on your own machine.](assets/02a-two-back-ends.svg)
*Same front door, two back ends. The base URL decides which one answers.*

So when you press Enter, there's no magic. A client sends JSON over HTTPS, a stateless service answers, and your app does the remembering.

Next up are the two things I skipped. Where the system prompt goes, and how the answer streams onto your screen word by word.