Build your own ChatGPT (9/9): putting it together and the bill of three real products

Eight parts, nine pieces. One thing left: putting them together without the cost getting away from you. This part closes the series with the full architecture, the mechanism that saves the most money, and the monthly bill of three products that genuinely exist.

The complete architecture

                        ┌─────────────────┐
     user request ─────▶│  ORCHESTRATOR   │  (your code — not a model)
                        └────────┬────────┘
                                 │  classifies intent and routes
        ┌────────────────────────┼─────────────────────────┐
        │                        │                         │
   ┌────▼─────┐          ┌───────▼────────┐        ┌───────▼────────┐
   │ RETRIEVAL│          │     BRAIN      │        │  SPECIALISTS   │
   │ your docs│─chunks──▶│   per token    │◀──────▶│   on demand    │
   │ (Qdrant) │          │   5 models     │        │ OCR · image    │
   └──────────┘          └───────┬────────┘        │ video · voice  │
        ▲                        │                 └────────────────┘
        │                        ▼                  start · run · stop
    always on             streamed answer
    (small)                                    ┌────────────────┐
                                               │     TOOLS      │
                                               │ live data,     │
                                               │ APIs, agents   │
                                               └────────────────┘

The rule that organises the whole bill

The entire series reduces to one line: separate what must be running from what only needs to exist when called.

The most expensive structural mistake — and the most common — is putting the brain in the first row. It is the highest-frequency, least predictable piece: exactly the worst candidate for a machine running 24 hours a day.

The model router: where the money is made

In part 2 we saw the same product costing R$ 112 or R$ 4,973 a month depending on model choice. The router is what lets you sit near the first number while delivering quality close to the second.

The idea: the overwhelming majority of calls in a real product are simple work — classify, extract, rephrase, answer a direct question. Only a small slice needs hard reasoning.

SIMPLE = ("classify", "extract", "summarise", "translate",
          "rewrite", "correct", "moderate", "title")

def choose_model(task: str, text: str, attempt: int = 0) -> str:
    # 1) Mechanical task declared by your code: always the cheapest.
    if task in SIMPLE:
        return "gpub-fast"

    # 2) Reprocessing: if the first answer failed validation, step up.
    if attempt > 0:
        return "gpub-base"

    # 3) Conversation: a cheap heuristic on length and type of request.
    hard = len(text) > 4000 or any(
        w in text.lower() for w in ("why", "compare", "analyse", "code", "error")
    )
    return "gpub-base" if hard else "gpub-plus"

Three products, with the bill closed

Product A — Internal assistant for 40 people

Chat over the company document base, with PDF reading. 44,000 messages a month, 2,000 scanned pages a month.

For comparison: the same product using only the most capable model everywhere would cost R$ 4,980. The router accounts for 96% of the difference.

Product B — Customer support

5,000 conversations a month, 8 messages each, with retrieval over the help base and input moderation.

That is R$ 0.028 per conversation handled. And the detail worth more than the table: without the sliding window, resending the whole thread every time, the same operation would cost R$ 379 — nearly three times more, for identical answer quality.

Product C — Content studio

3,000 images, 20 videos of 30 seconds and 200 minutes of narration a month.

August 2026 prices, single GPU, no scheduled interruption. Prices are in Brazilian reais (BRL), the billing currency. The catalogue moves — check the live one before committing to a number.

Notice the distribution: video is 94% of the bill with only 20 pieces a month. If that product grows, video is the piece that needs attention — not the rest.

The five mistakes that blow the budget

1. The expensive model as the default. 20 to 45 times the cost for the same delivery. Always mistake number one.
2. The whole history on every message. Triples the cost of long conversations and degrades answer quality on top.
3. The whole document in context. Up to 95 times more expensive than retrieving the right passages (part 4).
4. A GPU left on, waiting. R$ 780 to R$ 2,030 a month of idleness — it disappears on its own once the queue starts the machine.
5. An agent with no ceiling. A stuck loop runs all night burning balance. Always set maximum steps and tokens.

Three honest things before you go live

• Measure from day one. Log tokens, model, latency and cost per call. Without it you cannot tell which feature is consuming what — and you will optimise the wrong thing.
• Keep a model plan B. Because the API follows the OpenAI standard, switching models is switching a string. Make it configuration, not something buried in code.
• Talk to us before launch if volume will be large. There is a per-account ceiling on concurrent calls, sized for normal usage. An application with hundreds of simultaneously active users needs that ceiling raised before going live — it is a five-minute conversation, and far better had in advance than discovered on launch day.

The whole path, in one table

Nine pieces, none of them closed, every one with a mature open equivalent. What was a multi-million project three years ago is now a matter of assembling the pieces in the right order and paying for each one with the cost model that suits it.

Back to the start: part 1 — what is actually inside these platforms.

Keep reading: part 2: the brain · serverless vs dedicated GPU · deploying to production