Build your own ChatGPT (2/9): the brain, and why the answer costs more than the question

Of the ten pieces in part 1, this is the only one invoked on every single interaction. Build just this and you have a product. Get its arithmetic wrong and none of the other nine will save your unit economics.

The token: the unit you are buying

A language model reads neither letters nor words: it reads tokens, which are word fragments. "Computer" might be one token; an unusual proper noun might be five.

The working rule of thumb: 1 token ≈ 4 characters, or 1 word ≈ 1.3 tokens. One A4 page of prose is about 600 tokens. A 20-page contract, about 13,000.

Why the answer costs more than the question

Look at the table and notice that the two numbers are never the same:

Price per million tokens, in Brazilian reais (BRL), August 2026. Live table at gpubrazil.com/en/llm-api.html.

The gap is not commercial, it is physical. Input is read in one pass: the 10,000 tokens of your prompt enter the GPU together, in parallel, and the card does what it is best at.

Output is generated one token at a time. To write word 500 of the answer, the model must already have written the previous 499 and must make another full pass through all of its billions of parameters. Five hundred output tokens are five hundred sequential passes.

That is why output costs two to five times input at every model, at every provider on earth. And why the most profitable instruction you can put in your system prompt is "be concise".

The arithmetic of one conversation

An internal assistant; a typical question with retrieved document context:

At product scale — 40 people, 50 messages a day each, 22 working days, so 44,000 messages a month:

• On the economy model: R$ 112 a month.
• On the mid model: R$ 206 a month.
• On the most capable model in the catalogue: R$ 4,973 a month.

Forty-four times the cost for the same product. This is where an architecture decision becomes a business decision — and the subject of part 9.

The trap that blows up budgets: history

This is the part most teams discover late. The model remembers nothing. On every message your application resends the entire conversation, because that is the only way the model can know what is being discussed.

Which means: on message 20 of a conversation, you are paying for message 1 for the twentieth time.

Notice the shape: doubling the length of a conversation quadruples what you have paid for it. A long support thread of 40 exchanges costs 410,000 input tokens — the equivalent of more than fifty short conversations.

Context window: what actually fits

Context is the token ceiling of one call — input plus output. The catalogue models range from 200,000 to 1,000,000 tokens, which on paper is a great deal:

• 200,000 tokens ≈ 300 pages ≈ a whole technical book.
• 1,000,000 tokens ≈ 1,500 pages ≈ a medium-sized codebase.

Two honest caveats. First, fitting is not understanding: every model loses precision in the middle of very long contexts, and a fact buried on page 400 is retrieved less reliably than one on page 3. Second, a large context is expensive by definition, because context is input and input is billed. Stuffing 800,000 tokens into a simple question means paying R$ 14 for something good retrieval would answer for fractions of a cent.

A wide context is a safety net, not a strategy. The strategy is part 4.

Choosing between the five

In practice: the code

The API is OpenAI-compatible, so any library, framework or tool you already use works by changing two lines — the base URL and the key.

curl https://gpubrasil.com.br/v1/chat/completions \
  -H "Authorization: Bearer gpub_live_yourkeyhere" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpub-plus",
    "messages": [
      {"role": "system", "content": "You are an internal company assistant. Be direct and brief."},
      {"role": "user", "content": "Summarise the refund policy in 3 bullets."}
    ],
    "max_tokens": 500
  }'

In Python, using the official OpenAI library pointed here:

from openai import OpenAI

client = OpenAI(
    base_url="https://gpubrasil.com.br/v1",
    api_key="gpub_live_yourkeyhere",
)

stream = client.chat.completions.create(
    model="gpub-plus",
    messages=[
        {"role": "system", "content": "Be direct and brief."},
        {"role": "user", "content": "Explain what a token is, for a non-technical reader."},
    ],
    max_tokens=400,
    stream=True,          # the answer arrives word by word
)

for chunk in stream:
    piece = chunk.choices[0].delta.content
    if piece:
        print(piece, end="", flush=True)

About stream=True

Not decoration: it is the difference between a product that feels fast and one that feels frozen. Without streaming, the user stares at a still screen for six seconds until the whole answer lands. With it, the first words appear in under a second and the perception of speed changes completely — even though the total time is identical.

It costs the same. Always turn it on.

Two defences worth having on day one

• Always set max_tokens. It is your handbrake: without it, one badly phrased prompt produces a 4,000-token answer and you pay for it.
• Never put the key in the browser. The call goes out from your server. A key in public-page JavaScript is a published key — and whoever finds it spends your balance.

So when is a dedicated GPU worth it?

It genuinely is — later than most people assume. The break-even depends on the size of model you want to serve, and there are two very different scenarios:

In other words: until your product passes the million-messages-a-month mark, a dedicated card is an expensive machine sitting idle overnight.

Beyond volume, a dedicated GPU still wins in two situations:

1. A model fine-tuned to your domain — if you trained your own adapter, it only runs on your machine.
2. An isolation requirement — when policy demands the processing happen on a dedicated instance of your own, with you controlling the model, the weights and the logs.

Outside those cases, starting per token and migrating later is almost always right — and it does not lock you in: because the API follows the OpenAI standard, moving to your own server later means changing the base URL.

Next: the eyes — making your assistant read images and PDFs.

Keep reading: part 4: the memory · token economics vs self-hosting · open model comparison 2026