Why "Hello" Costs Thousands of Tokens
In an AI coding video, a session opened at roughly 8,000 tokens of context before any real work, and a single “hello” pushed it to 9.4K with the model’s reply already on screen. The line sounds wrong until you think about what an agent is actually doing.
The memorable version is that it took thousands of tokens to say “hello” to an AI coding agent. That phrasing is useful as long as we keep it honest. Saying “hello” does not inherently cost 8,000 tokens. The number came from a particular workflow with a particular tool, repository, and session state, and it is not a rule about every AI coding agent.
What matters is where the cost goes. By the time the interaction looked simple, the agent had already spent tokens orienting itself inside a software project.
Why does a short prompt cost so much?
Most developers still carry a chat mental model for AI tools. You write a message, the model replies, and the cost feels attached to that exchange. A short message should mean a small interaction.
Coding agents are preparing to act, not just answering a question. Before an agent can change a file, it needs enough local context to make the next move: what kind of project it is in, which files matter, how tests are run, and what has already happened in the session.
That is why the “hello” example lands. The visible part was tiny. The preparation behind it wasn’t.
Where do the tokens actually go?
The first expensive step in many coding sessions is context gathering, not generation. An agent may inspect the repository tree to find likely entry points. It opens source files, tests, and configuration. It runs commands and reads back terminal output, build logs, and failed tests. Each step produces more information that becomes part of the working context.
Some of that context is exactly what the agent needs. A test failure points to a broken assumption. A build log shows that a dependency is missing. A nearby implementation reveals the pattern the project already uses. And some of it is just the cost of searching — a file that turns out not to matter, a false lead, tool output that was useful for one step and noise for the next.
All of this can happen before the final code change exists. A one-line fix may require reading three files and a stack trace. That small diff is the last visible piece of a much larger process: repository exploration, command output, build logs, and accumulated session state.
The code survives in Git. Most of the exploration disappears.
What does this mean for controlling AI coding costs?
If the cost of AI coding came mostly from generated code, the obvious answer would be to ask the model to write less. That can help at the margins, and smaller diffs are usually better engineering anyway. But for most coding-agent work, telling the model to write less is a distraction: cutting generated code barely lowers the bill because the money is on the input side, not the output.
The larger cost is the information the model reads, rereads, receives from tools, and carries forward while it works. The agent is spending tokens to reduce uncertainty. It is trying to answer ordinary engineering questions: Where is the relevant code? What did the tests say? What state has already accumulated in this session?
Those are orientation questions, not writing questions, and they give us a better mental model. AI coding costs are driven as much by the price of understanding a software system as by the price of producing the final patch.
This is why the “hello” story is more than a curiosity. The bill can begin forming before any of the work is visible.
Where the guide goes deeper
This post is the narrow version of the idea: a small interaction can hide a large amount of context work. The full question is bigger.
Once you see that AI coding cost is tied to understanding, other engineering topics start to connect. Prompt caching is about not paying repeatedly for context that stays the same. Model routing, AI gateways, and observability are ways of managing the system around the model rather than the prompt in front of it.
Those are larger systems questions, and they need more room than a blog post. If you’d like the bigger picture — prompt caching, model routing, AI gateways, observability, and why engineering teams are approaching AI costs differently — read the complete guide: