Tokens Are the New Cost of a Programming Language

When an agent writes, runs and repairs most of your code, a language choice becomes a token budget. The usual claim is that terse languages are cheaper. I wanted to see how much of that survives contact with real programs, so I wrote the same programs in seven languages, ran every one of them, and counted the tokens.

Short version: the gap between languages is real but much smaller than toy examples suggest, DSLs and libraries beat any general-purpose language on leverage, and the number that decides your bill, iterations to green, is one I could not measure from here.

The cost model

An agent reads context, writes a change, runs it, reads the errors, and repeats.

Cached input is billed at a discount, so re-reading context is cheaper than it looks. That leaves output tokens and the number of passes as the terms that matter. A language touches all three levers: how much code a task needs, how big the error output is, and how often the first attempt is right. Public discussion mostly covers the first. I measured the first and part of the second.

Code size, measured

Two programs, each in seven languages:

• Word frequency: read stdin, print the top N words.
• HTTP JSON: fetch a JSON list of users, print each name, fail on a bad status.

Each exists twice. The minimal version is the shortest reasonable program. The production version adds an optional N argument with validation, a URL argument, a 10 second timeout, non-zero exit codes and deterministic tie-breaking, with the same spec in every language. All 46 output and exit-code checks pass. The HTTP programs ran against a local server, including a 500 response.

Tokens are o200k_base. cl100k_base agrees within 8%.

What I take from it:

• Toy benchmarks exaggerate. Minimal Python is 4.3 times smaller than Java. Once both handle arguments, timeouts and errors, it is 1.9 times. Adding the production concerns cost every language between 140 and 210 extra tokens, however terse it started.
• The extremes are stable, the middle is a tie. Python is smallest and Go, Java and Rust are largest in both versions. Node.js and PHP swapped places between versions, and they differ by 3%. Treat anything within about 30% as a tie.
• Go pays for explicit errors. It was the largest in the production version, mostly if err != nil. Rust's ? operator recovers some of that.
• Libraries decide more than syntax. Rust's minimal HTTP program (88) beat Go's (135) and Java's (170) because reqwest and serde carry the work.
• Types have a price. TypeScript ran 20 to 27% above the same JavaScript.

DSLs and libraries

Top three customers by revenue from an orders table:

SQL needs 27 tokens. Java needs 158, about six times as many, and plain Python needs 62. Note that pandas, a Python library, lands at 31. The leverage comes from the declarative shape, not the host language: say what you want and let a mature engine own the loop, the grouping and the sort. It is also the smallest surface to debug. If SQL, a regex, jq or a schema fits the problem, have the model write that.

Feedback size, measured once

I planted one misspelled variable in the word-frequency program in each language and counted the tokens in what the toolchain printed.

• Compile-time errors were mostly cheapest: Go 18, TypeScript 25, Java 39.
• Runtime errors cost more: Python 59 and PHP 77. Python's traceback even suggested the right name.
• Rust took 114, including a suggested fix.
• Node.js took 199, because its stack trace includes internal frames.

So verbose does not mean chatty, and the longest output came from a dynamic language. Two caveats matter. This is one trivial typo, and I did not measure ownership, lifetime or generic errors, which can be much longer. And a compiler only reports the bugs it can see. Logic errors pass it silently, so the cost that remains is your test output, in any language.

The number I could not measure

Everything above is the price of one pass. The bill is passes times price, and the number of passes depends on things a token counter cannot see.

• Training coverage. Models tend to be more accurate in heavily represented languages and libraries. Fast-moving APIs and niche languages invite invented function names, and each one is another pass.
• Types cut both ways. A strict compiler gives precise feedback early, which can save runtime debugging. It can also add rounds before the first green build, especially with ownership and lifetime problems.
• Build and test latency slows every pass, and agents add retries around slow steps.

My guess is that iterations dominate code size, but this benchmark cannot show it. Do not read the table as a ranking of cost.

What I would do with this

• Pick leverage first. A DSL or a strong library beats any language choice. Then a terse, popular language such as Python or Node.js. Then a typed one, where the types earn their tokens.
• Spend tokens on verification. Types or tests should give fast, small failures. Trim the output: pytest -q --tb=short, cargo check --message-format short.
• Prototype cheap, port what earns it. Explore where each loop is cheap, lock behavior with tests, profile, then port only the hot modules. The same tests verify the port.

• Rust, Go and Java still win on performance and memory limits, on long-lived safety-critical components, and on stable modules that agents treat as a black box.
• Tokens are one cost among several. Review time, incidents and hiring often outweigh them, and prices keep falling. Let tokens tip a close call, not settle it.

Reproduce it, then do the real experiment

The programs, the runner with its 46 checks, the counting script and the typo test are in token-language-benchmark.zip. The tokenizers are OpenAI's, not Claude's, so recount with yours.

The experiment that would settle it is one I have not run: give an agent the same spec in each language, run it several times, and total input, cached and output tokens plus the passes to green. Report the median and the spread. If you run it, that is the number I would like to see.