Back to Blog
article·

RTK says I saved 3.5M tokens in the last 3 months. Really? How?

rtkclaude codetokensclillm

I've been using RTK for some months now. Quick pitch if you haven't seen it: it's a CLI proxy that sits between your coding agent and your terminal. RTK reduces the outputs of console commands, so LLMs have to read less, use less context and, in consequence, burn fewer tokens. When Claude runs git status or jest or eslint, RTK intercepts the output, strips the noise (ANSI codes, progress bars, blank lines, the 400 lines of jest output that say nothing) and hands the agent a compact version instead. Simple idea, works great.

The other day I ran rtk gain and saw this:

RTK Token Savings (Global Scope)
════════════════════════════════════════════════════════════

Total commands:    6819
Input tokens:      6.0M
Output tokens:     2.5M
Tokens saved:      3.5M (58.1%)
Total exec time:   168m20s (avg 1.5s)
Efficiency meter: ██████████████░░░░░░░░░░ 58.1%

By Command
────────────────────────────────────────────────────
  #  Command             Count   Saved    Avg%
────────────────────────────────────────────────────
 1.  rtk jest run           89    1.1M   97.8%
 2.  rtk:toml ps aux         4  437.8K   99.1%
 3.  rtk ls -la ios          3  349.8K   70.2%
 4.  rtk git stash show      4  288.3K   86.2%
 5.  rtk grep              806  125.9K   18.3%
 ...

Over 3 million tokens saved. Nice. And then I thought: okay, but how does RTK put this number together? How does it know I've saved that many tokens?

It's a local CLI. It never talks to a model. So are they just counting each space they "save" from the original command output and counting that as saved? Because if that's the case, that number actually represents the amount of context tokens saved, not the tokens I've avoided to burn. Do they know the conversion rate from input tokens saved to token burnage avoided? Is there even a known conversion rate from input tokens to burned tokens? Does it vary with models?

All those questions made me investigate a bit more, to find out if I had really saved 3 million tokens or what.

Opening the box

RTK keeps a SQLite database with one row per proxied command (~/Library/Application Support/rtk/history.db on macOS). The schema tells you most of the story already:

CREATE TABLE commands (
    ...
    input_tokens INTEGER NOT NULL,   -- the raw command output
    output_tokens INTEGER NOT NULL,  -- what RTK actually printed
    saved_tokens INTEGER NOT NULL,   -- the diff
    ...
);

rtk gain just sums the saved_tokens column. So the real question is how those per-command numbers get computed. RTK is open source (Apache-2.0), so I pulled the source for the exact version I had installed and found this in src/core/tracking.rs:

/// `tokens = ceil(chars / 4)`
pub fn estimate_tokens(text: &str) -> usize {
    (text.len() as f64 / 4.0).ceil() as usize
}

And the "saved" math:

let saved = input_tokens.saturating_sub(output_tokens);

That's it. That's the whole thing. Bytes divided by four. No tokenizer, no model awareness. RTK captures the raw output, captures the filtered output, divides both byte counts by 4, and calls the difference "tokens saved."

So yes, my suspicion was basically right: they're counting each character they strip (spaces included) as a quarter of a token. The 4-chars-per-token rule is a decent average for English prose, but terminal output is full of paths, hashes and punctuation, which real tokenizers chew into more tokens (closer to ~3 chars each). So the estimate is loose, and it's loose in both directions depending on what got stripped.

So, is there a conversion rate from input tokens to burned tokens?

Short answer: no. There's no single conversion rate, and yes, it varies with models. But the two numbers are more connected than it might seem. The relationship just pulls in both directions at once.

What RTK measures is tokens that never entered the context window. Command output that Claude would have had to read, and didn't. What it does not measure is "burned tokens avoided" in any billing sense. Here's why the gap exists:

Where the counter understates the savings: conversation history gets re-sent on every API request. That 30k-token wall of jest output doesn't cost you once. It rides along in the context and gets re-billed on every following turn until the session compacts. So a one-shot per-command diff actually undercounts the lifetime cost of the junk it removed. Prompt caching softens this (cached input re-reads cost ~10% of base price), but it doesn't erase it.

Where it overstates them: the chars/4 heuristic isn't your model's tokenizer, and every model tokenizes differently, so the same stripped bytes are a different token count depending on what you're running. And critically, RTK does nothing about output tokens, the ones the model generates, which are the expensive ones (~5x input price on Claude models). RTK only touches the input side.

So the honest label for that 3.5M isn't "tokens saved". It's "an estimate of tool output kept out of my context window, expressed in approximate tokens." Less catchy, I get it.

The important take

Even though the "3 million saved tokens" actually means 3 million tokens that were not put in context, and not 3 million burned tokens saved, we can factually say that using RTK does indeed save tokens.

Tool output kept out of the context window pays off three ways:

  • Cost: those tokens would've been billed as input, on every turn, for the rest of the session.
  • Speed: smaller context means faster requests. Everything is just slightly snappier, and over a whole session it's noticeable.
  • Quality: this one's underrated. The model reasons better over 5k tokens of signal than 50k tokens of jest noise, and it hits compaction later. Less junk in context isn't just cheaper, it's smarter.

How much does it save, exactly? I don't know. Nobody does, not even RTK. The real number depends on your model's tokenizer, how long your sessions run, and how much of that context was cache-discounted. Could be less than the counter says, could honestly be more once you count the re-send multiplier.

But it does save tokens, and it makes everything slightly faster. With that, I'm happy enough.

Share this post