Peggy Wu's Life Lab | Rethinking AI Costs After Working with Claude Code (Part 1)

 

From “Which Model Is the Cheapest?” to Rethinking Tokens, Context, and AI Attention

Recently, I’ve had the chance to listen to several really talented colleagues share their hands-on experiences with AI agents, context engineering, and Claude Code.

Along the way, I also spent some time asking Claude Code questions, digging through documentation and articles, and experimenting with different ways of working.

Little by little, I realized that the way I think about AI costs today is actually quite different from where I started.

The AI world moves incredibly fast. What feels like a best practice today might look completely different a few months from now.

But that’s also what makes it interesting.

Every once in a while, I think it’s worth pausing to write down what I’ve learned, what I’ve tried, and maybe more importantly is how my thinking has changed.

My original question about AI efficiency was actually very simple:

Which model is the cheapest to use?

There are so many models now, all with different input and output token prices.

So my first instinct was pretty straightforward:

Just pick a cheaper model. Problem solved, right?

Well… after digging into this for a while, I realized it’s not quite that simple.

What affects AI cost and efficiency isn’t just which model you use.

A lot of it comes down to:

How much stuff are you asking the model to process, and how are you asking it to work?

So before talking about optimization, let’s first look at where AI costs actually come from.

AI Cost Is More Than Just Model Pricing

I roughly think about AI costs in three buckets.

1. Input: How much are you giving the AI?

Code, API responses, documents, logs, conversation history, all of these eventually become tokens.

And in real-world workflows, the amount of input can be much larger than we realize.

We might see the AI produce a 500-token answer.

But before writing those 500 tokens, it may have already processed tens of thousands of tokens behind the scenes.

2. Output: How much does the AI generate?

For Claude models, output tokens are generally significantly more expensive than input tokens—roughly around 5x for some models.

So if I ask:

“Please generate an extremely detailed, comprehensive report covering absolutely everything.”

Well, that output has a very direct cost. 😅

There’s another important difference between input and output:

Generated output can’t benefit from prompt caching in the same way reusable input can.

New output has to be generated and paid for each time.

That said, in many coding workflows, the amount of output is still much smaller than the amount of input being processed.

3. The “fixed costs” you may pay every session

This was something I hadn’t paid much attention to before.

Things like:

• CLAUDE.md

• Memory

• Tool schemas

• System instructions

• MCP tool definitions

Some of these may enter the context every session, or even repeatedly during a conversation.

I think of them a little bit like transaction fees.

You start a session, and there’s already some token cost before you’ve really started doing anything.

If this layer gets too heavy, those costs add up.

Once I realized this, I immediately went back and put my Memory and CLAUDE.md on a little diet. 😂

A Quick Note on Prompt Caching

Another interesting thing I learned is that not every input token necessarily costs the same every time.

If a large block of input:

• appears repeatedly,

• stays unchanged,

• and is reused within the cache window,

then prompt caching can significantly reduce the cost of processing it again.

Relatively stable content like CLAUDE.md or Memory can potentially benefit from this when structured properly.

Newly generated output, on the other hand, doesn’t have the same advantage.

That’s when I realized cost optimization was getting a little more interesting than I originally thought.

It’s not enough to compare:

Which model has the lowest price per million tokens?

You also have to ask:

Where are my tokens actually going?

 

The Bigger Realization: More Context Isn’t Always Better

After using Claude Code for a while, there were definitely sessions where I wanted to tell Claude to take some ginkgo supplements because its memory seemed… not great. 😂

(Although apparently ginkgo doesn’t really help memory anyway, so never mind.)

At first, I didn’t think too deeply about it.

If something was really important, I’d put it into Memory or turn it into a Skill and move on.

Then one of my colleagues mentioned “Lost in the Middle,” and suddenly a few things clicked.

Lost in the Middle describes a common behavior in large language models:

LLMs often handle information near the beginning and end of a long context better than information buried somewhere in the middle.

So as context gets bigger and bigger, agents can start showing some very familiar—and occasionally eye-roll-inducing—symptoms:

• Forgetting the original acceptance criteria

• Continuing to follow assumptions that are no longer valid

• Fixing A and accidentally breaking B

• Re-investigating something that was already confirmed earlier

• Acting like something you explained ten minutes ago is completely new information

In the past, my reaction was usually something like:

Claude is getting dumb again. 🙄

Now, after the eye roll, I try to ask a different question:

Is my context simply getting too large and too messy?

That small shift in thinking has actually been pretty useful.

Practical Tip: How I Manage Context in Claude Code

Claude Code has a built-in /context command that lets you see how much context you’re currently using and how it’s being allocated.

You can also keep an eye on the Status Line at the bottom of the terminal for a quick view of context usage.

I’ve developed a very simple personal rule:

Once my context gets to around 70%, I start thinking about cleanup.

This isn’t some magical 70% industry standard—it’s simply the threshold that currently works well for me.

If I’m still working on the same task, I’ll usually use /compact to compress and summarize the conversation history before continuing.

If I want to try a different approach or explore a branch while keeping all the history and context I’ve already built up, I’ll use /fork.

And if I’m moving on to a completely different topic?

I just start a new session.

My mental model is basically:

Same task, keep going → /compact

Same context, explore another path → /fork

Different task → New Session

It’s a tiny habit, but I’ve found it surprisingly useful.

I used to keep one session going forever.

Now I’m much more likely to stop and ask:

Do I actually need all of this context for what I’m doing next?

Instead of waiting until the context is almost full and Claude starts “forgetting things” before I finally clean it up. 😂

Context Window Is Capacity, Not Quality

These days, I don’t look only at:

How much context do I have left?

Because having room left in the context window doesn’t mean the context itself is good.

If it’s filled with outdated discussions, irrelevant files, old assumptions, or information I no longer need, all of that is still competing with the important information for the model’s attention.

So I’ve started thinking about context differently:

The problem isn’t just the number of tokens. It’s the signal-to-noise ratio.

The goal is to preserve the model’s limited attention for the things that actually matter.

Tools Eat Context Too

Another thing I hadn’t really thought about before was the cost of tools themselves.

I used to have a pretty natural assumption:

More MCP servers = a more capable agent.

But tool definitions also take up context.

Anthropic shared an interesting example: when an AI agent was connected to multiple services such as GitHub, Slack, and Sentry, 58 tool definitions alone consumed around 55K tokens.

By switching to on-demand tool search, the relevant context dropped from roughly 77K tokens to 8.7K tokens—an 85% reduction.

That number really stuck with me.

The agent hasn’t even started working yet.

You’re just telling it:

“Here are all the tools you can use.”

And you’ve already spent a pretty significant amount of context.

There’s another downside to having too many tools available at once:

The agent is also more likely to choose the wrong one.

So more tools don’t automatically mean a better agent.

Stop Thinking Only About “Saving Tokens”

At this point, I started realizing that asking:

Which model is cheaper?

only optimizes one small piece of the entire AI workflow.

There are many other things that affect both cost and efficiency:

• What goes into the context

• What gets loaded every session

• How tools are connected and designed

• How large amounts of data enter the context

• How long conversation history is allowed to grow

• Whether tasks are properly separated

• Which repetitive tasks should become scripts

• Which tasks actually need an agent in the first place

And the interesting part is:

These things don’t just affect cost. They directly affect the quality of the AI’s work.

Sometimes saving tokens and improving AI quality are actually the same thing.

Because what you’re removing is often just noise.

Rethinking Context Engineering Through WSCI

In 2025, LangChain discussed four major strategies for managing context in AI agents:

Write / Select / Compress / Isolate

Sometimes summarized as WSCI.

After learning more about token costs, context behavior, and context rot, these four ideas suddenly felt much more practical to me.

I tend to translate them into slightly different words when thinking about my own Claude Code workflow:

Persist — What’s worth keeping?

What information is stable, important, and likely to be useful again?

That’s the kind of information worth putting into CLAUDE.md, Memory, or another persistent location.

Retrieve — What should I fetch only when I need it?

Instead of putting everything into context upfront, search for and retrieve information when it actually becomes relevant.

Compress — What should be summarized?

As context grows, which parts of the history no longer need to be preserved in full?

Maybe all I really need is the conclusion, the decision, and the next step.

Isolate — What should happen somewhere else?

Some exploration tasks involve reading a huge number of files and generating lots of intermediate information, while the main agent only needs the final result.

In those cases, it can be cleaner to move the work into another agent or an isolated context and bring back only a high-signal summary.

After reading about these ideas and experimenting with them in Claude Code, I realized that many techniques that initially looked unrelated actually fit nicely into these four categories.

For example:

• I started cleaning up my CLAUDE.md and Memory, moving out information that doesn’t need to be loaded every session.

• For large logs, JSON files, or CI output, I stopped dumping everything into the AI. I search and filter first, then bring only the relevant information into context.

• I started using subagents for broader exploration and bringing only the conclusions back into the main conversation.

• Repetitive Jira / Confluence / curl workflows are gradually becoming scripts instead of something I ask an agent to rediscover every time.

Even my coding workflow has slowly changed from:

Think of something → Ask something → Change something → Repeat

to something much more intentional:

Explore → Plan → Implement → Verify → Handoff

All of these changes are really doing the same thing:

Don’t make every piece of information, every task, and every bit of history compete inside the same AI context.

 

So What Am I Actually Trying to Optimize?

Looking back, one of the most interesting parts of this learning process is how my questions kept changing.

I started with:

Which model is the cheapest?

Then:

How do I save tokens?

Then:

How do I manage context?

And now I find myself thinking about a much more interesting question:

How do I design a better working environment for AI?

Because AI doesn’t necessarily perform better just because it knows more.

In that sense, it’s actually pretty similar to us.

Imagine sitting at a desk with 30 documents spread out in front of you, 10 things on your to-do list, yesterday’s unfinished work still sitting there, and 58 different tools next to you that you have to choose from.

I’m pretty sure most of us wouldn’t be doing our best work either. 😂

So after going down this rabbit hole, this is the idea I keep coming back to:

What we need to manage isn’t just tokens. It’s AI’s attention.

Good context engineering, at least in the way I currently understand it, is largely about doing exactly that:

Putting the right information in front of the AI at the right time.

So… how do we actually do that?

I’ll save that for Part 2.

I want to spend some time doing the experiments and adjustments I’ve been making lately—how I’m applying Persist / Retrieve / Compress / Isolate in Claude Code, what I’ve changed in CLAUDE.md and Memory, how I’m using /compact, /fork, subagents and scripts, and how my coding workflow has gradually evolved along the way.

A lot of this is still experimental for me.

But after actually getting my hands dirty and changing things around, I’ve already found a few pretty interesting lessons worth sharing.