development/AI modified

I Wasn't Actually Using Luna Everywhere

Devlog - Discovering that Claude Code was still using Sonnet in part of my workflow, tightening the model configuration, and learning to verify the model rather than assume it.

September 21, 20269 min read1648 words
A close-up of code on a monitor, representing the difference between the model I expected and the model actually running.

In my last post, I wrote that Luna was my default model. I believed that was true. I had chosen Luna deliberately, configured my workflow around it, and was trying to keep the model count low so that my token spending stayed predictable.

It turns out I was describing my intention, not the whole execution.

I recently discovered that Claude Code was still using Sonnet somewhere in the workflow. I had thought I had locked the CLI to Luna, but one part of the configuration was not doing what I assumed. I have now updated .claude/settings.json, and the behavior appears to match what I intended.

That small discovery changed the lesson from the previous post. Choosing a model is not enough. I also need to verify which model is actually handling the work.

The correction

I was not using Luna everywhere. Luna was the model I meant to use, but Sonnet was still being selected in part of the Claude Code workflow. I have updated the CLI settings and am treating the result as something to verify, not assume.

My assumption was too simple

The mental model I had was straightforward:

Choose Luna → start Claude Code → Luna handles the session

That model was incomplete. A CLI workflow can have more than one place where a model is selected. There can be a general default, settings at different scopes, and separate behavior for tasks that are delegated or launched through another part of the tool.

I was looking at the choice I had made and treating it as proof of the choice the CLI was making. Those are different things.

The important distinction is not specific to Luna or Sonnet. It applies to any tool that can select a model through configuration. A note in a project, a remembered command, or a setting I believe I changed is not an audit trail. The running workflow is the source of truth.

Why I noticed it now

The discovery came from revisiting the cost and model assumptions in my Godot workflow. My previous article compared the models I had intentionally tried and concluded that Luna gave me the best balance of speed, quality, and cost for the work I am doing.

That conclusion became less certain once I realized that every response might not have come from Luna.

The problem was not that Sonnet was unusable. It was that I was making a cost and quality comparison without knowing that the comparison was mixed. If I thought I was testing one model, but part of the work was being completed by another, then my notes were less precise than I believed.

That makes this a measurement problem before it becomes a model problem.

What I intended:
  Luna → consistent workflow → predictable cost

What was happening:
  Luna → some expected interactions
  Sonnet → another part of the workflow
  Result → unclear comparison

A good model can still produce a misleading experiment if I do not know when it is being used.

The setting I had missed

I had previously assumed that selecting Luna at the main CLI level meant that every relevant operation would inherit that choice. It did not. Something in the configuration still allowed Sonnet to be selected.

I am deliberately keeping the explanation at that level because the useful lesson is broader than one exact setting name. Configuration has scope. A value in one place may not override a more specific value somewhere else. A setting can be valid, visible, and correctly spelled while still not controlling the operation I care about.

The fix was to update .claude/settings.json so the intended model selection is explicit. After making the change, the CLI now appears to behave as expected in my tests.

The important word is appears. I do not want to replace one assumption with another. I need to keep checking the active model when the distinction matters.

What this changes about my cost experiment

My previous article argued that one fast model was enough for my current project and that I should not pay for extra delegation by default. I still agree with the direction, but I need to qualify the evidence.

The actual question is not only:

Which model do I prefer?

It is also:

Which model is being used for each part of the workflow, and what does that part cost?

That affects four things I care about:

  • Cost: the session may be charged at a different rate than I expected.
  • Speed: response time can change the feel of the development loop.
  • Quality: different models may make different assumptions or need different amounts of correction.
  • Learning: I may attribute a useful or frustrating behavior to the wrong model.

Without that information, it is easy to tell a convincing story about a test that was never controlled.

A better measurement rule

Before comparing models, confirm which model actually handled the request. Record the configuration and the observed result separately.

The difference between a default and a lock

I used the word “default” too confidently. A default means “use this unless something more specific changes the choice.” A lock means “this choice is explicit enough that another part of the workflow cannot silently replace it.”

Those are not the same guarantee.

For my current project, I want a simple setup:

One primary model
One clear configuration
No unexpected delegation
A quick way to verify the active model

That does not mean I will never use another model. Sonnet may be the right choice for a future task. The point is that I should choose it deliberately rather than discovering it after the fact.

This is also why I am less interested in making a large list of model settings and more interested in understanding where each setting applies. More configuration can create more confidence without creating more control.

What I am checking now

I have updated the settings, but the follow-up work is verification rather than another broad configuration exercise.

My checklist is short:

  1. Start a fresh Claude Code session.
  2. Confirm the model shown by the CLI or the relevant session information.
  3. Ask for a small, observable task rather than relying on a long conversation.
  4. Check whether delegated work follows the same model rule.
  5. Compare the result with the expected configuration.
  6. Record any exception before drawing conclusions about cost or quality.

A small task is useful here because it limits the number of variables. If I change the settings, the prompt, the project, and the amount of delegation at the same time, I will not know what fixed the problem.

The same inspect-before-edit habit I use in Godot applies to the AI tooling around it. Read the current state. Make one change. Run a focused test. Look at the result.

Why this matters for Godot work

The game project is still small enough that I can notice the effect of a model choice. I am moving between design notes, GDScript, scenes, MCP tools, and the running game. A model change can show up as a different amount of explanation, different assumptions about the project structure, or a different number of correction loops.

That is useful only if I know what I am observing.

If Luna helps me keep the loop moving, I want to know that the loop is actually using Luna. If Sonnet handles a delegated task, I want that to be a deliberate trade rather than an invisible part of the bill. If a response is unusually good or unusually frustrating, I want enough context to learn from it.

The goal is not to turn every coding session into a scientific experiment. It is to avoid drawing strong conclusions from a workflow I have not inspected.

The revised setup

This is the setup I am aiming for now:

Primary model: Luna
CLI configuration: explicit in .claude/settings.json
Project: Godot game prototype
MCP: enabled for project-aware work
Unexpected model switching: investigate before accepting
Priority: predictable behavior, then value, then speed

That is a more honest description than simply writing “Default model: Luna.” It says what I chose, where I configured it, and what still needs checking.

I am also revising my notes from the earlier experiment. I will keep the conclusion that Luna is currently the model I prefer for this workflow, but I will not present every earlier observation as a controlled comparison. Some of those sessions may have involved Sonnet without me realizing it.

The larger lesson

The surprising part is how easy it was to confuse configuration with reality. I am comfortable checking a Godot scene, reading an error, and testing a mechanic in the running game. I was less careful about checking the tool that was helping me do that work.

That is backwards. The assistant workflow is part of the system too.

A model name in a setting is not the same as a verified runtime choice. A lower token balance is not enough to explain what happened. A good response does not prove which model produced it. The useful evidence is the combination of configuration, observed behavior, and actual usage.

My rule has changed from:

Luna is my default.

To:

Luna is my intended model, and I need to verify that the workflow is honoring the choice.

That is a less tidy statement, but it is more accurate.

The earlier token usage experiment was about choosing a sustainable model and turning down delegation that I did not need. This follow-up is about the layer underneath that decision: making sure the tools are doing what I think they are doing.

I still want the same outcome. More budget should go toward building Rugby Runner, not toward invisible activity around it. The difference now is that I am checking the model path before I trust the conclusion.