Microsoft PM Keiji Kanazawa Let Codex and Claude Refactor His Code for 3 Weeks. Then the Token Bill Arrived. — BigGo Finance

Microsoft PM Keiji Kanazawa Let Codex and Claude Refactor His Code for 3 Weeks. Then the Token Bill Arrived. — BigGo Finance

By BigGo Finance
Publication Date: 2026-11-09 05:08:00

The failure that Keiji Kanazawa remembers most vividly wasn’t a crash. It was a sentence in the wrong place.

A digest his agent system produced had misattributed a line to the wrong section. Doing what he called proper error analysis, he asked Claude Code where the traces were for that session. The answer: there were none, because the thing he’d been calling an agent wasn’t one — “it’s not really an agent,” the tool told him, according to his account on the AI Engineer podcast.

What he found when he read deeper is a warning for anyone who has handed a coding agent a large project and trusted the labels it applied: “They have these plausible names for these constructs in the system, but they weren’t what they say they are. A skill was not really an agent skill. It had no skill.md. It had no scripts.”

Three weeks of refactoring produced architecture that only looked real

The project started small — a personal podcast pipeline that downloaded, transcribed, and summarized episodes…