Better Models Worse Tools, Fable's $149 Final Review & the AGENTS.md Purge

Better Models, Worse Tools

Friday's phantom-tool-parameter mystery got solved, and the answer is worse than a bug. Armin Ronacher: "I had some vibes that Opus 4.8 was performing worse than older ones for some uses that are off distribution and now I have the receipts." The write-up, Better Models: Worse Tools, hit the HN front page (149 points) within hours.

The finding: Opus 4.8 and Sonnet 5 call pi's edit tool with invented trailing fields in the nested edits[] array — a whole zoo of them (requireUnique, matchCase, oldText2, even event.0.additionalProperties) — while none of the older models do. The actual oldText/newText payloads are byte-correct; the model then appends nonsense at exactly the highest-entropy point (right after closing a several-hundred-token escaped string). It's heavily context-dependent: fresh prompts never reproduce it, one user's resumed session failed ~20% of the time, stripping thinking blocks halved the failure rate, and strict tool mode eliminated it.

His hypothesis is the interesting part. Digging through Claude Code's minified client he found it's a slop-absorbing harness: parameter aliases (old_str, path, etc.), Unicode escape repair, silent filtering of unknown keys, retry state machines for leaked <invoke markup. If RL happens inside a harness like that, malformed tool calls still complete the task and get rewarded — "there is little gradient against inventing an alias, adding a stray field or using a nearby parameter name." And the better-trained model's prior is stronger: "Alternative tool schemas might not just be unfamiliar. They might be implicitly punished by post-training that optimizes for one particular, forgiving tool ecology. And that ecology is not documented." (He deliberately didn't test Fable, unsure whether the classifiers would silently downgrade him to Opus — a replier reported clean refusals instead.)

The thread sharpened the stakes:

Armin notes the general topic was already in his PyAI talk this year (video), and pi is now planning cadenced model-compatibility testing.

Fable's $149.25 Final Review

Simon Willison handed the stop-reading-code debate its best receipt yet: "Somewhat humbling to have Claude Fable do a final review of some software that you're about to release and have it then find (and fix) FIVE release blockers, for an estimated (unsubsidized) cost of $149.25." Full write-up: sqlite-utils 4.0rc2, mostly written by Claude Fable.

The prompt was one line — "Final review before shipping a stable 4.0 release — very important to spot any last minute things that would be a breaking change if we fix them later" — sent from Claude Code for web on his iPhone, with Fable churning while he attended the Half Moon Bay 4th of July parade. The worst blocker was genuine data loss: delete_where() ran outside the atomic() wrapper, leaving the connection permanently in-transaction so every subsequent write silently evaporated. The cleanup ran to 37 prompts, 34 commits, and +1,321/−190 across 30 files.

Two details worth stealing:

  • Cross-vendor review is now habit, not superstition. "I used to think that the idea of having one model review the work of another was somewhat absurd... The problem is it really does work." A one-line Codex Desktop / GPT-5.5 xhigh review of the RC diff found two more P1 transaction bugs; he pasted them into a fresh Fable session which experimentally confirmed both and fixed them.
  • Read the docs diff first. "Reviewing the documentation edits first is an excellent way to build an initial understanding of what has changed" — that's how he spotted the Python 3.12 autocommit incompatibility Fable had quietly documented.

The replies did the economics: $30 per blocker, a reviewer with no incentive to rubber-stamp as a new price point, and the sharpest reframing of the week — "'Do you ship code without reading it?' should really be 'Do you ship code without AI reading it?'" Asked to extrapolate to a datasette-sized project, Simon ballparked 4–5 figures — and noted year-2000 SLOC models price datasette at $2.1M and 15.7 developer-years. Also worth a chuckle: the same model wrote most of the code and then caught five of its own blockers on the second pass.

The AGENTS.md Purge

Matt Pocock, quote-tweeting "AGENTS.md/CLAUDE.md is largely an anti-pattern": "It is an essential primitive, but misused by nearly everyone" (143K views). Pressed to be less vague, he delivered the line of the day: "90% of people can delete their AGENTS.md and their output will improve. Use it as an index of pointers to important files, and even then aggressively prune it for no-ops."

The replies converged hard on the same doctrine:

This lands as the counterweight to a week of context-engineering maximalism — the same author whose /wayfinder skill orchestrates hundred-session planning maps is telling you the entry-point file should be nearly empty.

Theo's Phone-Only Weekend & Fable's iOS Gap

Theo spent the 4th shipping from bed: "T3 Code Mobile app is getting way too good way faster than I expected. It's screwing with my sleep. I just prompted for hours without touching my laptop" — including a first thread built entirely from his phone. Input method is nothing fancy ("Just typing on phone. Apple's built in voice to text is fine"), an onboarding overhaul is "coming very soon," and the replies are full of people wanting Android builds and iPad treatment. This is the same convergence Cursor for iOS and Claude Code for web started the week before — every harness is racing to make the laptop optional, and per Simon's parade story above, the workflow is real.

The counterpoint from the same evening: "Fable's intuition for iOS development is significantly worse than other areas I've used it in. Great at infra, databases, web and more, just easily confused about how mobile apps work for some reason" (62K views, 105 replies). The dominant theory in the replies is training-data scarcity — there just isn't much good open-source iOS code compared to Android — with corroboration from SwiftUI users ("amazing, but suboptimal with Swift + SwiftUI"), a claim that macOS Swift is somehow fine, plenty of "opposite experience here" dissent, and the summary nobody asked for: "fable's just an android developer, dw about it." Note it echoes Armin's off-distribution thesis from a different angle: the models are increasingly shaped by what their training saw most.

Agentic Retrieval Grows a Harness

Jerry Liu announced a Retrieval Harness for agentic retrieval: a persistent pipeline that connects to a data source, indexes and updates a large knowledge base, and exposes it to any agent as filesystem-shaped tools — semantic/keyword search, regex grep, file find, read. The reference implementation is legal-kb (project-scoped KBs, visual citations in responses, version control for the knowledge base itself), built on LlamaIndex's Index v2.

The framing that stuck, per the replies: "hybrid search across the whole index with read, grep, and find is basically giving an agent the same tools a developer would use on a local filesystem." That's the same design conclusion Claude Code reached from the code side — agents navigate with search-and-read loops, not top-k similarity dumps — now packaged for documents. It also completes an arc Jerry flagged himself this week: three years after his "Advanced RAG" talk at the first AI Engineer conference, the workarounds have consolidated into harness-level abstractions. (Launch hiccup: the repo wasn't public for the first few hours — "sorry about the above — just made it public!")

Also Worth a Look