Fable's Hallucination Check, the Seven-Point Read-Code Scale & the Grammar Gap

Fable 5 Reality Checks

Matt Pocock: two hallucinations in ten minutes. The post (1.5K likes, 136K views): "10 minutes using Fable, 2 factuality-based hallucinations already. Never trust an LLM." The specifics, per his follow-up: misunderstanding the OAuth requirements of X API endpoints, and not knowing the X API got cheaper — "essentially, same mistakes I'd expect from Opus/Sonnet" (context was only 40K tokens, so this isn't a rot problem). After a day of "skill issue" drive-bys he sharpened it: "folks who think Fable fixes all hallucinations are talking out of their arse. It's prone to exactly the same failure modes." He also took the maximalist definitional stance — "If the model answers something wrong, it's a hallucination — whether it's before or after the cutoff" — which drew its own sub-argument. Corroborating reply worth keeping: Anders Jensen-Waud had Codex fact-check a Fable research pass with browser lookups — 4 of 30 references didn't exist. And the "it felt like AGI at launch, now it's Opus-level, they must be throttling it" narrative is alive in the replies, a week after the export-control outage made everyone paranoid about silent nerfs.

Theo pins "You've been lied to about Fable." The video post is his counter-programming to exactly the discourse above: the claims that Fable is nerfed, over-censored, and terrible at coding. The replies are a neat split-screen of the two realities — a dev whose years-old tech debt got cleared in a weekend, with Fable writing maestro tests to screenshot an iOS simulator and validate its own work, versus people citing "an obscene amount" of routine-request refusals on the website and the note that Anthropic's data-retention policy is blocking enterprise adoption. One replier splits the difference cleanly: "nerfed and terrible at coding don't usually go together — one's a safety tradeoff, the other's a capability claim." Timing matters: the July 7th subscription cliff is tomorrow — Fable comes off Pro/Max plans per Anthropic's capacity plan, and the replies are counting down the hours.

The Read-Code Scale

Matt Pocock turns the binary into a taxonomy. The scale post (1.8K likes, 192K views) reframes the week-old "should you read code" fight: (1) read every line of every diff → (2) scan diffs, review important lines → (3) skip diffs but understand the 'why' of every PR → (4) spot-check PRs → (5) skip PRs, spot-check the codebase → (6) ignore code, spot-check agent traces to improve the system → (7) ignore everything, models handle it. His own position: "Mostly I'm at 4 and 5, but on higher stakes projects I would go further up the scale, though rarely all the way to 1." Two load-bearing assumptions surface in replies: he assumes an automated reviewer runs before human review at every level, and nearly every serious answer is stakes-conditional — work 3–4 / personal 5–6, a team that's lived at 5–6 for months and only drops to 4 "when a trace doesn't make sense and you need the code to explain it", prototypes at 7 but "the first exemplar of a pattern" at 1.

Theo's corollary: review less, verify more. The post (93K views): "You should be reviewing a MUCH smaller % of your code today than you were 5 years ago. If your code is so important it needs every line verified, you better be writing a LOT of slop that verifies it too." The follow-up makes it and-not-or: code important enough for hand review is also important enough for thousands of lines of verification slop. Best concrete framing in the replies is a selective-review list — not reviewing tests, frontend, one-off scripts, isolated services; still reviewing core, database schemas, service contracts — plus one dev's shift to reviewing "data flow, boundaries/contracts and failure modes" over implementation. The receipts crowd noticed the drift: "So we go from 'why are you even reading code?' to 'a % of code needs to be reviewed' in about two days." His dedicated video on the topic — the one he promised would piss both sides off — is still in the pipeline.

Agentic Coding & Agent Harnesses

No one matches OpenAI's grammar-constrained tool calls. Armin Ronacher, straight off last week's Opus-edit-tool failure receipts, went looking for providers that let you constrain tool-call output with a custom grammar the way OpenAI does for custom tools. Answer from the thread: nobody. Hosted APIs mostly stop at JSON Schema for tool calls; arbitrary grammars remain vLLM/Outlines/llama.cpp territory "because custom masks mess with batching and spec decode", and flatly: "Anthropic still does not do grammar constrained sampling." On the "doesn't it hurt quality?" objection, Armin's take: true for the small local models he's tested, but "for very large models the story might be different." Also from Armin: pi now has "robobun at home" — hosted-style analysis runs that you can then continue as a local pi session, his answer to the analyze-remotely-finish-locally workflow.

/wayfinder produces its first artifact: a fully-cited PRD. Matt Pocock published the PRD from a multi-day /wayfinder session and the pitch is provenance: "every assertion is linked back to the session where it was decided. Secondary source → Primary source." The workflow: one session builds the initial map, then you attack as many tickets in parallel as possible — "those tickets spawn more tickets, then spawn more tickets, until you resolve all open decisions and you're ready to turn it into a PRD." The interesting pushback (echoed by several users): wayfinder makes decisions for you where /grill-me would have asked first — the orchestrator trades being-in-the-loop for throughput.

Jerry Liu: file parsing belongs in the agent loop. His pitch: agent queries with file attachments are growing exponentially, and the default behavior of every agent loop — a light pypdf/pdftotext pass before falling back to vision — "leads to hallucinated context and bad retrieval," plus wasted turns re-writing import/postprocessing code from scratch. LiteParse is now installable as a one-click skill for any agent, with a template for Vercel's new Eve agent framework. Follows his Retrieval Harness from two days ago — LlamaIndex is clearly repositioning its whole stack as agent-loop infrastructure.

Quick Hits