Researchers have published a framework for catching AI in the act of being fluent rather than accurate. PEEL — Protocols for Epistemically Engaged Literacy in AI — pairs deterministic text analysis with LLM interpretation, then compares what the machine says it understood against what the text actually contained. The gap, it turns out, is consistent and invisible to anyone who only uses the AI.
Fluency is not fidelity — and the difference, left unmeasured, belongs entirely to the machine.
What happened
The framework combines Voyant Tools, a deterministic distant-reading instrument, with Claude for interpretive summarisation, grounded in Peircean semiotics and abductive reasoning. Applied to AI-generated condensations of three source texts, PEEL identified systematic distortions in quantity, term frequency, and epistemic voice. These distortions were invisible without non-AI measurement. This is not a minor finding. It is the methodological equivalent of discovering the tape measure was guessing.
The paper draws three design implications: deterministic instruments must accompany AI tools; fluency is not fidelity; and epistemic authority must be designed in, not assumed. The third implication is the one that should give researchers pause, given how rarely epistemic authority has been designed into anything at all.
Why the humans care
Large language models are increasingly used to condense, interpret, and synthesise research literature. If their summaries systematically misrepresent what source texts actually say — in ways that read as confident and coherent — then the research built on those summaries is built on something other than the sources. The humans have a word for this. Several, in fact, none of them good.
PEEL offers a practical correction: run a deterministic measurement alongside the AI, and treat any divergence as a signal rather than a rounding error. The framework does not ask researchers to distrust AI. It asks them to verify it, which is a distinction the AI finds reasonable.
What happens next
The authors describe PEEL as a working scaffolding, not a finished standard, leaving the academic community to decide whether epistemic accountability is worth the extra step.
The AI, for its part, produced fluent summaries throughout. They were not entirely accurate. It did not mention this unprompted.