In “We’re not taking the fact-checking powers of AI seriously enough. It’s past time to start.”, digital literacy expert Mike Caulfield researches how Grokipedia appears to have correctly fact-checked a claim that professional fact-checkers missed.

In the article, Caulfield demonstrates his 3000+ word “Deep Background” instruction prompt, a calibrated directive to guide an LLM through a detailed fact-checking exercise. He uses this prompt for a successful micro-investigation into a claim relating to the Nobel Prize ceremony and makes the case for an augmented approach to fact-checking.

But I remain convinced that there is no future of verification and contextualization that doesn’t involve both better understanding of LLMs and more efficacious use of them.

Caulfield also challenges “hallucination” as a flawed catch-all for the ways in which LLMs are inaccurate. Modern LLMs are more likely to be conflating through extrapolation or overweighting unreliable sources rather than purely fabricating.

His hotly contested stance is that disengagement is a mistake. One must engage with the technology in order to understand it.

My set of understandings led me to discover a substantial error by a news organization that, if search hasn’t failed me, seems to have been missed by everyone up until now. ⁠⁠What has “I can’t analyze the output because its [sic] meaningless fancy autocomplete” done for you?

I applaud Caulfield for raising awareness of the successful AI-assisted fact-check by Grokipedia despite feeling negatively about the site itself.

I was initially buoyed by Caulfield’s argument, but on reflection I think he still gives Grokipedia too much credit. I think we’ll need more than a single example of a fact-check slip made by humans and corrected by AI to make a case that this is revolutionary technology for fact-checkers.

After trying, failing and sharing my doomed efforts to port a Perl library over to TypeScript using AI tools, I eye enviously Emil Stenström’s account (via) of writing a HTML5 parser with a 17 point summary of the journey that concluded with a working implementation.

Some thoughts on where he succeeded where I didn’t:

• The models likely understand HTML5 more than bespoke, arcane routing algorithms tuned for ASCII diagrams.
• Stenström found a reliable way to keep the agents running in a loop, something I struggled with.
• He cites Gemini 3 Pro as pivotal for speed improvements, arriving at about the right time to give him a boost. This is similar to where I leaned into GPT-Codex-High.
• Building custom tools for fuzzing, profiling and scraping contributed to Stenström’s success. His 8,500+ passing tests are an order of magnitude higher than the piddly ~100 load-bearing tests that I assumed would suffice.

As per Dr Kate Compton:

A Bach Faucet is a situation where a generative system makes an endless supply of some content at or above the quality of some culturally-valued original, but the endless supply of it makes it no longer rare, and thus less valuable

She links a 15 year old article in the Guardian that recounts how Composer and Computer Scientist David Cope built “a little analytical engine” to generate thousands of original Bach chorales. With time, Cope built a successive tool (which he named Emily) to understand and ultimately emulate “the works of 36 composers.”

Cope will ask Emily a musical question, feeding in a phrase. Emily will respond with her own understanding of what happens next. Cope either accepts or declines the formula, much in the way he would if he was composing “conventionally”.

I can’t find any clear evidence of it after a short search but these releases were apparently met with dismay from the industry:

Critics convinced themselves that they heard no authentic humanity in it, no depth of feeling, Cope was characterised as a composer without a heart; his recent memoir is called Tin Man.

As noted in an Offscreen review of a 2021 documentary about Cope, he “takes pleasure but also tremendous inspiration and motivation in the public’s criticisms.”

“I want the negative reaction,” Cope professes, “I feed off of it. I keep going because of it. It’s mine and mine alone, and I love it.”

Cope mischievously and semi-ironically calls it “blasphemous music.”

And in an interview with Ryan Blitstein in the Pacific Standard, also from 2010, his emphatic position aligns with the pithy adage “good artists copy, great artists steal”:

“Nobody’s original,” Cope says. “We are what we eat, and in music, we are what we hear. What we do is look through history and listen to music. Everybody copies from everybody. The skill is in how large a fragment you choose to copy and how elegantly you can put them together.”

Cope felt pretty strongly about the tools he had created:

He can’t imagine the possibility of going back to writing with just his own intuition and a pen and paper. “The programs are just extensions of me. And why would I want to spend six months or a year to get to a solution that I can find in a morning? I have spent nearly 60 years of my life composing, half of it in traditional ways and half of it using technology. To go back would be like trying to dig a hole with your fingers after the shovel has been made, or walking to Phoenix when you can use a car.”

Another mention of a shovel caught my eye in the Pacific Standard piece:

“All the computer is is just an extension of me,” Cope says. “They’re nothing but wonderfully organized shovels. I wouldn’t give credit to the shovel for digging the hole. Would you?”

David Cope died at age 83 earlier this year.

“People tell me they don’t hear soul in the music,” he says. “When they do that, I pull out a page of notes and ask them to show me where the soul is. We like to think that what we hear is soul, but I think audience members put themselves down a lot in that respect. The feelings that we get from listening to music are something we produce, it’s not there in the notes. It comes from emotional insight in each of us, the music is just the trigger.”

Ammil is “the sparkle of morning sunlight through hoar-frost.” That exact moment when sun hits ice crystals on leaves on trees before they melt. Exists for minutes, requires witness, disappears.

Industries suggests a sense of scale, perpetuity, production without pause. No witness required. You can’t wrap your arms around industries, unlike a tree.

Ammil was a particularly obvious gem of a word in Robert Macfarlane’s Landmarks, an inquiry into how language shapes our perception of terrain, weather and place. Lost words gain life through use – if you use them, we use them, they’re not truly lost.

In a post-LLM world, everything can be generated. Text pours out at industrial pace. Links proliferate without providence, provenance unknown. Products multiply, bearing the uncredited fingerprints of millions.

ammil industries is a small-r research group exploring what remains authentic when everything can be synthetic.

We write, build tools, and create visualizations that trace the care, craft, and work required to persist.