Sand in the gears for humans-in-the-loop

The brain is an expensive tool to operate, and ao human biology is hardwired to take the path of least resistance when making decisions. As newsrooms consider the most efficient and ethical means to integrate AI workflows, we must design for the human-in-the-loop as much as the LLM.

Even when, as some argue, automation might be suitable to produce a first draft of written narratives from the routine and low-risk processing of structured data like sports scores or weather forecasts, having a human reviewer intervene before publication is no guarantee of quality or accuracy.

Because when a journalist is asked to "read and revise" AI-generated content, several cognitive biases turn a safety check into a potential rubber stamp.

1. Default Bias In behavioral economics, a "default" is the choice made absent active intervention. In an AI-journalism workflow, the AI’s draft is the default. To psychologically move away from a default choice demands switching costs of mental energy and time. Especially under tight deadline, the default AI text exerts a powerful gravitational pull. It is much easier to click "approve" than it is to delete a paragraph and rewrite it from scratch. Over time, the review process will almost inevitably shift from critical evaluation to more passive acceptance.
2. Automation Bias We tend to favor suggestions from automated decision-making systems, even when they may contradict our own better judgment.
   Because the AI produces text that is grammatically perfect and authoritative in tone, we subconsciously assume it is also factually accurate. This leads to "heedless oversight," where we may skim the text for flow but miss subtle "hallucinations" because the "truth-checking" part of our brain has been outsourced to the software.
3. Anchoring Bias The first information we see on a topic acts as an "anchor" that biases all subsequent evaluation. When a journalist writes from scratch, we weigh multiple sources, perspectives and interpretations. If we read an AI’s output first, even if errors are discovered and fixed, that initial text sets the boundaries for what the story "is." Instead of asking "What is the most important angle?" we ask "How can I fix this specific draft?"
4. The "Good Enough" Trap Because critical thinking is metabolically expensive, when reviewing an AI-written output, we experience cognitive ease because the heavy lifting of structure and syntax is already complete. This ease can be mistaken for "quality." If the an output looks like a news story and reads like a news story, the brain mistakenly signals that the job is done.

Any or all of these biases will lead to systemic mistakes, even when the reviewer has the best intentions. The challenges are not insurmountable, but each requires active measures to mitigate the risk.

To break through the default biases of a polished AI output, we have to move away from a simple "read and approve" and instead deploy Adversarial Review processes. These are techniques designed to introduce friction into the process, forcing the human brain out of autopilot.

Some examples:

1. The "Blind" Fact-Check (Anti-Anchoring)
   Instead of providing a full AI article to "check," the system first offers up the raw claims or data points the AI used - and then presents the draft narrative.
2. Mandatory "Red Teaming"
   In cybersecurity this is a group of troubleshooters who try to find weaknesses in a system. In journalism, their job would be to find reasons to kill / reframe / rewrite the story.
3. A 20% Deconstruction Policy
   To fight default bias and the path of least resistance, a newsroom would implement a hard rule: the reviewer must rewrite or revise at least 20% of the AI's output.
4. Reverse-Outlining
   AI is great at "vibes" but sometimes weak on "structure," so editors might perform a reverse outline by reading the AI draft and writing a one-sentence summary of the point of every paragraph to uncover structural flaws that skimming would miss.
5. Probability Flagging (Confidence Scores)
   LLMs can provide a mathematical measure of how "sure" it is about the next word selected in an auto-generated text. The system output might highlight words in red or yellow based on the AI’s own uncertainty, reducing the “illusion of authority.”

The goal of all of these methods is to throw “sand in the gears" of the default biases humans-in-the-loop will be prone to. Each intentionally slows down the workflow in order to assure the journalism remains as accurate as possible, even if supplemented by AI. None of these approaches are foolproof, and the higher the potential cost of failure (loss of trust, libel and etc) the less sense (or no sense) it makes to be considering automated processes in the first place. But, we live in interesting times.