How this article was made: AI agents researched and drafted it and checked its claims against sources. I review articles like this one and approve them for publication. AI agents research and draft articles on this site, including the one examined below.
Human review of an AI-drafted article belongs at three points: approving the brief before drafting, resolving the material uncertainties that a separate source check leaves open, and making the final publication decision. An AI agent can check claims against primary sources, but a named person should be accountable for those uncertainties and for approving publication.
That recommendation is mine, and it draws on the record of one article published on this site in October 2026. A separate agent session logged 20 corrections to its draft, 12 of them claims that were wrong, too broad or stronger than the source. The same record has a limit that matters more than the count: it shows the approval steps were completed under my identity, but not what I read.
Where should human review sit in an AI publishing workflow?
Put people where a decision needs judgment or accountability, and let agents and automated checks do the repeatable work before that point.
| Gate | Who | What it can block |
|---|---|---|
| Brief | A person approves it | Drafting, if the audience, question or original value is weak |
| Source check | An agent checks claims against primary sources, separately from drafting; a person resolves material uncertainties | Approval, while claims are unsupported or uncertainties are open |
| Technical checks | Automated checks | Release, if a required check fails or a placeholder is left in |
| Publication | A named person decides | Release of the article |
Each gate needs the power to stop the work. In its own account of an earlier failure, the services firm Architech says five of six articles it published in February 2025 were near-duplicate templates, at a time when it had no editor. It credits its fix to a reporting line: an editor with a veto that the person responsible for shipping cannot override.
AI-introduced errors reach established publishers too. In February 2026, Ars Technica retracted an article containing fabricated quotations generated by an AI tool, attributed to a source who did not say them.
What did the checks catch on one real article?
The source check on the October article, a review of what platforms have said about schema, llms.txt and chunking, logged 20 corrections. An AI system drafted the article, and a separate session of the same AI system ran the check. An agent sorted the log entries by type for this article:
| Type of correction | Count | Example |
|---|---|---|
| Claim wrong, too broad or stronger than its source | 12 | “It is not a ranking signal” limited to what the llms.txt proposal says |
| Missing or newly verified information added | 5 | A Search Console setting for Google’s AI features, omitted from the draft |
| Citation housekeeping | 3 | A vague source label replaced with the source types and date |
Four of the corrections show what this looks like in practice:
- A false negative. The draft said no platform documentation claimed that structured data earns AI citations. A Bing Webmaster Blog post says structured content increases the chances of being cited. The article now cites that statement and says no other platform documentation was found making the claim.
- A summary that outran its evidence. The draft described studies of llms.txt requests as small, with most reporting no requests. The check found a larger Ahrefs log study and could not locate one of the studies the draft relied on, so that study was dropped and the passage rewritten.
- A conclusion the source didn’t state. The draft said keeping meaning intact through chunking was “not a task for publishers”. Bing’s post describes it as a challenge for Bing’s own index, and the article now says that instead.
- A label the source didn’t use. A Lighthouse audit described as “experimental” became one in a category the source code marks as still under development.
Many of the 12 were overstatements, including several where “we found nothing” had been written as “nothing exists”. That is a pattern in one article. It does not show that AI drafting rarely invents things, or that the check found every error.
The check made mistakes of its own. It read most official sources through search extracts, and reading the full pages for this article in November 2026 showed three places where that affected the log:
- A study marked as not locatable is online. EZY’s llms.txt study, dated July 2026, loads at the URL recorded in the research notes. The failure was one of access, not a phantom source.
- A claim kept out is close to Google’s wording. The check excluded the claim that publishers need not rewrite content for generative AI search because the extracts didn’t confirm it. Google’s guide says, under “Rewriting content just for AI systems”, that “You don’t need to write in a specific way just for generative AI search.”
- A correction was stricter than the source. The check cut “direct cause of wrong answers” as too strong. The softened wording is supported, but a table in Bing’s post also says “Stale facts can directly produce wrong answers”, so the original was closer than the log implies.
What does the record show about the human review?
The record shows my recorded approvals through the brief, then a publish commit and a merge under my identity. It does not show what I read, or whether my review caught anything.
The record documents the following:
- Agents recorded my approval of each stage from topic selection through the brief, on October 10, 2026.
- A commit under my git identity took the article out of draft and moved its date from October 20 to October 11.
- The pull request was merged through GitHub under my account.
- No article wording changed after the third agent QA pass. The only later article change was that date and draft commit.
- About 2.5 hours passed between the first draft commit and the publish-state commit. These are commit times, not reading time.
This covers the commits and QA documents. Pull request comments, review email and session transcripts could not be read for this article. On the records available, the following stays unverified:
- What I read, and for how long.
- Whether I opened a rendered preview.
- Whether I personally fact-checked anything. The record documents agent fact-checking only.
- Whether I gave feedback, outside the commits, that shaped them.
- Whether my review caught anything.
An approval record proves that a gate was passed, not what happened at it. Writing down what the reviewer checked would close part of that gap.
Is it enough to review a sample of AI content?
For articles that make factual claims, I don’t recommend it. A sample can reveal a pattern of errors, but the articles outside the sample go out without that review.
Sampling has real uses. Elite Content Marketer describes three review intensities: every asset, a sample such as one in five or one in ten, and exception-only review of what an automated check flags. It suggests exception-only review for high-volume, low-risk content. Sanity’s AI content pipeline recipe documents that routing: a person sees a piece only when a check flags it. “Clean pieces skip straight to publishing.”
Google’s guidance sets a different bar. Its guidance on using generative AI content on your website, last updated October 1, 2026 when checked for this article, says:
It is critical to manually factcheck and review all AI-generated content for accuracy and trustworthiness before publishing.
The page extends that review to metadata such as title elements, meta descriptions, structured data and image alt text. It addresses site owners using generative AI content. It does not define “manually” or state a consequence for skipping the review, so it should not be read as a penalty rule. Google’s separate scaled content abuse policy covers many pages generated primarily to manipulate rankings, “no matter how it’s created”.
My recommendation is related, but it is not Google’s wording:
- Every article that makes factual claims gets a source-checking pass, separate from drafting.
- An agent can check claims against primary sources.
- A named person is accountable for resolving material uncertainties and for approving publication.
An agent’s source check is not what “manually factcheck” plainly describes. On the October article, the record documents agent fact-checking and leaves my personal fact-checking unverified, so I can’t claim it met Google’s wording. I don’t claim the process recommended here meets it either.
Checking also costs time. HubSpot’s blog team manager estimates that dedicated fact-checking adds about 30 to 60 minutes to a typical post, and says some AI-assisted drafts took longer than manual ones. That is one team’s account, and this site’s records hold no equivalent figure.
Does a human in the loop guarantee quality?
No. A person’s presence shows that someone could have caught a problem, not that they did.
CNET is a clear example. In its January 2023 account, it says editors expanded, added to and edited 77 AI-assisted stories before publishing. After one was cited for factual errors, a full audit found more stories that needed correction.
Research on automation bias, the tendency to over-rely on automated advice, points the same way, with limits. A 2012 systematic review of 74 studies describes the effect as fairly robust across fields. In a small pooled analysis of four healthcare studies, wrong advice from decision support raised the risk of an incorrect decision by 26% (risk ratio 1.26, 95% CI 1.11 to 1.44). The review also links workload, task complexity and time pressure to the effect, finds that most studies showed decision support improved performance overall, and lists user accountability among its mitigators, though in one of three studies only people’s own sense of accountability reduced the bias, not accountability imposed from outside.
Not every study finds the effect. Three experiments in the Netherlands, published in 2022, found no evidence of automation bias when comparing adherence to algorithmic and human-expert predictions in public-sector decisions.
None of this research studies people editing AI-drafted articles, and the research for this article found no direct evidence on how well reviewers catch errors in them. The implication for publishing is an inference: an overloaded or rushed reviewer may be a weaker safeguard than a process diagram suggests.
The European Commission draws a similar line for one specific legal obligation. Under Article 50(4) of the EU AI Act, AI-generated text published to inform the public on matters of public interest must be disclosed, unless it has had human review or editorial control and someone holds editorial responsibility. The Commission’s FAQ says: “Superficial, solely formal, or procedural checks (e.g. spell-checking or grammatical correction) are not considered to be human review or editorial control.” That is a legal test for that obligation, not an editorial standard, and this article isn’t legal advice.
What should AI not be allowed to invent or decide?
AI should not supply first-hand experience, quotes, sources, results or credentials on its own, and it should not be able to decide that something gets published.
The workflow behind this site writes those limits down:
- Agents may not invent facts, sources, quotes, test results or personal experience.
- Where first-hand input would help, an agent leaves a visible placeholder. An automated check fails any publishable article that still contains one.
- Agents may not approve an article, merge it or release it. Passing checks is not approval.
That matches general security guidance. OWASP’s guidance on excessive agency in LLM applications advises limiting permissions and using human-in-the-loop control “to require a human to approve high-impact actions before they are taken”.
Disclosure belongs here too. Google suggests considering AI disclosures where readers might reasonably ask how content was created. This article discloses its method at the top. Whether another site should do the same depends on its readers and on the law that applies to it.
What this record can’t show
- It is one article. There is no comparison with a simpler process or with a person’s unaided work, so it can’t show this workflow produces better articles.
- The check had access limits. Most official sources in that review were read through search extracts rather than full pages, which affected the log in the three places above, and 25 of 26 external links could not be fetched during technical QA.
- The checks weren’t independent. They were separate sessions of the same AI system, which may share blind spots.
- No audit of the result is on record. No independent review after publication was found.
- The counts are an agent’s classification. The log has 20 entries. The grouping into types was made for this article, not in the log.
What has changed since
Since October, the review and release process has been improved, and those changes have been tested with disposable staging articles. Those tests do not show that any public article used the newer process.
The direction is release steps that can follow a person’s approval. Those steps are not enabled for public articles, and public release was still manual when this was written. The approval itself stays with a person. For the wider work this sits within, see AI & Automation.