← Content Evaluation
Content Evaluation

Analysis workbench

Choose content
Preview ready
Text under review

demo-content/before-you-publish-an-ai-assisted-post.md

Before You Publish an AI-Assisted Post

AI can help you draft faster. It can also make a weak article look finished before it deserves to be published.
That is why a final review step matters.
If a post was written with AI help, I think there are five questions worth asking before it goes live.

1. Which claims would be embarrassing to get wrong?

Do not try to verify everything equally. Start with the claims that create real risk: numbers, product details, legal statements, health advice, timelines, and any sentence that sounds more certain than the surrounding evidence.
Many teams now use AI across most of their publishing workflow, which means small factual errors can spread much faster than they used to. The faster the drafting gets, the more deliberate the checking has to become.

2. Does the source actually support the sentence beside it?

A real link is not the same thing as support.
Sometimes a source is outdated. Sometimes it is relevant but weaker than the claim being made. Sometimes the article says one thing and the linked report says something narrower. This is one of the easiest ways for an AI-assisted draft to look credible while still being wrong in an important way.
Google's guidance on AI content is useful here because it basically says the production method matters less than whether the page is helpful and trustworthy. If that is the standard, then source checking cannot be optional.

3. Does the piece sound like a person with a point of view?

AI-generated writing often has a recognizable style: balanced, clear, and a little interchangeable. Even strong drafts can flatten out when every section sounds equally reasonable and equally safe.
That does not mean every smooth paragraph is machine-written. It does mean you should look for places where the article stops sounding argued and starts sounding assembled.
A simple test helps: which line in the piece could only have been written by this author for this audience? If there is no answer, the article probably needs another pass.

4. Are you revising locally or re-prompting globally?

This is where a lot of teams lose control.
If each round of editing means asking the model to rewrite the whole post, good sections get disturbed along with weak ones. Writers end up paying a penalty for specificity. It is often easier to accept a draft that is vaguely correct than to keep pushing toward something sharper.
That is one reason so many AI-assisted posts sound tidy but forgettable. The workflow favors broad regeneration over targeted editing.

5. Who is actually making the final decision?

This is the most important question.
AI can suggest, summarize, and flag issues. It can even do a decent job of explaining why a paragraph feels generic or why a source may be weak. But publication still needs a human owner.
That person should decide whether the article is accurate enough, specific enough, and worth attaching their name or brand to. AI detectors can help with that review because they are now fairly reliable signals, but they still work best when paired with source checks and human judgment.

A useful standard

I do not think the right question is "Was AI used?"
The better question is: can you stand behind the post?
If the answer is yes, great. If the answer is "mostly," the draft probably needs more review than more prompting.

Under the hood

Tech stack

  • Next.js App Router on the frontend, with CSS modules and strict TypeScript
  • FastAPI and LangGraph orchestration, with LangChain provider adapters
  • Configurable OpenAI, Anthropic, or Gemini analysis models; Tavily supports live research
  • Artifact-first data model: every review state, comment, and revision attaches to a versioned artifact, not a chat thread

Design decisions

  • Agent comments pin to the exact source offsets instead of living in a free-floating sidebar
  • Two upstream revision modes, surgical and rewrite, for narrow edits or a full candidate
  • Post-apply state preserves the previous draft and its findings as historical context
  • Captured fixtures and local state power this demo; it makes no backend, model, or research calls

The full source (including the backend, agent prompts, and evaluation pipeline) is on GitHub: JonahZander/content_evaluation.