Skip to content
NetProjects
Menu

Insights

Two AI judges, one laptop: how we fact-check AI writing in under a second

Paris Michalas, Partner, NetProjects

We gave five AI "judges" the same 2,341 fact-checks on a single MacBook. The most accurate was not the biggest general model. It was a model trained only to make decisions. And the best setup of all was a team of two: a tiny, fast judge that handles most claims in a fifth of a second, and a careful second opinion that steps in only when the first one is unsure.

No cloud. No per-call bill. Here is what we found, and how you can build the same thing.

Why check AI writing at all?

AI writes fast and fluently. That is the problem: it can slip in a wrong date, a score that never happened or a detail that sounds right and isn't, and it reads just as confidently as the true parts.

We publish a lot of content written with the help of local AI models. So every sentence goes past a "judge" before it goes live. The judge reads the source material and the sentence, and answers one question: does the source actually say this? Supported, unsupported, or contradicted.

The judge has to be accurate, but it also has to be fast and cheap, because it reads every single sentence.

The test: 2,341 claims, five judges, one MacBook

We used 2,000 claims from two public fact-checking datasets where people had already marked the right answer (LLM-AggreFact and VitaminC), plus 341 sentences from real articles we publish. Everything ran on one MacBook Pro M5 Max with 128 GB of memory.

The five judges, all running locally:

  • H2O-Lightning 4B: a small decision model from H2O.ai, which we ported to the Mac.
  • GLiDE: a 27B decision model from Fastino, the top model on the Decision Index. We released the first Mac (MLX) version of it this week.
  • Qwen 27B: a strong general-purpose model of the same size as GLiDE.
  • Qwen 35B-A3B: a larger, faster "mixture of experts" model.
  • Granite Switch 8B: IBM's compact model with built-in decision skills.
JudgeRight answers (2,000 public claims)Time per claimMemory
GLiDE 27B86.0%1.2 s26 GB
Qwen 27B83.9%1.1 s32 GB
H2O-Lightning 4B83.1%0.18 s10 GB
Granite Switch 8B81.6%0.35 s11 GB
Qwen 35B-A3B81.0%0.25 s32 GB

Two things jump out:

  • Training beats size. GLiDE and Qwen 27B have the same size, but GLiDE, trained only to make decisions, gets about 2 more claims right in every 100, and lets fewer false claims slip through.
  • The little one punches above its weight. H2O-Lightning is about a seventh of the size, yet lands within a point of Qwen 27B and answers six times faster.

The trick: a fast first reader and a careful second opinion

Think of a newsroom. A quick junior editor reads everything. When they are sure, the sentence moves on. When they hesitate, they pass it to a senior editor.

That is exactly how we run our judges. H2O-Lightning reads every claim and also says how sure it is. If it is at least 70% sure, its answer stands. If not, GLiDE gives the second opinion.

SetupRight answers (public claims)Our own articlesSent to the second judgeTime per claim
H2O, then GLiDE when unsure85.3%87.4%25%0.50 s
GLiDE on everything86.0%87.7%100%1.2 s
H2O, then Qwen 27B when unsure83.8%84.8%25%0.46 s
H2O alone83.1%85.6%0%0.18 s

The two-step team is within a whisker of the big judge on its own (85.3% against 86.0%) in well under half the time, because 3 in 4 claims never leave the small model.

We had tried this before with Qwen 27B as the second opinion. It made the setup more careful, but not more accurate. GLiDE is the first second opinion that actually lifts the score: it fixed enough of the small model's doubtful calls to add more than 2 points. When the two judges disagree, GLiDE is right 7 times out of 10.

What it changes day to day

In our live setup we play it safe. A claim the small judge is unsure about is never waved through: if GLiDE says it is not supported, it is blocked; if GLiDE says it is fine, a person takes a quick look.

For every 1,000 claims, that works out as:

  • 484 pass automatically, and only 40 of those should not have.
  • 425 are blocked automatically.
  • 92 go to a person, down from 254 without the second opinion.

The human review pile shrinks by almost two thirds, from 25% of claims to 9%. And the claims that do reach a person are mostly fine (8 in 10), so the review is quick.

Both judges together need about 37 GB of memory and run comfortably next to the models that write the content in the first place.

Try it yourself

You don't need a data centre. The whole recipe fits on a desk:

  • 1. A Mac with Apple Silicon. 64 GB of memory runs both judges comfortably; there is also a smaller 4-bit GLiDE for 32 GB Macs.
  • 2. Download the two judges, free, from our Hugging Face page: H2O-Lightning 4B and GLiDE (links below).
  • 3. Start each one as a small local server. Each GitHub page has the one-line command.
  • 4. Ask the small judge first. Give it your source text and the sentence to check. It answers "supported", "unsupported" or "contradicted", and tells you how sure it is.
  • 5. If it is less than 70% sure, ask GLiDE the same question.
  • 6. Decide what happens next: publish, block, or send to a person.

The 70% line is your dial. At 70%, a quarter of claims go to GLiDE. Turn it up to 80% and GLiDE sees 4 in 10; at 90%, 6 in 10, for even fewer slips.

The same pattern works for any yes/no or pick-one question you ask at volume: is this review spam, does this answer match the question, which team should handle this ticket.

Credits and links

Built with Fastino GLiDE. GLiDE is by Fastino and H2O-Lightning 4B is by H2O.ai; the Qwen models are by Alibaba's Qwen team and Granite Switch is by IBM. Our Mac versions are free to download. NetProjects is not affiliated with Fastino or H2O.ai.

How we measured: 1,500 claims sampled across the LLM-AggreFact sources plus 500 from VitaminC, each with a human-marked answer, and 341 sentences from real articles we publish. Every judge got the same question. Tests ran in October 2026 on a MacBook Pro M5 Max, 128 GB.