Veracity is an open-source fact-checking system that does something most AI answers still refuse to do: it shows its working. Paste a claim into the chat box, and instead of a confident paragraph you get a number between 0% and 100%, a plain-English interpretation of that number, an explicit recommendation about whether the claim is safe to share, and a panel listing every single source the system consulted to reach its verdict. The system is built on a large language model paired with a web retrieval agent, and the sources panel is not a footnote — it is the point.
That design decision matters more than the headline number. Anyone can bolt a percentage onto a chatbot response. What Veracity does is make the evidence trail a first-class part of the interface, so the user can disagree with the machine on the basis of the same material the machine used. The team behind it published the system as a research paper, put both halves of the codebase on GitHub under an open licence, and shipped a hosted version that looks deliberately like WhatsApp.
This article covers what Veracity actually is, how the 0-to-100% reliability score is defined and where its 60% cutoff gets uncomfortable, what the sources panel really shows, how the pipeline runs end to end, why a tool like this arrived in 2026 rather than 2016, and — the part the paper conspicuously does not answer — how accurate automated fact-checking is when somebody measures it properly. There is a real answer to that last question, and it is neither the dismissal nor the triumph you might expect.
Table of contents
- What Veracity Actually Is
- The Veracity Reliability Score, Explained
- The Sources Panel Is the Real Feature
- How the Veracity Pipeline Runs End to End
- Why a Tool Like Veracity Arrived Now
- How Accurate Is AI Fact-Checking, Really?
- Where Veracity Is Weak
- What Veracity Means for Businesses
- How to Evaluate a Fact-Checking Layer Before You Trust It
- The Verdict on Veracity
- Frequently Asked Questions About Veracity
- References and Further Reading
What Veracity Actually Is
The short description is easy to get wrong, so it is worth being precise about what was built and what was not.
The one-line version
Veracity is a web application that takes a user-submitted claim, decides whether it needs to search the internet, retrieves sources if it does, asks a large language model to assess the claim against those sources, and returns a numerical reliability score with an explanation and the source list. The paper describing it, Veracity: An Open-Source AI Fact-Checking System, was submitted to arXiv on 18 June 2025 and appears in the IJCAI 2025 proceedings as a demonstration paper.
Who built it
The author list runs to eleven names — Taylor Lynn Curtis, Maximilian Puelma Touzel, William Garneau, Manon Gruaz, Mike Pinder, Li Wei Wang, Sukanya Krishna, Luda Cohen, Jean-François Godbout, Reihaneh Rabbany and Kellin Pelrine — working out of the Complex Data Lab in Montreal, whose GitHub organisation hosts both repositories. Rabbany, Pelrine and Godbout have a long track record in misinformation detection research, which is why Veracity reads like a research artefact rather than a startup MVP.
What Veracity is not
It is not a model. Nothing was trained. Veracity is an orchestration layer plus an interface, and the intelligence is rented from a hosted large language model that the paper does not name. It is also not a replacement for a professional fact-checker: it produces a probabilistic assessment in seconds, not a sourced ruling with editorial accountability behind it. Conflating those two things is the fastest way to misuse the tool.
Where it sits against the alternatives
The table below places Veracity beside the three things people actually use today when they want to know whether something is true.
| Approach | Output | Evidence shown | Speed | Accountable party |
|---|---|---|---|---|
| Veracity | 0-100% reliability score plus share advice | Full source list with credibility | Seconds | Nobody — you are the editor |
| Professional fact-checker | True/false/misleading verdict | Cited in the article | Hours to days | A named newsroom |
| Platform community notes | Crowd-written context note | Whatever contributors link | Hours, if it lands at all | Diffuse |
| Asking a chatbot | Fluent prose, no score | Often none, sometimes fabricated | Seconds | Nobody |
That last row is the honest baseline. Millions of people already treat a general-purpose assistant as a fact-checker, and it answers without a score, without a confidence signal and frequently without a verifiable citation. Measured against that, Veracity is a straightforward improvement even before you argue about its accuracy.
The Veracity Reliability Score, Explained
The number is the feature people will notice first, so it deserves a careful reading.
What the number means
The paper defines it plainly: the score “reflects the reliability of the claim, where 0% maps to completely unreliable or false and 100% maps to completely reliable or true.” It is generated by the language model after it has seen the retrieved sources. That phrasing is doing more work than it looks. The score is not a probability in the statistical sense, and it is not a calibrated confidence interval. It is a model’s self-reported judgement rendered as a percentage.
The 60% cutoff
Alongside the score, the interface shows an actionable message interpreting it, plus a share recommendation. The rule is fixed and public: the score “must be greater than 60% for a positive recommendation.” Below 61%, Veracity tells you not to pass the claim on.
Why the 60% line is the most interesting design choice in the system
A single global threshold collapses a continuous score into a binary at a point somebody chose. A claim at 59% and a claim at 12% are wildly different epistemic situations, and the share advice treats them identically. Meanwhile a claim at 61% gets a green light despite the system effectively saying it is only slightly more likely true than not. Anybody deploying Veracity in a serious workflow should treat 60% as a starting parameter, not a law of nature.
Why a number beats a label anyway
The counter-argument is strong. Traditional fact-checking verdicts — “mostly true”, “half true”, “missing context” — are notoriously slippery, vary between organisations, and compress genuine uncertainty into a label chosen by an editor. A continuous score at least admits that uncertainty exists and hands the reader the raw quantity. The 1-to-5-star feedback control in the interface then lets users push back on individual assessments, which is the only correction channel the system has.
| What the Veracity interface returns | Defined in the paper? | Note |
|---|---|---|
| Reliability score, 0% to 100% | Yes | 0% = completely false, 100% = completely true |
| Actionable message | Yes | Interprets the score in plain language |
| Share recommendation | Yes | Positive only above 60% |
| Sources panel | Yes | Every source used, with documented credibility |
| Source count and average credibility | Yes | Shown as aggregate metadata |
| Named score bands below 60% | No | The paper defines one threshold only |
| Calibration evidence for the score | No | No benchmark numbers are reported |
| The specific model behind the score | No | Described only as a hosted LLM |
The Sources Panel Is the Real Feature
Strip the score out and Veracity would still be more useful than most AI answers, because of what sits next to it.
Retrieval happens before judgement, not after
When a claim arrives, the language model first decides whether web retrieval is necessary. If it is, a web agent goes and gets sources, and the model is shown those sources before it renders a verdict. This ordering is the whole ballgame. A system that answers first and cites second is producing justification; a system that retrieves first and answers second is producing an assessment. The retrieval approach follows the Complex Data Lab’s own earlier work on web retrieval agents for evidence-based misinformation detection, published at COLM 2024, which reported macro F1 improvements of as much as 20 percentage points over a language model working without search.
Documented credibility, not just links
The user is shown the list of sources “as well as their documented credibility”, plus aggregate metadata: how many sources were used and their average credibility ranking. That aggregate is a quietly important signal. A 78% score built on nine high-credibility sources and a 78% score built on two low-credibility blogs are not the same claim, and the panel lets you tell them apart without reading anything.
Why showing sources changes the failure mode
An unsourced AI fact-check fails silently — you cannot tell a good answer from a hallucinated one. A sourced fact-check fails loudly, because a bad source list is visible at a glance. Veracity does not eliminate error; it relocates error into a place where a human can catch it. That is a genuinely different safety property, and it is the same argument we made when looking at why analysts still do not trust what AI tells them about their own data.
The interface deliberately looks familiar
The paper is explicit that the interaction model “was inspired by standard messaging applications (WhatsApp, Messenger, etc.)”. You type or paste a claim into a box. That is not a cosmetic choice — misinformation spreads inside messaging apps, so a fact-checking tool that feels like one removes a context switch at exactly the moment a user is deciding whether to forward something.
How the Veracity Pipeline Runs End to End
The architecture follows a model-view-controller split with three encapsulated elements, and the runtime path is short enough to describe in full.
Stage 1 — claim intake
The user submits a claim as free text in the chat interface. There is no form, no source field and no structured input to fill in, which is the correct decision for a tool aimed at the general public rather than at researchers.
Stage 2 — retrieval triage
The language model decides whether it needs external evidence at all. Claims that are self-evidently definitional or arithmetic do not need a web search; claims about events, people or statistics do. This triage step is what keeps latency and cost sane at scale.
Stage 3 — web retrieval
If retrieval is needed, a search engine selects relevant sources and the web agent collects them. The sources are the evidence base for everything downstream.
Stage 4 — grounded assessment
The model is shown the retrieved sources and produces the reliability score along with its explanation. Because the sources are in the context window, the explanation can point at specific evidence rather than at the model’s own priors.
Stage 5 — presentation and feedback
The interface renders the score, the actionable message, the share recommendation and the sources panel. Users can rate an assessment between 1 and 5 stars, and that feedback is stored. The whole exchange persists in a PostgreSQL database, so a Veracity conversation has history in the way a chat does.
| Layer | Technology | Why it matters |
|---|---|---|
| Frontend | Next.js, TypeScript, Sass, Chart.js | Chart.js renders the score visualisation |
| Backend API | FastAPI with SQLAlchemy ORM | Conventional, easy to fork and self-host |
| Database | PostgreSQL on Cloud SQL | Stores claims, verdicts, sources, ratings |
| Deployment | Kubernetes on Google Cloud Platform | Horizontal scaling for bursty demand |
| Intelligence | Hosted LLM, model not named | Swappable — and a reproducibility gap |
| Retrieval | Web search agent, Tian et al. method | The evidence layer under every verdict |
| Repositories | veracity-eval-frontend, veracity-eval-backend | Both public under Complex Data Lab |
| Paper licence | CC BY 4.0 | Free to reuse with attribution |
Nothing here is exotic, and that is the point
A Next.js frontend talking to a FastAPI backend on Kubernetes is the least surprising stack in modern web engineering. Any competent team can read the Veracity repositories and stand up their own instance. That reproducibility is worth more to the field than a clever architecture would be, because the barrier to open fact-checking infrastructure has never really been the engineering.
Why a Tool Like Veracity Arrived Now
The paper’s motivation section is short and blunt, and the surrounding data explains why.
Misinformation keeps topping the risk tables
The paper opens with the observation that experts have rated the dissemination of misinformation and disinformation as the number one risk the world faces. The World Economic Forum’s Global Risks Report 2026 keeps it in the same neighbourhood: second most severe risk over the next two years, behind geoeconomic confrontation, and one of very few risks that stays severe across both the two-year and ten-year horizons. That is the third consecutive year it has ranked among the most severe global risks.
Platforms are handing the job back to users
The paper names the mechanism directly: “as large-scale social media platforms actively eliminate their content moderation teams, they pass to the user the personal and social responsibility to assess the reliability of claims.” That transfer of responsibility is the market Veracity is built for. If the platform will not adjudicate, the individual needs a tool.
And the professional fact-checking corps is shrinking
The Duke Reporters’ Lab census is the closest thing the field has to a headcount, and the direction is not good. Its 2026 count found 437 active fact-checking projects worldwide as of 1 June 2026, down from 441 at the end of 2025 and from a peak of 464 in 2024. In 2025, ten new projects launched while more than thirty stopped — a net loss of twenty-three. The United States went from 65 projects to 61 over the same period.
Here is the trajectory, drawn as a share of the 2024 peak of 464 projects.
The decline is modest in percentage terms — roughly 4% since June 2025 — but it comes after a decade of growth, and it lands while claim volume goes the other way. The same census records fact-checkers still working in 116 countries and more than 70 languages, so the network is broad; it is the funding underneath it that is thinning.
The finances behind those numbers are worse than the headcount
An International Fact-Checking Network survey of more than one hundred fact-checkers found roughly three-quarters describing themselves as financially vulnerable or in crisis, nearly half reporting falling revenue, more than a third making staff cuts, about 30% reporting government pressure or interference, and almost two-thirds reporting harassment.
A sector where three in four organisations describe themselves as financially precarious is not a sector that scales to meet generative-AI-speed claim production. That gap is the argument for automation, and it is why the Veracity team framed the work as empowering individuals rather than as replacing newsrooms.
How Accurate Is AI Fact-Checking, Really?
Here is the awkward part. The Veracity paper contains no accuracy numbers at all — no benchmark table, no dataset evaluation, no error analysis. It is a demonstration paper describing a system, and it should be read as one. So the honest way to assess whether a Veracity-style approach works is to look at independent measurements of the same underlying task.
The best public benchmark we have
Originality.AI published a comparison on 21 May 2026 running three fact-checking systems across three standard datasets, filtered down to binary true/false outcomes: 928 curated claims from FEVER, 693 from SciFact, and 3,017 from AVeriTeC. The results are more encouraging than the “AI cannot fact-check” consensus suggests.
| System | FEVER (928) | SciFact (693) | AVeriTeC (3,017) | Overall accuracy |
|---|---|---|---|---|
| Originality.ai checker | 97.8% | 88.7% | 82.7% | 86.69% |
| GPT-5 | 99.6% | 80.8% | 84.0% | 86.67% |
| GPT-4o | 94.1% | 79.5% | 80.9% | 83.40% |
Read the hardest column, not the easiest
FEVER claims are generated from Wikipedia and are close to a solved problem — a frontier model scores 99.6%. AVeriTeC is the one that matters, because its claims are real-world and its evidence has to be retrieved rather than assumed. Every system lands between 80.9% and 84.0% there. That is the realistic ceiling for automated fact-checking on messy claims today.
What roughly 83% means in practice
Better than a coin flip is a low bar; better than an unaided reader scrolling a feed is the relevant one. But one claim in six being wrong is unusable as an automated gate on anything consequential. It is perfectly usable as a triage signal that tells a human which claims to look at first. That distinction — assistive triage versus autonomous verdict — is the line every deployment of a Veracity-style tool has to draw explicitly.
The retrieval is what buys the accuracy
The Complex Data Lab’s own COLM 2024 result is the cleanest evidence for why Veracity is architected the way it is: adding a web search agent to a language model raised macro F1 on misinformation detection by up to 20 points. The score is only as good as the sources panel underneath it, which is a nice symmetry — the feature that makes Veracity auditable is also the feature that makes it work.
Where Veracity Is Weak
A fair assessment of Veracity has to include the things its own authors flag and the things they do not.
Complex claims
The paper’s future work section names improving the handling of complex claims as an open problem. Multi-part claims, claims that are technically true but misleading, and claims whose truth depends on a definition are all cases where a single scalar score is the wrong output shape. Compressing “true but missing crucial context” into 64% loses the only part a reader needed.
Splitting a sentence into the separate assertions it actually makes is a natural language processing problem long before it is a retrieval problem, and Veracity does not attempt it — one claim in, one number out. Until that changes, the tool is strongest on simple factual assertions and weakest on exactly the rhetorical constructions that political misinformation favours.
The score has no published calibration
Because Veracity reports no benchmarks, there is no evidence that a 70% score is right about 70% of the time. It may well be well-behaved; nobody has shown it. Anyone building on the system should generate their own calibration curve against a labelled set before they trust the number as a threshold.
The unnamed model is a reproducibility gap
The codebase is open, the paper is CC BY 4.0, and the stack is conventional — but the paper does not say which hosted large language model produces the verdicts. Since the model is the component that actually decides, results from a self-hosted fork may not match the hosted demo, and neither may match a run six months later after the provider ships an update.
Retrieval inherits the web’s biases
A retrieval-grounded system is only as good as what a search engine surfaces. On contested topics, recent events, and languages with thin online coverage, the evidence base is thinner and more skewed. The paper lists broadening language and context support as future work, and notes multilingual support as a feature without specifying which languages are covered.
Adversaries can target the evidence layer
If the verdict depends on retrieved sources, then flooding the indexable web with plausible-looking corroboration is a direct attack path — and generative models make that cheap. This is the structural risk that makes the credibility ranking in the sources panel more than a nicety. The same dynamic shows up wherever AI systems consume open web content, as we found when looking at how far you can actually get by learning to spot AI-written text.
What Veracity Means for Businesses
Most coverage of a tool like this stops at the consumer use case. The organisational implications are more concrete.
Communications, PR and brand teams
Claim monitoring is already a job somebody does badly with alerts and spreadsheets. A scored, sourced assessment of an inbound claim about your company — with the source list attached — is a materially better artefact to put in front of a communications lead at 7am than a link and a gut feeling.
Regulated and high-trust sectors
Financial services, healthcare and public bodies all have processes for verifying external claims before acting on them. A Veracity-style layer does not replace those processes, but it does front-load them, and the sources panel produces exactly the kind of evidence trail a compliance reviewer wants to see attached to a decision.
Anyone publishing at volume
Content operations that push out dozens of pieces a week have no realistic way to verify every factual assertion. Automated triage that flags the lowest-scoring claims for human review is a sensible control, and it is the deployment pattern that fits the roughly 83% real-world accuracy ceiling rather than fighting it.
Build, buy or embed
| Option | What it involves | Best when | Main risk |
|---|---|---|---|
| Use the hosted Veracity demo | Nothing to run | Ad-hoc checks, evaluation, training staff | No control over the model or uptime |
| Fork and self-host the repositories | Next.js plus FastAPI plus Postgres plus your own LLM key | Data residency matters, or you need custom thresholds | You now own an evidence pipeline |
| Build your own on the same pattern | Retrieval-first architecture, your sources, your scoring | Domain-specific claims with private evidence | Calibration and evaluation are your problem |
| Buy a commercial checker | Vendor API, published benchmarks | You need an SLA and someone to call | Opaque method, per-claim pricing |
The retrieval-first pattern generalises well beyond fact-checking. Any process where a model must justify a decision from named evidence — claims triage, supplier due diligence, policy checks — benefits from the same ordering, and it is the pattern we build into autonomous AI agents rather than bolting citations on afterwards.
How to Evaluate a Fact-Checking Layer Before You Trust It
If you are assessing Veracity, a fork of it, or any commercial equivalent, these are the questions that separate a usable tool from a confident one.
Does it retrieve before it answers?
Ask for the ordering explicitly. A system that generates a verdict and then searches for supporting links is performing citation theatre. Veracity gets this right, and it is the single most important architectural question.
Can you see every source, not a curated subset?
The value of the sources panel collapses if it shows the three best links rather than everything consulted. Ask whether the panel is the complete evidence set.
Is the score calibrated, and against what?
Ask for a calibration curve or a benchmark table on a named dataset. If the answer is a demo rather than a number, treat the score as an ordering signal — useful for ranking claims by risk — not as a probability.
What happens on claims outside the training and retrieval distribution?
Non-English claims, very recent events, niche technical assertions and contested political claims are where these systems degrade. Test those cases yourself before you deploy.
Who owns the threshold?
Veracity ships a fixed 60% share cutoff. In your workflow, that number should be yours, set from your own tolerance for false positives against false negatives, and revisited when the model behind the system changes.
| Evaluation question | Good answer | Warning sign |
|---|---|---|
| Retrieval ordering | Sources fetched before the verdict | Citations added after generation |
| Source transparency | Complete list plus credibility | Top three links only |
| Calibration | Published curve on a named dataset | A demo video |
| Model disclosure | Named model and version | “A leading LLM” |
| Threshold control | Configurable per workflow | Hardcoded global cutoff |
| Failure behaviour | Abstains when evidence is thin | Always returns a confident number |
The Verdict on Veracity
Veracity is a small system making a large argument: that the useful unit of AI fact-checking is not the answer but the evidence, and that a number plus a source list beats a paragraph of confident prose. On that argument it is right, and the fact that both repositories are public means nobody has to take the claim on faith.
The weaknesses are real and mostly honest ones. There are no published benchmarks, the model is unnamed, the 60% cutoff is arbitrary, and complex claims are an acknowledged open problem. Independent measurement of the wider task suggests roughly 83% accuracy on realistic claims, which makes any Veracity-style system a triage instrument rather than an arbiter.
That is still worth having. The professional fact-checking corps is down to 437 projects and three-quarters of it is financially precarious, while claim volume rises. A free, open, retrieval-grounded tool that hands people the sources and lets them judge is not a solution to that gap, but it is a considerably better default than the alternative most people currently use, which is asking a chatbot and believing it. You can read more analysis of releases like this in our AI models and tools hub.
Frequently Asked Questions About Veracity
Is Veracity free to use?
The research paper is published under CC BY 4.0 and both the frontend and backend repositories are public on GitHub under the Complex Data Lab organisation. Running your own instance requires your own language model credentials and cloud costs.
What does a Veracity score of 60% actually mean?
It sits exactly on the system’s share threshold. The paper specifies that a claim must score greater than 60% to receive a positive share recommendation, so 60% itself gets a negative one. Treat any score near that line as unresolved rather than as false.
Which language model does Veracity use?
The paper describes it only as a hosted large language model and does not name the provider or version. That is the system’s biggest reproducibility gap, because the model is the component producing the verdict.
Can Veracity fact-check claims in languages other than English?
Multilingual support is listed as a key feature, but the paper does not specify which languages are covered, and broadening language support is named as future work. Test your own languages before relying on it.
How does Veracity compare to a professional fact-checker?
It is faster by orders of magnitude and shows more of its evidence, but it carries no editorial accountability and publishes no accuracy figures. A professional ruling is slower, sourced and attributable to a named organisation. They are complements, not substitutes.
Does Veracity store the claims people submit?
The system runs on a PostgreSQL database that persists claims, assessments, sources and the 1-to-5-star user ratings. If that matters for your use case, self-hosting the backend is the way to control it.
References and Further Reading
Veracity: An Open-Source AI Fact-Checking System (arXiv:2506.15794)
Veracity: An Open-Source AI Fact-Checking System — IJCAI 2025 Proceedings
ComplexData-MILA/veracity-eval-frontend on GitHub
ComplexData-MILA/veracity-eval-backend on GitHub
Web Retrieval Agents for Evidence-Based Misinformation Detection (arXiv:2409.00009)
2026 Census: Fact-Checking Losses Continue Amid Funding Pressure — Duke Reporters’ Lab
Global Risks Report 2026: Key Findings — World Economic Forum
AI Fact Checking Accuracy Study — Originality.AI
Fact-Checkers Persevere as Politicians and Platforms Turn Up the Heat — Poynter
More AI coverage: explore Progressive Robot's AI Models, Tools & Releases hub — hands-on reviews, setup guides and benchmarks in one place.