Citations
Is this paper real?
Assay looks each citation up in public records of papers and books, and says found, look again, or not found. It also checks that the link points to that paper.
Looks it up onlineFree and open source · MIT
AI writes with total confidence. Sometimes it makes things up.
Assay reads what the AI wrote and tells you which citations are not real papers, which quotes are not in the source, and which numbers nobody gave it. You decide what to publish.
The checks that use your own text run in your browser and send nothing anywhere.
Ten checks, each for one job
They answer different questions, so each is its own check with its own import. Use the one that fits; they are not meant to run all at once. Each picture loops to show what the check does.
Quotes
Is this quote really in the source?
It searches your source text for the exact sentence. It forgives curly quotes, dashes and capital letters, never different words.
Stays on your deviceFigures
Did I make up a number?
It finds every number in the draft and checks that you supplied the same number in your own materials. “three” matches 3.
Stays on your deviceGate
Is this claim sourced well enough to say?
Sources are ranked from your own data down to news aggregators. Some claims, like a loss estimate, need a specialist source. With only press coverage behind it, the gate drops the claim.
Stays on your deviceGrounding
Does every number trace to a fact I approved?
Each line carries a tag naming the approved fact it came from. Assay checks that the fact exists and that the line states the same figure.
Stays on your deviceValues
Is this figure right for that year?
It asks the source’s own data, such as the World Bank or the IMF, for that exact year and unit. A nearby year never counts as a match.
Looks it up onlineLinks
Does this link work, and is it safe to open?
From a server only. It checks the page is there, and refuses addresses that point into your own network without making any request.
Makes web requests, server onlySupport
Does the source actually say it?
A model you supply proposes a sentence from the source as evidence. Assay finds that exact sentence in the source, or marks the evidence as invented. Even when found, a person confirms the reading.
Stays on your device (your model may not)Retractions
Has this paper been retracted?
It asks Crossref, which includes the Retraction Watch database, whether the paper has a retraction, withdrawal, correction or expression-of-concern notice.
Looks it up onlineLegal
Is this case citation real, and is it that case?
It asks CourtListener what case is held under each citation, and catches a real citation attached to the wrong case. Server side only; needs a free token.
Sends the text to CourtListener, server onlyWhere AI goes wrong
The last four are real runs: Assay checked the text described, when this page was built. Open “Show the code” to see the call.
A paper that does not exist
Asked for sources, AI models write convincing ones: a real author with a title they never wrote, or a link that goes to a different paper. In a test of free models, some citations were invented outright.
What Assay does
It looks each citation up in public records of papers and books, and says found, look again, or not found. If a record service is down, it says it could not look. It never calls that “not found”.
A quote nobody wrote
You asked for an exact quote from a job posting. The AI wrote “Relocation is available for all staff.” The posting says relocation is for senior staff only.
Assay says
The quote is not in the source text. It may be paraphrased or invented.
Show the code
const posting = "We offer visa sponsorship for this role. Relocation support is available for senior staff only.";
checkQuote(posting, 'Relocation is available for all staff');import … from '@extrafemi-ai/assay/quotes'
A number from nowhere
Your draft says “5,000 beneficiaries”. The brief you gave the AI never mentions that number, but it does mention 40% and three districts.
Assay says
"5,000" does not appear in the materials you gave. Check it, or replace it with a blank.
Show the code
const draft = "We reached 5,000 beneficiaries, a 40% increase, across three districts.";
const brief = "Our programme worked in three districts, and attendance rose 40% on last year.";
checkFigures(draft, [brief]);import … from '@extrafemi-ai/assay/figures'
A number tied to the wrong fact
The AI says the rate moved to ₦425 and tags the line as coming from a fact that says ₦645.19.
Assay says
States a figure that doesn't match c1's stated figure: possible mis-citation
Show the code
const claims = [{ ref: 1, text: 'The official rate averaged ₦645.19 in 2023.' }];
checkGrounding('The rate moved to ₦425 in 2023 [[c1]].', claims);import … from '@extrafemi-ai/assay/grounding'
The right source, the wrong year
A draft says Nigerian remittances reached $25 billion in 2023. The World Bank figure for that year is about $19.5 billion.
Assay says
The claim says 25,000,000,000; World Bank says 19,549,549,365 for 2023. That is outside tolerance.
Show the code
reconcile(
{ value: 25e9, unit: 'USD', period: 2023, geo: 'NGA', subject: 'remittances' },
{ source: 'World Bank', indicatorId: 'BX.TRF.PWKR.CD.DT', unit: 'USD',
points: [{ year: 2023, value: 19549549365 }] },
);import … from '@extrafemi-ai/assay/values'
What it has caught, so far
These are counted from the project's own work, and each one links to where it comes from. They are not usage statistics: Assay has no telemetry, so it cannot see what it catches in other people's apps. That is a promise, not a gap.
- 15 of 30
citations written by free models that turned out invented or wrongly linked, in a hand-labelled sample from 3 November 2026.
The sample, and why it is not a benchmark - 34 of 36
references in the project's two technical papers were verified by Assay itself; two could not be checked because a source was throttling, and none failed or was retracted.
How that was run - 1
real bug in Assay found by Assay: a correctly cited paper looked wrong because its title carried markup. Fixed in 0.4.0.
In the changelog - 10
places where Assay's own text parsing could be made to run for minutes on hostile input, found by measuring and fixed before release.
In the security notes
Want a counter for your own app? createTally() keeps only numbers, never text, and you decide where they go. Try the checks yourself: the demo counts what it catches for you, on your device only.
Why a red mark is worth believing
It says what it compared against
Every result names the thing it checked the text with: a public record, your own source text, a list of facts you approved.
It does not accuse when it could not look
Free record services get busy or break. When that happens Assay says “could not check”, and tries again later. It keeps earlier answers and uses them, marked as old, until the service is back.
It is honest about its limits
It can show that a source exists, not that the source says what the sentence claims. Books and reports are thinly covered, so a real work can come back as not found. Read the limits.
Add it to your app in a minute
For developers. It is a small package with no dependencies that runs on Node 20 or later, in browsers and on Cloudflare Workers.
npm install @extrafemi-ai/assay