Reference

• Citations
• Quotes
• Figures
• Gate
• Grounding
• Values
• Links
• Support
• Retractions
• Legal

Citations

AI models cite with complete confidence and are often wrong. In a test of three replies from free models, they wrote DOIs that do not exist, DOIs that belong to unrelated papers (a Coase 1960 citation pointed at a 1963 book review), real authors with titles they never wrote, and one paper that was invented outright. A check that only asks "is there a paper with this title?" passes the first two. Assay asks more.

• Finds the paper, not just the title. Author, title and year against CrossRef, OpenAlex and (for books) Open Library. With two authors cited, both must be on the same record.
• Checks the link it gives. A DOI, an SSRN abstract, an arXiv paper or a PubMed link is looked up in the registry that holds it. A real paper with an invented DOI is called out, with the closest real DOI offered.
• Notices a paper cited wrongly. The journal or year in the record differs from the citation (journal articles only).
• Tells "not found" from "could not look". If a source is throttled or down, the result is unavailable, never an accusation.
• Acts on the result. One summary line, a verdict for each citation in plain words, and strip() to take failed citations out.
• No dependencies. Plain fetch. Node 20+, browsers, Cloudflare Workers.

Use

import { createAssay, strip } from '@extrafemi-ai/assay';

const assay = createAssay({ mailto: 'you@example.org' });

const text = 'Aid weakens accountability [Rose, 1985, "Sick individuals and sick populations" — IJE — https://doi.org/10.1093/ije/14.1.32].';
const report = await assay.check(text);

report.summary.text;   // "Citations: 1 · ✓ 1 verified"
report.ok;             // false if any citation failed
for (const v of report.verdicts) console.log(v.bucket, v.message);

const clean = strip(text, report);                 // failed citations become "[citation removed: could not be verified]"
const gone  = strip(text, report, { mode: 'remove' });

What it reads

A citation is a square-bracket tag with a quoted title and a year. The same format works in a prompt, so your model produces it:

[Author, Year, "Exact Title" — Journal — https://doi.org/…]                   a title and a link
[Author, Year, "Exact Title" — Journal — search: https://scholar.google.com/scholar?q=…]
[Author, Year, "Exact Title" — Publisher]                                      no link: still checked
[CITATION — approximate: Author, years, topic — search: …]                    an area, not one paper

[Logical inference — …] and any other bracket are ignored. A prompt snippet that teaches a model this format, and tells small models never to write a DOI (they cannot recall them), is in Whetstone (hosted/src/models.js).

Verdicts

Each verdict has status (found, badlink, mismatch, notfound, approx-ok, approx-none, no-title, unavailable…), message (one sentence), match (the record found, so a person can compare), notes, and for a wrong DOI, suggestedDoi.

Options

Showing a verdict in a page

message, match.title and the citation's own fields contain text from the web or from a model. Use textContent, or escapeHtml() (exported) before putting any of it in HTML.

What regulates itself, and what does not

The safeguards above react while Assay runs. Be clear about what that is:

So Assay is self-regulating, not self-updating. The safeguards stop a failing source from causing a wrong verdict; keeping the sources themselves current is maintenance, and the weekly check is what tells the maintainers when it is due.

Adding your own source

You can add a source; Assay will not add one to itself. A new source is a new thing whose answers are believed, and a library that adopted sources automatically could be pointed at a poisoned one, and would break its promise about which hosts it contacts. So adding one is a decision a person takes, in code or in data.

With data, for any service that answers JSON over https (the description is checked when you make it; a reply of the wrong shape is an outage; a canary is required):

import { createAssay, defaultSources, jsonSource } from '@extrafemi-ai/assay';

const myIndex = jsonSource({
  name: 'My Institution Repository',
  url: 'https://repo.example.org/api/search?title={title}&author={surname}',
  headers: { 'x-api-key': 'key:myrepo' },                       // 'key:NAME' is read from keys.NAME below
  results: 'data.items',                                        // where the array of records is in the reply
  fields: { title: 'name', authors: 'writers', year: 'meta.year', doi: 'ids.doi' },
  authorName: 'fullName',                                       // authors are objects: where the name is inside each
  canary: { title: 'A work this repository certainly holds', surname: 'Surname', authors: ['Surname'] },
});
const assay = createAssay({ sources: [...defaultSources, myIndex], keys: { myrepo: process.env.REPO_KEY! } });

With code, implement SearchSource ({ name, canary, search(query, http, ctx) }), or for numbers ValueSource, or seededSource for figures a person read off a primary publication. Add the source to your own scheduled check with createAssay({ sources }).health(): it will tell you when your source starts answering wrongly. To add a source to Assay itself, open a pull request with the source, a fixture recorded from the real service, a canary, and a note of whether it was verified live.

Quotes

A model asked to back a claim with "an exact quote from the posting" can invent the quote. checkQuote checks it against the source text.

import { checkQuote, checkQuotes, verifyQuote } from '@extrafemi-ai/assay/quotes';

verifyQuote(jobAd, 'Visa sponsorship is available for this role');   // true or false
checkQuote(jobAd, 'Relocation is available for all staff');          // { status: 'missing', bucket: 'failed', message: '…' }
checkQuotes([{ id: 'a', quote, source: jobAd }, …]).summary.text;     // "Quotes: 3 · ✓ 2 found · ✕ 1 not found — …"

It forgives typography only (whitespace, curly quotes, dashes, case) and never words. The quote must start and end at a word boundary, and a quote under 12 characters is refused ("sponsorship" would match almost any posting; set minChars to change it). Statuses: found, missing, too-short, empty.

Figures

An invented "5,000 beneficiaries" or "a 40% increase" reads as plausible and nobody checks it. checkFigures finds every number in a draft and checks that the same value appears in the materials you supplied (a CV, a brief, a call for proposals).

import { checkFigures } from '@extrafemi-ai/assay/figures';

const r = checkFigures(draft, [cv, brief, call], { ignore: [/\[ADD:[^\]]*\]/gi] });
r.summary.text;   // "Figures: 3 · ✓ 2 supported · ✕ 1 not in your materials — check them before using the text."
r.verdicts.filter((v) => !v.supported).map((v) => v.figure.raw);   // ["5,000"]

It reads numbers in any notation (5,000, 12.5, 40%, 40 percent, 50k, 1.5m, 2 million, three, twenty-five), so "3" in your brief supports "three" in the draft, and "1.5m" supports "1,500,000". A percentage and a plain number are different figures: a "40 women" in a CV does not support an invented "40%" (kinds: false matches on value alone). tolerance (default 0, exact) lets values within a share of each other match. ignore removes text that is not a claim, such as your own placeholder syntax. One verdict per distinct figure. Left out on purpose: the word "one", ordinals ("40th"), list numbering, and digits glued to letters (B2B, Q3).

@extrafemi-ai/assay/figures also exports extractFigures, extractYears, and evalFormula: arithmetic over named values ("(c3-c7)/c7*100") evaluated without eval, which throws on anything that is not arithmetic, on an unknown name and on division by zero.

Gate

Is a claim sourced well enough to be stated? The gate checks claims against a policy about their sources: no network, no model. It judges how a claim is sourced, not whether it is true.

import { createGate, rules } from '@extrafemi-ai/assay/gate';

const gate = createGate({
  domains: { 'woodmac.com': 2, 'kpler.com': 2 },          // your own tier list (a small neutral default is built in)
  firstParty: ['mybrand.example'],                         // always tier 0
  rules: [rules.estimatesCausalLegalNeedSpecialist],       // or write your own
});

const report = gate.check([
  { text: 'Delays cost traders an estimated $400m.', category: 'loss-estimate', sources: [{ url: 'https://reuters.com/x', date: '2026-01-01' }] },
]);
report.dropped[0].reason;   // "a loss estimate, causal or force-majeure claim needs one source at Tier 2 or better"
report.summary.text;        // "Claims: 1 · ✕ 1 dropped — do not state the dropped ones as fact."
report.unlistedDomains;     // domains the policy did not know: review them and add the good ones

Tiers: 0 your own first-party data, 1 an authority within its mandate, 2 a specialist, 3 the press, 4 an aggregator (it points to the real source; it is never the source). Outcomes: pass, hold (only backing is news under 24 hours old), needs-corroboration, low-tier-only, no-source, and error (a claim that could not be judged, which never passes). A rule says which claims need stronger backing (by label or by what the text reads like, so a wrong label does not dodge it) and what satisfies it, as "any of": [{ atLeast: 1, tier: 1 }, { atLeast: 2, tier: 3 }] is "one authority or two news sources", counted by different sites. Nobody can raise an unknown domain above tier 3 by saying so, including the claim's own author. A broken policy entry is dropped and reported in gate.warnings rather than taking the gate down. summariseRecency() lists sources older than two years.

Grounding

A model that tags its own sentence [[c3]] has proved nothing until something checks that c3 exists, that the sentence's own figure matches c3's, and that a line carrying a number carries a tag that can justify one. checkGrounding does that against the claims you approved for the draft.

import { checkGrounding, stripTags } from '@extrafemi-ai/assay/grounding';

const claims = [{ ref: 1, text: 'The official rate averaged ₦645.19 in 2023.' }];
const r = checkGrounding('The rate moved to ₦425 in 2023 [[c1]].', claims);
r.ok;                       // false
r.violations[0];            // { code: 'figure-mismatch', reason: "states a figure that doesn't match c1's stated figure: possible mis-citation", … }
r.summary.text;             // "Lines: 1 · ✕ 1 not grounded — do not publish the flagged lines as they stand."
stripTags(draft);           // the published text, tags removed

Tags: [[c3]] or [[c3,c7]] cite approved claims; [[calc: c3*0.15]] is a calculation over claims (re-run, and the result must match the line); [[logic]] is reasoning and can never justify a number; [[eg]] marks an invented figure in a worked example; [verify: …] is a note for the editor. The syntax is a parameter (tags: { open: '{{', close: '}}', claim: 's', … }): [[c3]] is only the default. Every factual line is checked, including bullets, quotes and table rows. Headings, code fences and everything under ## Sources are not. Small counts with a unit ("12 years", "three postings") are figures too (countUnits adds yours).

Violation codes: untagged-figure, unknown-claim, logic-figure, figure-mismatch, calc-mismatch, calc-no-refs, calc-unreadable. A formula that cannot be read is a violation, not a way to carry a figure unchecked (the original let it pass). A calculation given only as a list of claims ([[calc: c1,c2]]) names its inputs but cannot be re-run: it is traced and reported in warnings. This checks that a figure is traceable and matches, not that the claim is true; when a claim carries several numbers it cannot tell which one the line means (use values for the period). Offline.

Values

A citation can be real and its link live, and the claim can still state the wrong year's figure. values asks the source's own data.

import { createValues } from '@extrafemi-ai/assay/values';

const values = createValues({ mailto: 'you@example.org' });

await values.check({ value: 19.5e9, unit: 'USD', period: 2023, geo: 'NGA', subject: 'remittances' });
// { state: 'verified', message: 'Matches World Bank (BX.TRF.PWKR.CD.DT) for 2023: 19,549,549,365 (exact).', … }

await values.checkText('Remittances reached $25 billion in 2023.', { url: 'https://api.worldbank.org/v2/country/NGA/indicator/BX.TRF.PWKR.CD.DT' });
// one verdict per year the sentence names; ok: false, "the claim says 25,000,000,000; World Bank says 19,549,549,365 for 2023"

It is strict on purpose: the unit must match (a GBP claim against a USD series, or a count against a percentage, is a different measurement, not a disagreement), the exact period must exist (a close number for a different year never counts: period-unmatched is a downgrade, not a pass), and a model cannot produce "verified". States: verified, mismatch, unit-mismatch, period-unmatched, no-data, no-source, unavailable, not-checkable.

Sources: worldBank and imf (free, no key; the IMF covers World Economic Outlook aggregates only), and seededSource(entries) for figures a person read off a primary publication (a central bank's report) where no API exists. Every seeded figure is reported as verified, so none should go in from memory or from a model. reconcile(claim, series) is the pure comparison, usable on its own. Add your own source with { name, fetch(claim, ctx) }.

crossCheck is an optional hook you supply (a model call, say; Assay makes none) for a claim no source covers. It can flag a mismatch (cross-check-mismatch). At best it says plausible, which is a check, never verification. Values are self-healing like citations: a changed reply shape or an error body is unavailable, never "no data" or a mismatch; a throttled source is left alone and its Retry-After believed; an earlier series is served (stale) when the source is down; and a source's "mismatch" is held back unless it first returns a known figure correctly (health() reports this).

Links

Is the page really there? Server-side only (Node, Bun, Deno): it makes requests, so it is deliberately absent from browser bundles ("browser": null in the package's exports).

import { createLinkChecker } from '@extrafemi-ai/assay/links';

const links = createLinkChecker();
const report = await links.check(['https://example.com/paper.pdf', 'http://169.254.169.254/latest/meta-data/']);
report.verdicts.map((v) => [v.url, v.status]);   // [['…/paper.pdf', 'live'], ['http://169.254.169.254/…', 'unsafe']]
report.summary.text;                              // "Links: 2 · ✓ 1 live · ✕ 1 broken or refused — do not present them as sources."

Statuses: live, broken (404 or 410: the page is not there), blocked (401, 403, 429, 5xx: a site refusing or struggling, never treated as broken), unreachable (could not connect; says nothing about the page), soft404 (it answered, but landed on the home page or a page titled "not found"), unsafe (refused without a request), limit. recheck(report) checks again what could not be settled.

This is the module that can hurt you, because it fetches URLs that came out of AI text, which is shaped by whoever wrote the prompt. A crafted link must never make your server reach your own network or your cloud's metadata service. See SECURITY for what it guarantees and what it does not. In short: the address that is validated is the address that is connected to (so a name that answers "public" to a check and "private" to the connection is refused); every redirect hop is re-checked from the URL down; only http and https on ports 80 and 443, no credentials, no localhost, no .internal or single-label names, and only public IP addresses (including every IPv6 way of writing a private IPv4 one). checkUrl() and isSafeAddress() are exported if you want the rules on their own.

Bounded: 5 seconds per request, 5 redirects, 2 requests at a time to any one host, 25 URLs per call, and only the first 24,000 characters of a page are ever read. A host that fails repeatedly, or sends Retry-After, is left alone for a while. The default User-Agent says what this is; some publishers block anything that is not a browser, which comes back as blocked.

Support

Everything above shows that something exists or is traceable. support asks whether a source text actually backs a sentence. Assay has no model in it: you give it a judge, a function that asks whatever model you trust, and Assay holds the judge to its one rule everywhere: a model may contradict, but it cannot verify.

import { createSupport } from '@extrafemi-ai/assay/support';

const support = createSupport({
  judge: async ({ claim, source }) => askYourModel(claim, source),   // must answer { verdict: 'supports' | 'contradicts' | 'unrelated', evidence?: string }
});
const r = await support.check([{ id: 'a', claim: 'Attendance rose 40% year on year.', source: brief }]);
r.summary.text;   // "Support: 1 · ≈ 1 to confirm"

• Numbers first, with no model. A figure in the claim that is not in the source fails at once and the judge is not asked.
• The evidence must be real. The judge has to quote a sentence from the source; the quote is checked with the quotes check. "Supports" with a quote that is not in the source is evidence-invented, a failure.
• Even then, "to confirm". A quote that is really there, which a model says means the same as the claim, is check, not ok: the model's reading is the part nobody verified. trustJudge: true counts it as ok if you accept that.
• A model that says "contradicts" is believed only if it shows where. With the quote in the source it is a failure; without, it is to look at.
• An unreachable judge, a timeout or an unreadable answer is unavailable and retryable, never an accusation. Assay makes no request itself, but your judge probably does: do not send private text to a model you would not send it to anyway.

Retractions

A citation can be real, its link live, and the paper withdrawn. Crossref's records carry retraction, withdrawal and correction notices, including the Retraction Watch database Crossref now holds.

import { createRetractions } from '@extrafemi-ai/assay/retractions';

const r = await createRetractions({ mailto: 'you@example.org' }).check(textOrListOfDois);
r.verdicts[0];   // { status: 'retracted', bucket: 'failed', notice: { type: 'retraction', date: '2010-02-06', source: 'retraction-watch' }, … }

Statuses: retracted (also withdrawn or removed; a title that starts "RETRACTED" counts), concern (expression of concern: to check), corrected (ok, with the note), clear (no notice recorded), not-in-crossref (skipped, not cleared), unavailable. "Clear" is not proof: notices reach Crossref from publishers and Retraction Watch, and some are late. It reads every DOI in a text, at most 50 by default.

Legal

Invented case law is the best-documented harm from AI-written text. legal asks CourtListener (the Free Law Project's database of US decisions) about every case citation in a text, through its citation-lookup API, which exists for this.

import { createLegal } from '@extrafemi-ai/assay/legal';

const r = await createLegal({ token: process.env.COURTLISTENER_TOKEN! }).check(brief);
r.verdicts[0];   // { status: 'wrong-case', bucket: 'failed', caseName: 'Martinez v. …', message: '925 F.3d 1339 exists, but it is …, not "Varghese v. China Southern Airlines".' }

Where the token goes. Create a free account at courtlistener.com and copy your API token from your profile (CourtListener's API help links to your usage page at courtlistener.com/profile/api-usage/). Then keep it out of your code and your repository: put it in an environment variable called COURTLISTENER_TOKEN, and read it where you create the checker, as above. On your own machine, export COURTLISTENER_TOKEN=… in the shell (or a .env file that is listed in .gitignore). In Cloudflare Workers, npx wrangler secret put COURTLISTENER_TOKEN; on Netlify, an environment variable marked secret; in GitHub Actions, a repository secret passed with env:. From the command line: COURTLISTENER_TOKEN=… assay legal brief.txt. The command line accepts the token only from the environment, never as an option, because options are kept in shell history. The token is sent to courtlistener.com only.

It needs a free CourtListener token, and sends the text to courtlistener.com: never pass confidential text. It is server-side only ("browser": null), so a token is never put in a page. Statuses: found; wrong-case (a real citation that belongs to a different case than the one named, the pattern in the 2023 sanctions case); notfound (not verified: CourtListener does not hold every court, so check before calling it invented); ambiguous, unreadable (a reporter it does not know), limit (over 250 citations in a request) and unavailable. Statutes, journals and id./supra are out of scope for the API. Text over 60,000 characters is sent in parts. Built from the published API reference; no live call has been made with a token, so treat the first real run as the test.