5 Signs Your Research Agent Is Hallucinating Sources
Five tests for whether a research agent invents sources, and what to ask of an AI research API with citations. Research runs over REST; MCP tools help check.
By the NeuralVerge team · 12 min read · Updated October 2026
▸On this page
An AI research API with citations is only as good as the citations are real. A research agent that cites sources feels trustworthy: a number, a link beside it, the tone of a careful analyst. But a citation is text, and a model can write citation-shaped text as easily as any other. This post gives five testable signs that an agent invents its sources, with a check and a fix for each. NeuralVerge research runs over REST; the tools around it are also on MCP.
The short version
- —The verdict. Judge citations by testing them: do the links resolve, do the pages contain the claim, is the source mix varied, are the dates possible, and is there a quote behind every number. Five checks, cheapest first.
- —Who it is for. Anyone evaluating a research product or debugging an agent that answers questions about companies and people.
- —The fix. The pipeline, not the model, should write the citations. That is how AI research works: every claim links to a source it retrieved.
- —What it costs. AI research is priced by depth tier, from 20 points (Lite) to 400 points (Ultima). Plans start at $50 a month for 50,000 points.
- —Where to read next. To build one that avoids these failures, see the research agent tutorial. This post is for the agent you already have.
Five signs of invented sources
"Hallucinating sources" covers several failures, and each leaves a different fingerprint. A fabricated source is a page or filing that does not exist. A misattributed claim cites a real page that does not contain it. A laundered claim came from the model's memory and got a respectable citation afterwards. A mismatched source is real and on topic but too old, too general, or about a similarly named entity. Only the first is caught by clicking the link, which is why the other three do the most damage. The five signs run from the cheapest check to the most involved.
1. Citations that do not resolve
The symptom. A cited URL returns a 404, points to a domain that does not exist, redirects to a generic homepage, or shows a page unrelated to the title the agent quoted. A model that writes URLs from memory follows the conventions of the site it has in mind. The site is real; the page is not.
How to test it. Request each cited URL from a batch of answers, following redirects, and record the final status and address.
r = requests.get(url, timeout=10, allow_redirects=True)
print(r.status_code, r.url, len(r.text))
Look for a failing status, a deep URL redirected to a shallow page (a soft 404), and a suspiciously small body. Some sites block automated requests; treat those as "could not verify." Several dead links per answer is the pattern to worry about.
What a fix looks like. Telling a model "only cite real URLs" does nothing; the URLs it writes feel real to it. Build the citation list from pages the pipeline actually fetched for this request, so the model can choose which retrieved source supports which claim but cannot add one that never passed through retrieval. Then resolve every citation before release.
2. The citation is real but does not contain the claim
The symptom. The link works, the page is about the right company, and the specific fact is nowhere on it. It is the most common failure in agents with a search tool: the agent reads a page, then writes a sentence broader or more precise than the page and attaches it as the source.
Suppose an agent is asked about Acme Oy (Finland) and writes: "Acme Oy was founded in 2019 and has between 50 and 100 employees," citing the company's about page. The page gives a founding year and says nothing about headcount. Half the sentence is supported, half was supplied by the model, and one citation covers both.
How to test it. Break the answer into claims and check each against the cited page in two passes. Pass one is mechanical: for every number, date, name or quoted phrase, check whether that string, or a close variant, appears in the page text. It catches invented figures cheaply but misses paraphrases, so use it to sort answers, not to give a verdict. Pass two is judgement: give a second model call or a human the claim, the page text and one narrow question, does this passage state or directly imply the claim, and which sentence does? Requiring a quote matters: a reviewer who must point at a sentence is hard to satisfy with a page that is merely on topic.
What a fix looks like. Cite at the claim, not the answer, since a source list at the bottom cannot tell you which sentence relies on which page. Keep to one claim per citation. And have the pipeline store the passage each claim relied on, dropping any claim for which it cannot produce one.
3. The source mix is suspiciously uniform
The symptom. Across many answers on very different questions, the same one or two domains supply almost all the citations. It is invisible in a single answer and obvious across fifty. Usual causes: answering from memory, a narrow search step, or stopping at the first plausible page. An ownership question has a natural mix: a registration record, a filing or press release, perhaps independent reporting. If every answer cites the same general-purpose site, the agent is reciting, not researching.
How to test it. Run a batch of varied questions, extract the domain from every citation and tabulate. Check concentration (the top domain's share), per-answer diversity, and fit to the question: write down which kinds of source you would expect to be authoritative for ten questions, then compare. A question about a legal entity's status that cites a blog post and never a registry record is a mismatch even if every link resolves.
What a fix looks like. Route by question, not habit. A pipeline that matches each part of the question to the right category of source produces a varied mix; that is how NeuralVerge research works against its source catalog. Where sources overlap on a claim, use both and say when they disagree. Uniformity is not proof, since some questions really are answered by one site, but it deserves an explanation.
4. The source is older than the claim
The symptom. The answer describes something recent, such as a funding round, a new role or a change of address, and the cited page was published or last modified before it could have known. A page dated March cannot support a statement about June; the agent has usually recalled the claim and attached a real page on the same topic.
How to test it. For each citation, find the best date you can and compare it with the time reference in the claim. Source dates come from markup, the byline or the Last-Modified header; none is fully reliable. Pull out phrases like "last quarter," "recently" and "currently," and flag claims about changeable facts, such as job titles, that carry no time reference. A claim about a time after the source's date is a hard failure; an old source behind a claim phrased as current is a softer one.
What a fix looks like. Store retrieval time and the source's own date with every citation so a reader can judge freshness. Reject a claim whose stated time is later than its source's date, or downgrade it to "as of the source date." And be honest about currency: retrieving live is not the same as being current, because a registry record is only as fresh as the registry's own last update.
5. Confident, precise numbers with no quote behind them
The symptom. Exact figures, such as revenue, headcount, funding amounts or dates, stated flatly and not found on the cited page. Precision makes a hallucination dangerous: "raised a significant amount" is harmless, an exact euro figure invites a decision. The tell is the absence of a quote. A grounded number traces to a sentence or table cell on a retrieved page; a hallucinated one has nothing to trace to.
How to test it. Make quotation the test. For each numeric claim, find the passage where the number appears and copy the exact text next to the claim. Check the unit (switched currencies, "in thousands" tables read as raw values), that the figure belongs to this company and not a parent or a namesake, and the period, since a figure with no year is a claim about no time at all. Ask about a small private company whose headcount is not public; an honest system returns "not found."
What a fix looks like. Extract, do not recall: pull numbers from retrieved text into structured fields and keep the source passage. Return, for each numeric claim, the value, the supporting text and the source, so a downstream check can confirm the value appears in the text. And define an "I don't know," because if the only allowed outputs are confident ones the agent will produce confident ones.
Running all five on one answer
Suppose an agent is asked for the ownership and recent funding of Acme Oy (Finland) and returns three cited sentences. Two citations load; the third redirects to a news homepage (sign 1). Both shareholders in the ownership sentence appear on the cited registry page, but the round size appears in neither working source (sign 2). All three citations sit on one aggregator, where ownership calls for a registration record (sign 3). The funding sentence says "recently," yet the only working funding source predates the round (sign 4). The round size is exact and no retrieved passage contains it (sign 5).
One answer fails four of five checks, all visible from outside. The ownership sentence passed cleanly, which is also useful: the aim is to route trust to the parts of an answer that earned it.
REST and MCP
Each fix points the same way: the citation should be something the pipeline guarantees, not something the model writes. NeuralVerge research decomposes a question, matches sub-questions to sources (structured ones such as corporate registries before the open web), cross-checks facts across sources and returns inline citations.
A research run is started and polled over REST. This is the request from the AI research page, with Acme Oy (Finland) as the illustrative company:
curl https://api.neuralverge.ai/functions/v1/run-research \
-H "Authorization: Bearer <API_KEY>" \
-H "Content-Type: application/json" \
-d '{
"instructions": "Summarize the corporate structure and ownership of Acme Oy (Finland)",
"settings": { "country_code": "fi", "search_enabled": true, "deepsearch_model": "core" }
}'
The call returns a session_id immediately. You then poll get-session-status with that id until the status is complete. The result has this shape, shortened:
{
"session_id": "11111111-1111-1111-1111-111111111111",
"status": "complete",
"results": {
"kind": "deepsearch",
"human": "# Acme Oy\n\nAcme Oy is a private limited company registered with YTJ...",
"machine": { "summary": "Acme Oy is a private limited company registered with YTJ..." }
}
}
human is the readable answer and machine the structured form. The example is illustrative, not a real company's response. Check the API reference for the exact citation fields, then run the five checks on them.
AI research itself is REST only and is not an MCP tool. What an agent can reach over MCP are the pieces around it: Search, AI extraction (the run_extract tool) and every data source in the catalog, each exposed as a tool. An agent can run several of the checks mid-task with those tools. run_search finds a second source for a claim. run_extract pulls a figure from a page into a structured field, the "extract, don't recall" fix from sign five. A registry lookup such as run_uk_companies_search returns a structured record to compare against.
Add https://mcp.neuralverge.ai/mcp to an MCP client such as Claude, ChatGPT, Cursor or VS Code, sign in through the browser and click Allow. Setup guides are at https://docs.neuralverge.ai/connect-a-client, and the MCP server for Claude walkthrough covers Claude. Tool calls return the same response shape as REST and cost the same, so choosing MCP is an integration decision, not a cost one. A practical split: run research from a pipeline over REST, and give an interactive agent the MCP tools to verify a citation or look up a record while it works.
Cost and limits
Figures are from the pricing page at the time of writing. AI research is priced by depth tier, not a flat rate: Lite is 20 points (typically 30 to 90 seconds), Base 50, Core 100, Pro 200 and Ultima 400 (typically 5 to 12 minutes). Plans start at $50 a month for 50,000 points. A narrow question with one clear authoritative source suits Lite; a broad or ambiguous one needs the cross-checks of a higher tier.
The checks in this post cost no points when run in your own code. An agent that re-checks a figure with Search or AI extraction spends points on those calls at their own rates, shown on the pricing page.
Limits: citations make an answer checkable, not correct, and a lighter tier runs fewer cross-checks than a heavier one. Run the five checks on your own queries before trusting any product.
How to decide
Answer each by running a real query, not by reading a features page.
- —Who writes the citations, the pipeline or the model? If the model can add a source that was never retrieved, signs one and two are open.
- —Is there a per-claim link to a supporting passage? Ask for a sample where one sentence maps to the exact text behind it.
- —What does the source mix look like across ten varied questions? Tabulate the domains and compare with what you would expect.
- —Are dates carried with citations? The response should distinguish "as of the source" from "today."
- —What happens with an unanswerable question? An honest system reports the gap.
- —Is there a verification step, and can you see why it rejected something?
- —Is this a one-off review or a standing control? For a one-off, run the five checks by hand on twenty answers. For a standing control, put the mechanical ones before release and sample the rest.
- —Does the answer drive a decision? For KYB, due diligence or anything a person will act on, keep a human reviewer on the claims that matter.
A product that cannot pass a hand-built version of these tests will not be rescued by its description.
Frequently asked questions
Can I automate these five checks?
Mostly, yes. Resolving URLs, checking whether a cited page contains the claimed figure, and comparing dates are mechanical and fit in a test harness or a verification step before an answer is released. The source-mix check is a report you run over a batch of answers. Judging whether a passage really supports a claim often needs a second model call or a human sample, and a human should stay in the loop for high-stakes answers.
If every citation passes all five checks, is the answer correct?
Not necessarily. The checks test whether the answer is faithful to its cited sources, not whether the sources themselves are right. A registry record can be out of date and a news article can be wrong. What passing gives you is an answer where every claim points to something real that says what the answer says, so an error can be traced to a specific source and corrected.
How often should I run these checks?
Run the cheap mechanical checks on every answer that reaches a user or a downstream system. Run the heavier checks, such as claim-in-source review and source-mix reports, on a regular sample and whenever you change a model, a prompt or a retrieval setting. Sources change over time, so re-check a citation before relying on it again later.
What should an agent do when it cannot find a source for a claim?
Say so. A good answer marks the claim as unsupported or leaves it out, and states which sub-question came back inconclusive. An answer that fills the gap with a plausible figure and a plausible-looking link is the failure these five checks are designed to catch.
Can I check research citations over MCP, or only through the API?
AI research itself is called over the REST API and is not an MCP tool. Search, AI extraction and the data sources are MCP tools, so an agent connected to https://mcp.neuralverge.ai/mcp can use them to look for a second source, extract a figure from a page or pull a registry record while it checks a citation. Calls return the same response shape and cost the same as over REST.
Related articles
An AI deep research API plans, searches, cross-checks and cites. How it works, what each tier costs, and why research runs over REST while Search is on MCP.
A grounded LLM answers API plans, searches, cross-checks and cites each claim. How the stages work, where products differ, and what runs over REST or MCP.
LLM citation grounding beats a confidence score because a citation can be checked. Cited research runs over REST; Search and sources work over MCP.
About NeuralVerge
Give your agents structured, cited, real-world data from 300+ sources and the open web — through one API or MCP server.
AI Deep Research on the NeuralVerge blog.