AI Extraction Explained: URL & Document to JSON Without Selectors
An AI extraction API turns a page or PDF URL plus plain-language instructions into JSON, no selectors. How it works, 5 points per call, REST or MCP, and limits.
By the NeuralVerge team · 14 min read · Updated October 2026
▸On this page
An AI extraction API takes a URL, either a public web page or a PDF, plus plain-language instructions about the fields you want, and returns those fields as JSON. There are no CSS selectors to write and none to repair when a layout changes. In NeuralVerge it is the AI Extract capability, reachable over REST or as the run_extract tool on the hosted MCP server, and it costs 5 points per call. This guide covers how it works, what we tested and what we did not.
The short version
- —What it does: one call per URL. You send
urlandinstructions, optionally a JSON Schema, and get structured JSON back instead of markup. Pages are rendered like a browser would show them, then cleaned and mapped to your fields. - —Who it is for: teams that need specific fields from a known set of pages or public documents and are tired of maintaining selectors, plus agents that meet URLs mid-task. It is not a crawler and it does not log in or click through flows.
- —What it costs: 5 points per call, the same as a Search call. Points come from your plan's monthly allowance; current numbers are on the AI extraction product page and the pricing page.
- —What we tested: a public IRS Form W-9 PDF returned a correct structured result. One public sample-invoice PDF returned
nulland was still charged 5 points, so an empty call is not free. - —What we did not test: scans, direct file upload, size and page limits, accuracy beyond that one form, and speed. Run a sample of your own pages before you build on it.
What an AI extraction API is
An AI extraction API takes a URL and a description of the fields you want, and returns the matching content as structured JSON without the rest of the page. The request carries a url, plain-language instructions, and optionally a JSON Schema if you want to pin the output shape. It is easiest to place by contrasting it with what people usually reach for first.
AI extraction vs. selector-based scraping
A traditional scraper is a set of CSS or XPath selectors written against one page layout. It is cheap to run once written, but it is coupled to that exact layout. A redesign, an A/B test or a renamed class can break it, and it only breaks loudly if your monitoring catches empty fields as well as errors. When that happens you are not fixing a bug, you are rewriting the scraper.
AI extraction reads the page's content and maps it to the fields you described by meaning, not by position in the DOM. A layout change that a human reader would barely notice does not require you to touch the request, because nothing in the request depended on where the company name sat in the markup. What you give up is determinism: a selector returns the same thing every time, while a model's reading of a page is something you check rather than assume.
AI extraction vs. asking an LLM to "read this page"
Pasting raw HTML into an LLM prompt works as a one-off, but it is not a pipeline. You have to fetch the page yourself, deal with content that only exists after scripts run, keep long pages inside the context window, and coax a consistent output shape out of a prompt. An extraction endpoint packages those steps behind one call: fetching and rendering the page, stripping boilerplate, and returning the result.
AI extraction vs. a PDF text parser
A document parser pulls text out of a file format. It generally does not know that one line is a founding year and another is a headcount, so you write the logic that decides. AI extraction takes your description of the fields and returns those fields, which is why the same call shape can be pointed at a company's web page or at a PDF at a public URL. That is a statement about the call shape, not a promise about every document. The piece on turning PDFs into JSON covers documents in more depth, and invoice and receipt extraction covers what to check on supplier paperwork. The broader category is called intelligent document processing, which is explained in the guide to what intelligent document processing is.
AI extraction vs. browser automation and RPA
Browser automation tools are built to act on a page: click, type, navigate, wait. They are the right choice when the job is interacting with a site. AI extraction is narrower. It reads a public page or PDF that a URL points to and returns structured data. It does not log in, fill forms or click through a flow, so if the data sits behind an interaction, this is not the tool for that part of the job.
| Approach | You provide | You get | Breaks when |
|---|---|---|---|
| Selector scraper | CSS or XPath per page layout | Exactly what the selectors match | The layout or class names change |
| LLM on pasted HTML | A prompt plus fetched page text | Whatever shape the prompt coaxes out | The page is long, script-rendered or the output shape drifts |
| PDF text parser | A file and your own field logic | Text, not fields | The document layout varies |
| Browser automation | A script of clicks and waits | Whatever the script reaches | The flow or the interface changes |
| AI extraction | A URL and instructions, optionally a schema | JSON with the fields you described | The field is absent or the model misreads it, so check a sample |
How AI extraction works
NeuralVerge describes the pipeline as four steps. We list them as the product describes them; the internals of each step are not documented, so treat them as the intent of the step, not a specification.
1. Send a URL and instructions
The request names a public page or a PDF at a URL, and says in plain language what to extract, for example "the company name, founding year and employee range". There is nothing to install or configure per site.
2. The page is rendered
Fetching static HTML misses anything that loads client-side, which is a large part of the modern web. The product description says JavaScript-heavy pages are rendered the way a browser would show them, so content that only appears after scripts run is present when extraction happens. How reliably this works on a given site is something to test, particularly for pages that need an interaction before the content appears or that sit behind a login.
3. Content is cleaned and structured
Navigation, ads, cookie notices and boilerplate are removed, and the remaining content is mapped to the fields you asked for. This is the step that replaces a selector. Instead of "grab the text inside the company-name class", you get a mapping from meaning to field: whatever on the page reads as the company's name becomes the name field, wherever it sits in the markup.
4. You get structured JSON
The response is JSON, not raw HTML, so it can go into a database, a retrieval index or an agent's next step without a parsing pass of your own. Two details come from calls we ran rather than from a description. When you pass a schema, the typed result is in the machine field. When you pass no schema, the result comes back in the human field as JSON text and machine is null. Your code should read whichever field matches the way you called it.
A request and what comes back
The request shape is small. This is the cURL example from the AI extraction product page, using the fictional Acme example that appears across the site:
curl https://api.neuralverge.ai/functions/v1/run-extract \
-H "Authorization: Bearer <API_KEY>" \
-H "Content-Type: application/json" \
-d '{
"url": "https://example.com/company/acme",
"instructions": "Extract the company name, founding year and employee range.",
"settings": { "country_code": "us" }
}'
Three inputs matter. url is the page or PDF. instructions is the plain-language description of what you want. settings carries options, and it is where an optional JSON Schema string goes, in settings.extract_schema_json. Endpoint and parameter details change over time, so the documentation is the place to confirm them before you build against this.
The product page shows the response shape below for that Acme call. It is an illustration with a fictional company, not a recorded run:
{
"session_id": "11111111-1111-1111-1111-111111111111",
"kind": "extract",
"url": "https://example.com/company/acme",
"machine": {
"company_name": "Acme",
"founded": "2019",
"employees": "50-100"
}
}
Schema or no schema
There are two ways to tell the extractor what you want, and they suit different situations.
Instructions only. You describe the fields in plain language and pass no schema. This is the fastest way to start and the right one while you are exploring what a class of pages contains. The result comes back in the human field as JSON text, and machine is null. That text is JSON you can parse, but its structure is up to the model, so two calls on similar pages are not guaranteed to produce the same keys.
Instructions plus a schema. You pass a JSON Schema string in settings.extract_schema_json along with the instructions. Use this when a pipeline expects the same field names on every call, for instance when the result feeds a database table. The typed result is in machine. A schema narrows what the model can return, but you should still validate what arrives, because a defined shape tells you the keys, not that the values are right.
A sensible path is to start with instructions only on a small sample of real pages, look at what comes back, and then write the schema from what you saw rather than guessing up front.
Worked example: a company profile page
Take a concrete case: pulling a profile for a fictional company, Acme Oy, from the URL in the request above. This illustrates the flow and is not a recorded run.
- —You send the URL with the instruction to extract the company name, the founding year and the employee range.
- —The page is rendered, including anything a script adds after load, such as a dynamically loaded "About" section.
- —The navigation bar, footer and cookie notice are stripped, and the remaining content is mapped to the three fields you named, wherever each appears: a hero section, a sidebar or a table further down.
- —You get back one JSON object with those fields and no markup.
The part worth testing yourself is the case where the page does not list a founding year at all. The honest outcomes are an empty value or a made-up one, and the product does not state which you will get. Ask for a field you know is absent from a page you can check by eye, and look at what comes back before you put the call into anything that matters.
Worked example: a PDF at a URL
The same request shape works when the URL points to a PDF. Here we do have a recorded result. We sent a public IRS Form W-9 PDF URL with plain-language instructions and an optional JSON schema string in settings.extract_schema_json, and the call returned a correct structured result for the form's fields. We are not reproducing the output, because the point is the shape of the interaction rather than the form: a URL, instructions and optionally a schema went in, and structured fields came out.
We also ran the call on one public sample-invoice PDF, and it returned null. It was still charged 5 points. That single result tells you very little about invoices in general, since we do not know why it came back empty, whether the file, the instructions or the service was the cause. It does tell you two practical things: a call can come back empty, and an empty call costs the same as a successful one. Your code should handle a null result without crashing, and your cost estimates should assume some calls will not return what you hoped.
What we have not verified for documents
Several things people reasonably ask about documents are not stated in the product description, the documentation or the pricing page, and we have not tested them:
- —Uploading a file directly rather than giving a URL.
- —Scanned documents and image-only PDFs.
- —Maximum file size or page count.
- —How tables and multi-column layouts are read.
- —Accuracy on any document type beyond the one form we tried.
- —How long a call takes.
If any of these decides whether the product fits your use case, run your own files through it first. That is the fastest honest answer.
REST and MCP
The same extraction is reachable two ways, with the same inputs and outputs. Over REST you POST to the run-extract endpoint shown above. Over MCP, the hosted server at https://mcp.neuralverge.ai/mcp exposes it as the run_extract tool: a thin wrapper around the REST endpoint. Tool names use underscores where REST paths use hyphens. For connecting a client such as Claude, follow the MCP server for Claude setup instead of repeating it here.
Calling extraction directly from a pipeline makes sense when you already know which URLs need processing: a fixed set of company pages on a schedule, or a list of document links produced by an earlier step. You control when the call happens and what happens to the result.
Exposing it as a tool an agent can call makes sense when the agent meets URLs mid-task and needs to decide for itself that it wants structured fields rather than raw page text. The extraction step does not change between the two modes; only the way it is invoked does. The practical difference is who decides what to spend. In a pipeline, you decide the call count. With an agent, the model decides, so watch early runs to see how many 5-point calls a task actually makes.
Cost and limits
An AI Extract call costs 5 points, the same price the pricing page lists for a Search call. Points come from the monthly allowance on whichever plan you are on, so the cost of a workflow is the number of calls multiplied by 5, set against the points your plan includes. Current plan prices and allowances are on the pricing page, and the numbers there are the ones to trust over anything in a blog post.
The charge does not depend on the result. The sample-invoice call that returned null was charged 5 points like any other, so we are not going to tell you that failed or empty extractions are free. If your workload includes many URLs that may not contain what you are after, budget for the empty calls too, and check a sample before running the whole list.
On limits: the input is a public URL, so private storage and direct uploads are not covered by what is stated. The call reads one page or document and returns fields; it does not crawl a site, log in or interact with a page. Throughput, latency and per-file limits are not stated, so measure them on your own files.
How to decide
Run one real page and one real document through the call, then work through these questions:
- —Does the input fit? The input is a URL. If your documents live on private storage or arrive as uploads, confirm how they would reach the service before planning around it.
- —Does it render your pages? Anything loaded client-side must be present in the output. Test on the pages that matter, especially JavaScript-heavy ones.
- —What happens to absent fields? Ask for a field you know is not on the page. An empty value is useful; a plausible guess is a problem.
- —Can your pipeline absorb an empty, billed call? A
nullresult still costs 5 points. Handle it in code and in the budget. - —Do you need a pinned shape? Once downstream code depends on field names, pass a JSON Schema and validate the values.
- —Is it a fit for the job? Good cases are competitor and market monitoring with comparable fields from a set of pages, lead and account enrichment from a company's own website alongside ready-made data sources, ingesting cleaned content into a retrieval index, public document intake such as forms and filings, and agent tool calls. These are workflows we expect, not measured results. Check outputs against the source before trusting them at volume.
- —Do the limits fit your files? Size, page count, scans and tables are not stated for this product, so find out with a sample of your real documents.
Frequently asked questions
Does AI extraction replace a traditional scraper entirely?
For the specific pages a workflow needs, it can remove the selectors you would otherwise maintain, because you describe the fields in plain language instead of pointing at markup. It is one call per URL at 5 points each, so it is not a crawler, and fetching millions of pages a day is a different problem. Speed and throughput are not stated here, so test them on your own pages.
What happens if a page doesn't have the field I asked for?
Not stated; test it. Ask for a field you know the page lacks and see whether you get an empty value or a guess. In our own test, a call on one public sample invoice PDF returned null and was still charged 5 points, so an empty result is not free.
Can AI extraction handle JavaScript-heavy pages?
The product description says JavaScript-heavy pages are rendered the way a browser would show them. How well that holds for a particular site, or for pages behind a login, is not stated, so check it against your own URLs.
Do I need to define a schema before extracting anything?
No. Plain-language instructions are enough. Without a schema, the result comes back in the human field as JSON text and machine is null. If you need a fixed shape, pass a JSON Schema string in settings.extract_schema_json.
Does the same API handle both web pages and documents like PDFs?
The call takes a URL, and that URL can be a public page or a PDF. A public IRS Form W-9 PDF at a URL returned a correct structured result in our test. Direct file upload, scanned documents, and size or page limits are not stated, so test your own files.
Can I use AI extraction over MCP instead of REST?
Yes. NeuralVerge's hosted MCP server at https://mcp.neuralverge.ai/mcp exposes extraction as the run_extract tool, a thin wrapper around the REST endpoint with the same inputs and outputs and the same 5-point price. Tool names use underscores where REST paths use hyphens.
Related articles
A schema based web extraction API fixes field names, types and required fields before the call runs. Why that beats prompting, over REST or MCP.
An html parsing alternative ai teams can adopt step by step: what changes in code, where determinism goes, and how to call it by API or MCP.
A structured data extraction llm call turns competitor pricing pages into JSON you can diff. Build the price-monitoring agent over API or MCP, step by step.
About NeuralVerge
Give your agents structured, cited, real-world data from 300+ sources and the open web — through one API or MCP server.
AI Extraction on the NeuralVerge blog.