Structured Data Extraction LLM: Building a Price-Monitoring Agent

A structured data extraction llm call turns a competitor's pricing page into the same JSON fields on every run, and that stable shape is what a price-monitoring agent needs in order to compare today's prices with last week's. This guide builds the monitor in six steps (schema, extraction call, storage, diffing, alerting, scheduling) for pricing pages you name yourself, over REST or MCP. For prices on marketplace listings, see price monitoring across Amazon, Walmart and Target.

The short version

• —What you build. A scheduled job that extracts plan name, price, billing period and limits from each competitor's pricing page, compares the result with the stored one and alerts on real price changes.
• —Who it is for. Product marketing, sales and founders who today check competitor pages by hand, and developers building a competitive-intelligence agent.
• —What it costs. One AI extraction call is 5 points. Twenty competitors checked daily is about 600 calls a month, or 3,000 points, against 50,000 points on the $50 Base plan.
• —How you reach it. Over REST from a scheduled job, or as the run_extract MCP tool when an agent decides which pages to read. Same inputs, same output, same price.
• —What it does not do. It reads the URLs you give it. It does not find new pricing pages or log in to anything.

What the monitor has to do

Strip away the tooling and the task is small. For each competitor, on a schedule, pull the same fields from the pricing page, compare them with what you pulled last time and flag what changed. Done by hand, it is ten browser tabs on whatever cadence someone remembers, skipped in the busy weeks.

Three pieces make up the system, and each fails in its own way:

• —Extraction turns a page into structured fields.
• —Storage remembers the last good result per competitor.
• —Diffing and alerting decide whether a difference deserves a human's attention.

Home-grown monitoring scripts get fragile when these three are blended into one "scrape the page" step. Keeping them separate means you can fix the diffing logic without touching extraction, or swap the storage layer without touching either.

Building it with structured data extraction

An LLM-based extraction call fits the first piece because a diff needs a stable shape. You describe the fields once, in plain language and optionally as a JSON Schema, and the call returns those fields from whatever page you point it at. Cards in a row, rows in a table and a long scrolling page can all map to the same schema, because the fields are matched by meaning rather than by position in the markup. That is the claim the product makes; test it on your own competitors' pages before relying on it, since pricing pages are often the most customised pages a company has.

Step 1: Define the schema

The schema decides what is comparable across runs and across competitors, so it is worth the most thought. A starting point for pricing pages, as JSON Schema:

{
  "type": "object",
  "properties": {
    "plans": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "plan_name": { "type": "string" },
          "price": { "type": "number" },
          "billing_period": { "type": "string" },
          "included_limits": { "type": "string" }
        },
        "required": ["plan_name"]
      }
    }
  }
}

Two choices matter later. price is a number, not a string like "$49/month", so it diffs cleanly. plan_name is the only required field, because it becomes the comparison key in step 4. One schema serves every competitor; only the URL changes.

Step 2: Call the extraction endpoint

Pulling one competitor's page against that schema is one run-extract request: the page URL, a plain-language instruction and the schema as a JSON string in settings.extract_schema_json.

curl https://api.neuralverge.ai/functions/v1/run-extract \
  -H "Authorization: Bearer <API_KEY>" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://competitor.example.com/pricing",
    "instructions": "List every pricing plan with its name, price, billing period and included limits.",
    "settings": {
      "country_code": "us",
      "extract_schema_json": "{\"type\":\"object\",\"properties\":{\"plans\":{\"type\":\"array\",\"items\":{\"type\":\"object\",\"properties\":{\"plan_name\":{\"type\":\"string\"},\"price\":{\"type\":\"number\"},\"billing_period\":{\"type\":\"string\"},\"included_limits\":{\"type\":\"string\"}},\"required\":[\"plan_name\"]}}}}"
    }
  }'

The response uses the same envelope as every AI extraction call: session_id, kind, url, and the extracted fields under machine. For this schema, machine holds the plans array. A response would look like this (the values are illustrative, since the page is a placeholder):

{
  "session_id": "11111111-1111-1111-1111-111111111111",
  "kind": "extract",
  "url": "https://competitor.example.com/pricing",
  "machine": {
    "plans": [
      { "plan_name": "Starter", "price": 19, "billing_period": "month", "included_limits": "5 seats" },
      { "plan_name": "Growth", "price": 49, "billing_period": "month", "included_limits": "25 seats" }
    ]
  }
}

Run once per competitor per interval, this call is the whole extraction half of the system. Endpoint and parameter details live in the documentation rather than here, so they do not drift.

Step 3: Store the last good result

One extraction is only useful relative to the previous one. Keep, at minimum, the most recent successful result per competitor, keyed by URL or competitor name, plus a timestamp. A table with one row per competitor and a JSON column is enough for most teams. Two rules keep it honest:

• —Write only after a successful run. If an extraction fails or comes back empty, do not overwrite the baseline with nothing, or the next good run will report every plan as new.
• —Keep a short history. Storing each run, not just the latest, lets you chart price over time later and re-run the diffing logic against past data when you change it.

Extraction does not know what happened last time, and should not. The comparison belongs entirely to the next step.

Step 4: Diff on fields, keyed by plan name

Diffing is where most of the judgment sits, and where a naive version produces the most noise. Comparing raw JSON and flagging any difference catches real price changes, but it also catches a reworded limits description or a capitalisation tweak. Decide field by field what counts:

The second trap is array order. If a competitor reorders its plan cards, diffing by position reports a wall of changes when nothing changed. Key the comparison by plan_name and diff each plan's fields independently:

type Plan = { plan_name: string; price?: number; billing_period?: string };

function diffPlans(previous: Plan[], current: Plan[]) {
  const before = new Map(previous.map((p) => [p.plan_name, p]));
  const after = new Map(current.map((p) => [p.plan_name, p]));
  const changes: string[] = [];

  for (const [name, plan] of after) {
    const old = before.get(name);
    if (!old) changes.push(`New plan: ${name}`);
    else if (old.price !== plan.price)
      changes.push(`${name}: ${old.price} -> ${plan.price}`);
  }
  for (const name of before.keys()) {
    if (!after.has(name)) changes.push(`Plan removed: ${name}`);
  }
  return changes;
}

This is a sketch of the idea, not a library; extend it with the other fields from the table.

Step 5: Alert by severity

Not every detected change deserves the same response. A price increase on a competitor's flagship plan should reach whoever owns positioning the same day. A reworded limit might be worth logging and nothing more. If every diff fires the same notification, the channel gets muted after the second false alarm and the system stops doing its job.

A workable default: send an immediate message for any price change or any added or removed plan, and fold everything else into a weekly digest. Include the plan, the old value, the new value, the page URL and the time of the run, so the reader can open the page and confirm in seconds.

Step 6: Schedule by how often each page changes

Most B2B pricing pages change a handful of times a year, so a daily run is a reasonable default. Running more often than prices actually change adds calls without adding signal. The exception is a competitor known for time-limited promotions, where an hourly check on that one page is justified while the rest stay on a daily schedule. Match the cadence to each source's real change rate instead of applying one interval to the whole list.

Any scheduler works: cron, a workflow runner, or a cloud scheduled function. The job loops over the competitor list, calls extraction, diffs, writes the new baseline on success and sends alerts.

A worked example: one price change

Take an illustrative case. A competitor quietly raises its mid-tier plan from $49 to $59 a month, with no announcement.

The next scheduled run extracts the page against the standing schema and gets back the Growth plan with price 59, alongside the unchanged plans. The diffing step compares it with the stored result, finds price changed from 49 to 59 on the plan keyed Growth, and, because price changes alert immediately, sends a message naming the plan, the old price and the new one. The new result is written as the baseline for the next run.

Nobody had to notice the change by eye, and the alert goes out on the day of the first run after the change instead of whenever someone next opens that tab. If the same run had also returned a reworded limits line with the same number, the diff would have ignored it.

REST and MCP

The build above calls REST from a scheduled job, which is the right default when the list of pages is fixed. The same extraction is also an MCP tool. NeuralVerge hosts an MCP server at https://mcp.neuralverge.ai/mcp: add the URL to an MCP client such as Claude, ChatGPT, Cursor, VS Code or Gemini CLI, sign in through the browser and click Allow. There is no key to paste into a config file and nothing to host. The setup walkthrough covers each client.

Once connected, the agent sees Search, AI extraction and the data sources as tools. Here the one that matters is run_extract, which takes the same url, instructions and settings as the REST call. An agent asked "what does this competitor charge now?" would make a call like this, with a trimmed schema:

{
  "tool": "run_extract",
  "arguments": {
    "url": "https://competitor.example.com/pricing",
    "instructions": "List every pricing plan with its name, price, billing period and included limits.",
    "settings": {
      "country_code": "us",
      "extract_schema_json": "{\"type\":\"object\",\"properties\":{\"plans\":{\"type\":\"array\",\"items\":{\"type\":\"object\",\"properties\":{\"plan_name\":{\"type\":\"string\"},\"price\":{\"type\":\"number\"}},\"required\":[\"plan_name\"]}}}}"
    }
  }
}

The result has the same shape as over REST and the same price; there is no separate MCP rate, so the diffing and storage code does not care which route produced a result. Use REST for the monitoring job: the URLs are known, the schedule is fixed and your code decides when to call. Use MCP for the part a fixed pipeline cannot do, where an agent comes across a page that might be a new pricing page and chooses to extract it. Both can feed the same store. Connected clients can be revoked under Settings, then API, in the dashboard, where the label starts with "Connector".

Cost and limits

AI extraction is 5 points per call, whether it arrives over REST or MCP; current plan allowances are on the pricing page. Twenty competitors with one pricing page each, checked daily, is about 600 calls and 3,000 points a month. A competitor with separate pages per product line multiplies that, so count pages, not companies. At this volume the limit on how many competitors you track is rarely the call count; it is how much alert noise a team will read, which is why steps 4 and 5 matter more than the call budget.

Limits to plan around:

• —Empty results are billed. A call on a dead or unreadable URL can return null and still cost 5 points, so do not assume a failed call is free.
• —Fixed list only. The pipeline reads the URLs you give it. It cannot notice a pricing page a competitor just published, or a competitor you have not listed. Finding those needs a periodic crawl of each site or an agent that decides what to check next, calling run_extract once it picks a page.
• —Public pages only. Extraction does not log in, fill forms or click through a flow, so prices that appear only after an interaction, or only to signed-in customers, are out of reach.
• —Quoted prices only. Plans marked "contact sales" have no number to extract, which is why the schema leaves price optional.

How to decide

Use this pattern if most of these are true:

• —You track a known list of pages. A stable set of competitor pricing pages is the sweet spot. A list that must grow by itself is a harder problem on top.
• —The field you diff is a number. A price extracted as a number diffs cleanly; the same value inside a sentence invites false positives from rewording alone.
• —Alerts are tiered. If every diff pages someone, build the severity logic before you build the rest.
• —You can test on two or three real pages first. Run them for a couple of scheduled cycles before wiring alerts for the full list. It is the fastest way to see whether the schema and diffing hold up against how those pages are built.
• —You will not guess on empty pages. Check what your extraction returns when a field is missing or the page changed structure, and treat an empty result as a signal to look, not a value to store.

The same schema-and-diff shape also fits plan-limit tracking and a competitor's careers page as a hiring signal. For a marketplace listing with an ID, the marketplace price-monitoring guide is the better starting point.

Frequently asked questions

Do I need a separate schema for every competitor's pricing page?

Usually not. A schema-driven structured data extraction llm call maps content to the fields you ask for rather than to a page's markup, so one schema (plan name, price, billing period, included limits) can serve pages with different designs. Check the first run for each new competitor, and add optional fields rather than forking the schema.

How often should a price-monitoring agent actually run?

Daily is a reasonable default for most B2B pricing pages, which change infrequently. A competitor known to run frequent promotions can justify a faster cadence for that page alone. The right interval is set by how often prices actually change, not by how often it is possible to run.

What's the biggest source of false-positive alerts in a system like this?

Cosmetic page changes that do not touch the fields you care about, such as a redesign, a new testimonial section or reordered plan cards. Diffing on the extracted fields, keyed by plan name, rather than on the page's raw content avoids alerting on changes that do not matter.

Can this agent discover a new pricing page on its own, without being told about it?

Not by default. A fixed list of URLs is a pipeline decision made once. Discovering a page a competitor just published needs either a periodic crawl of the competitor's site or an agent reasoning over what pages exist, which then calls extraction on the page it picks.

Can I run the extraction step over MCP instead of the API?

Yes. NeuralVerge's hosted MCP server at https://mcp.neuralverge.ai/mcp exposes AI extraction as the run_extract tool, with the same inputs, outputs and price as the REST endpoint. Use REST for a scheduled monitoring run and MCP when an agent chooses which pages to extract mid-task.

What happens if a competitor blocks or restructures a page entirely?

Expect an empty or partial result, and test what yours looks like before relying on it. A call on an unreadable URL can return null and is still billed at 5 points. Treat an empty result as a signal worth alerting on, and never overwrite the stored baseline with it, since a page returning nothing usually means something changed on the source side.