# LLM Web Data Extractor

> Turn public web pages into JSON with your own OpenAI, Anthropic or Gemini key. List the fields you want or give a JSON Schema. One row per page.

URL: https://cliqtomedia.com/actors/llm-web-extractor/

Category: AI-ready content & documents · Status: coming soon · Apify profile: https://apify.com/cliqtomedia?fpr=cliqto

Give page links and the fields you want. The Actor reads each page and asks an AI model for the values. You get one JSON row per page. You use your own API key.

## How to use it: 3 steps

1. **Follow us on Apify.** The tool is built and waits in the queue for the Apify Store. Apify is the cloud platform where our tools run.
2. **Paste your input and press Start.** No code needed: you fill in a form (the input). It already has a working example. For example: `https://apify.com/pricing` in Page URLs. Each start is one run.
3. **Download the results.** When the run ends, the results are in a table (the dataset). Open Output, press Export and pick Excel or CSV. JSON and the API are there too.

## What it does

Turn public web pages into structured JSON with your own OpenAI, Anthropic or Google Gemini API key: give page URLs and the fields you want, and get one row per page with the values and the tokens your key used.

- Values that are not on the page come back as `null`. The model is told never to guess.
- List mode returns every record on a page (all plans, all products, all people) in `data.items`.

### Good for

- The same fields (name, price, date, contact page, specs) from pages on many different sites.
- Pricing tables, product pages, team pages, event pages, job posts: anything readable in the page HTML.
- Teams that already have an OpenAI or Gemini key and want to pay their provider directly for tokens.

### Not a good fit

- Pages that build their content with JavaScript. They are reported as `JS_REQUIRED` and no model call is made.
- Pages behind a login, paywall or bot protection. The Actor never logs in and never bypasses them; HTTP 401/403 pages are errors.
- Crawling a whole site. The Actor reads only the URLs you give (up to 300 per run). It does not follow links.
- Guaranteed accuracy. LLMs can make mistakes; check important values.

## Price

Prices will be shown after release.

## Questions

### Do I need my own API key?

Yes. Use a key from Google Gemini, OpenAI or Anthropic. Your provider bills the tokens.

### Is Anthropic tested?

Not yet with a live key. It follows the public API docs. Gemini and OpenAI are tested.

### Does it work on pages that need JavaScript?

No. Those pages are marked JS_REQUIRED and no model call is made.

### What if a value is not on the page?

It comes back as null. The model is told not to guess.

### Is Anthropic supported?

The Anthropic option follows the public API docs, but Anthropic: not tested with a live key yet. Gemini and OpenAI are tested.

### Are the values always correct?

No guarantee: LLMs can make mistakes. The model sees only the page text, is told to use `null` instead of guessing, and treats the page as untrusted data. Check important values.

### How fresh is the data?

Each page is downloaded during the run; `fetchedAt` shows when. Nothing is cached.

## Data fields

- `url` (string): Page URL from the input.
- `finalUrl` (string): URL after redirects; null when the page was not downloaded.
- `status` (string): complete = data extracted; not_found = HTTP 404/410, or the model found none of the requested data (error.code NO_DATA, only the $0.001 page read charged); error = see error.code; skipped = not processed (quota, budget, timeout); api_key_required / api_key_invalid = key problem, nothing processed.
- `data` (object): Extracted fields. Values not on the page are null. In list mode the records are in data.items. null unless status is complete or not_found with NO_DATA (then it holds the empty answer).
- `provider` (string): gemini, openai or anthropic.
- `model` (string): Model id used.
- `inputTokens` (integer): Prompt tokens the provider billed to your key for this page. null when no model call was made.
- `outputTokens` (integer): Output tokens billed to your key, including reasoning or thinking tokens.
- `reasoningTokens` (integer): Part of outputTokens spent on reasoning, when the provider reports it.
- `pageTitle` (string): Page title.
- `pageChars` (integer): Characters of clean page text before any cut.
- `truncated` (boolean): True when the page text was cut to maxPageChars before it was sent.
- `fetchedAt` (string): When the row was written, ISO 8601.
- `error` (object): Why the row is not complete: code (e.g. QUOTA_EXCEEDED, JS_REQUIRED, HTTP_ERROR, LLM_TRUNCATED) and a short message. null for complete rows.

## Input settings

- `urls` (array, required): Public pages to read (up to 300 per run; 300 pages take about 18 minutes on a free Gemini key, so raise the run timeout). One model call per page. Pages behind a login or paywall, private and internal addresses are not read. Pages that render only with JavaScript are reported as JS_REQUIRED without a model call.
- `fields` (array): Field name on the left, a short description for the model on the right. Add a type after a colon: price:number, count:integer, in_stock:boolean, tags:array (text by default). Missing values come back as null. Ignored when a JSON Schema is given.
- `mode` (string): One record: the fields describe the page itself (an article, a product). List: the page holds many records (plans, products, people) and you get them all in data.items.
- `provider` (string): Whose API your key belongs to. Gemini has a free tier in Google AI Studio. Anthropic support is built to the public API docs but not yet tested with a live key.
- `apiKey` (string): Your own key for the provider above. Required: without it the run stops at once with a hint and nothing is charged for extraction. The provider bills the tokens to this key. The key is stored encrypted, used for this run only and never written to logs or results. Gemini: aistudio.google.com/apikey · OpenAI: platform.openai.com/api-keys · Anthropic: console.anthropic.com/settings/keys
- `model` (string): Model id from your provider. Empty = a fast, low-cost default: gemini-flash-lite-latest, gpt-5.4-mini or claude-haiku-5-5. Reasoning models are slower and use more tokens.
- `jsonSchema` (object): Instead of fields, paste a JSON Schema whose root is an object with properties. It replaces the fields above. OpenAI uses strict mode only when every object lists all its properties as required and sets additionalProperties to false.
- `instructions` (string): Optional hints for the model, up to 4000 characters. Example: give prices as numbers in USD; skip enterprise plans.
- `maxPageChars` (integer): Page text longer than this is cut before it is sent, to cap the tokens your key pays for (about 4 characters per token). The row says when a page was cut.
- `maxConcurrency` (integer): Parallel pages. Lower it if your key has a small rate limit (for example a free tier).
- `keepBoilerplate` (boolean): Send the whole page, including menus, headers and footers. Off by default to save tokens.

## Step by step on Apify

1. Open the Input tab. The form is prefilled with `https://apify.com/pricing` and two fields: `plan_name` and `monthly_price`, in list mode.
2. Keep LLM provider = Google Gemini (default) or pick OpenAI.
3. Paste your key into Your API key.
4. Click Start.
5. Open the Output tab and check `status` and `data.items`.

## Use cases

| Task | Input | You get | Key fields | Next step |
|---|---|---|---|---|
| Compare competitor pricing | `urls` + `fields`, `mode: list` | one row per pricing page | `data.items`, `status`, `fetchedAt` | Google Sheets, CSV export |
| Product specs from many shops | `urls` + `jsonSchema` | one row per product page | `data`, `truncated`, `status` | Dataset API, webhook to your database |
| Company facts from About pages | `urls` + `fields` | one row per company page | `data`, `pageTitle`, `finalUrl` | CRM import via Make or Zapier |
| Track token use per page | any | one row per page | `inputTokens`, `outputTokens`, `model` | your cost report |

## Tips

- Use list mode when a page has many records, like a pricing table.
- Limit page text to keep token costs low.
- Check important values. AI models can make mistakes.

## Limits



- Max total charge (Apify run option): when the next page would go over it, the run stops, marks the rest `skipped` / `CHARGE_LIMIT` and ends `PARTIAL`. The model is not called for a page that cannot be charged.
- No data at the start: if the model finds none of your fields on 10 pages before any page had data, the run stops with `PARTIAL` / `NO_DATA_STREAK` to save your tokens.
- Dead sites or provider: after 10 timeouts or connection/model errors in a row, or when no page finishes for 120 s, the run stops with `PARTIAL` / `SOURCE_FAILING` and marks the rest `skipped`.
- Up to 300 URLs per run. Default timeout 300 s, memory 1 GB.
- Server HTML only, no JavaScript rendering, no login, no paywall or bot-protection bypass. Private and internal addresses are blocked.
- Pages up to 5 MB of HTML; text sent to the model up to `maxPageChars`.
- Each request to a site identifies itself as `llm-web-extractor-apify-actor`.

## Sample result

```json
{
  "url": "https://apify.com/pricing",
  "finalUrl": "https://apify.com/pricing",
  "status": "complete",
  "data": { "items": [
    { "plan_name": "Free", "monthly_price": "$0" },
    { "plan_name": "Starter", "monthly_price": "$19" },
    { "plan_name": "Scale", "monthly_price": "$199" },
    { "plan_name": "Business", "monthly_price": "$999" }
  ] },
  "provider": "gemini",
  "model": "gemini-flash-lite-latest",
  "inputTokens": 4906,
  "outputTokens": 121,
  "reasoningTokens": 0,
  "pageTitle": "Apify pricing - flexible plan + pay as you go · Apify",
  "pageChars": 28144,
  "truncated": false,
  "fetchedAt": "2026-10-22T05:50:56.165Z",
  "error": null
}
```

## Similar tools

- [RSS & Atom Feed Reader / Monitor](https://cliqtomedia.com/actors/rss-atom-feed-reader-monitor/)
- [WordPress Posts & Pages Scraper](https://cliqtomedia.com/actors/wordpress-posts-pages-scraper/)
- [News Article Extractor: RSS, URLs & Clean Text](https://cliqtomedia.com/actors/news-article-extractor/)
- [PDF & Image to Text OCR for AI Agents - No Login](https://cliqtomedia.com/actors/pdf-text-ocr/)
- [RAG Web Search: Google Results as Clean Markdown](https://cliqtomedia.com/actors/rag-web-search/)
- [Website Screenshot & PDF Generator: Full Page & Mobile](https://cliqtomedia.com/actors/website-screenshot-pdf/)