Web scraping and data glossary

This web scraping glossary explains common words in plain English. New to all this? Read the 8 words below first.

The 8 words you need first

All other words are below, in groups. Use your browser search (Ctrl+F or Cmd+F) to find a word fast. For the full steps, see How it works.

How to use this web scraping glossary

Web scraping basics

These are the main web scraping terms. You will see them on almost every tool page.

Web scraping

Web scraping means using a program to read data from web pages. The program saves the data in a table or a file. For example, it saves the title and price of every product on a page. All our Actors do some kind of web scraping or data collection.

Scraper

A scraper is a program that takes data out of web pages. It finds the parts you need, like names, prices or dates. Then it saves them as clean fields. Example: TrustRadius Reviews Scraper.

Web crawler

What is a web crawler? A crawler is a program that finds pages by following links. It starts at one page and visits the links on it, again and again. A scraper then takes data from the pages the crawler found. Website to Markdown for RAG: Crawl & Changed Pages crawls a docs site and saves each page.

Headless browser

What is a headless browser? It is a web browser with no window. Programs use it to open pages that need JavaScript. It sees the page like a real user does. It is slower than a simple request, so good tools use it only when needed.

HTML

HTML is the code that web pages are made of. It uses tags like <h1> for titles and <a> for links. Scrapers read the HTML to find data. Readers do not see the HTML, only the page it builds.

JavaScript rendering

Some pages build their content with JavaScript after they load. A simple scraper sees an empty page. A headless browser runs the JavaScript first, then reads the full page. Some of our tools have a "render JavaScript" option for this.

URL

A URL is the address of a web page, like https://example.com/page. Most tools take a list of URLs as input. Always use the full URL with https://.

Pagination

Pagination means that results are split over many pages, like page 1, 2, 3. A good scraper follows the pages for you. You set how many results you want, and the tool stops there.

Structured data

Structured data is data with clear fields and types. For example: "price": 19.99, "currency": "EUR". It is easy to sort, filter and use in Excel or AI. Our Actors always return structured data.

JSON-LD

JSON-LD is a way to add structured data inside a web page. Sites use it to tell search engines about products, events, recipes and more. It sits in a hidden <script> tag. Website JSON-LD Structured Data Scraper reads it for you.

Open Graph

Open Graph is a set of meta tags in a page. They tell social apps which title, text and image to show when someone shares a link. Example tag: og:title. Website Metadata & Open Graph Scraper collects these tags.

Metadata

Metadata is "data about data". For a web page, it is the title, the description, the language and more. For a video, it is the length, the date and the view count.

Sitemap

A sitemap is a file that lists the pages of a website. Most sitemaps are XML files, like /sitemap.xml. They help search engines and tools find every page fast. Try XML Sitemap URLs Extractor.

robots.txt

What is robots.txt? It is a text file at the root of a website, like example.com/robots.txt. It tells bots which pages they should not visit. It also often links to the sitemap. Good tools respect it.

SERP

SERP means search engine results page. It is the page you see after you search on Google or Bing. It has links, ads, and boxes like "People also ask". See Google Search Results Scraper (SERP) - No Login, Per Page and Search, SEO & ads.

ATS

ATS means applicant tracking system. Companies use an ATS to post jobs and manage candidates. Greenhouse, Lever, Ashby and Workable are ATS tools. Our Jobs & hiring tools read public job boards from many ATS.

API

An API is a way for programs to talk to each other and share data, without a person clicking. (API stands for application programming interface.) Some sites have an official API. Apify also has an API to run tools from your code.

Public data

Public data is data anyone can see without a login or payment. Most of our tools read only public data. Still, check each site's terms and your local law. See our FAQ.

Apify terms

Apify is the cloud platform where our Actors run. These words come from Apify.

Actor

What is an Apify Actor? An Actor is a cloud program on Apify. It takes input, does one task and saves the results. Our Apify tools are all Actors. Read more in How it works.

Apify Store

The Apify Store is a marketplace of ready-made Actors. Many developers publish tools there, including us. You can find all our Actors on our Apify profile.

Apify Console

Console is the Apify website where you run Actors. There you see your runs, datasets, tasks, schedules and bills. You open it after you sign in.

Input schema

The input schema is the list of fields an Actor accepts. Apify uses it to build the input form. It also sets default values and checks your input before the run.

Run

A run is one start of an Actor. Each run has a status, a log and its own storage. Example statuses: Running, Succeeded, Failed.

Log

The log is the list of messages an Actor writes while it works. It tells you what the tool did, what it skipped and why. Read the log first when a run fails.

Dataset

What is a dataset? A dataset is the table where a run saves its results. Each row is one item, like one job or one review. You can export it to CSV, JSON, Excel and more. How long unnamed datasets stay depends on your Apify plan and settings. Name a dataset or export it to keep your data.

Key-value store

A key-value store is a place to save files and single records for a run. For example, a screenshot, a PDF or a run summary. Each record has a name (the key) and content (the value).

Task

A task is a saved input for one Actor. You fill the form once and save it with a name. Then you can run it again, schedule it or connect it to apps.

Schedule

A schedule starts Actors or tasks at set times. For example, every day at 08:00. Apify schedules use cron rules and your timezone.

Webhook

A webhook is a message that Apify sends to a URL when something happens. For example, when a run succeeds. Your app can then load the results. See Integrations.

API token

An API token is a secret key for your Apify account. Code, apps and AI agents use it to run tools for you. Keep it private, like a password. You find it in Console under Settings → API & Integrations.

Pay per event

Pay per event is a pricing model on Apify. You pay a small fixed price for each event, like a run start or one saved result. The developer sets the events and prices. See Pricing.

Pay per result

Pay per result means you pay for each row a tool saves. It is a simple type of pay per event. No results, no result fees.

Compute unit

A compute unit (CU) measures computer work on Apify. One CU is 1 GB of memory for one hour. For most pay-per-event tools, you do not need to think about CUs.

Memory

Memory is how much computer power a run gets. More memory can make a run faster, but it can also cost more on usage-based tools. Our tools come with good default settings.

Data formats

These are the file types you can export from a dataset.

JSON

JSON is a text format for data. It uses names and values in curly brackets, like {"name": "Anna", "age": 30}. It is the main format for APIs, code and AI tools.

JSONL

JSONL means "JSON Lines". Each line of the file is one JSON object. It is good for very big files, because programs can read it line by line.

CSV

CSV means comma-separated values. It is a simple table in a text file. Each line is a row, and commas split the columns. Excel and Google Sheets open it easily.

Excel (XLSX)

XLSX is the file type of Microsoft Excel. Apify can export a dataset directly as XLSX. Use it when your team works in Excel.

XML

XML is an older text format with tags, a bit like HTML. Some business systems and sitemaps still use it.

Markdown

Markdown is plain text with simple marks. For example, # makes a heading and - makes a list. AI models read it very well. See Fast Webpage to Markdown API for AI Agents.

RSS

RSS is a feed format that lists the newest posts of a site. Atom is a similar format. News apps and feed readers use them. Try RSS & Atom Feed Reader / Monitor.

Access, blocking and proxies

Websites protect themselves from too many bots. These words explain how.

IP address

An IP address is the number that identifies a computer on the internet. Websites see the IP of every visitor. Many requests from one IP can look like a bot.

Proxy

What is a proxy in web scraping? A proxy is a server between your tool and the website. The website sees the proxy's IP address, not yours. Tools use proxies to spread requests, so no single address sends too many. Apify has its own proxy service.

Datacenter proxy

A datacenter proxy uses an IP address from a cloud company. It is fast and cheap. But some sites block these IPs more often.

Residential proxy

A residential proxy uses an IP address from a normal home internet connection. It is slower and costs more than a datacenter proxy.

Rate limit

A rate limit is the maximum number of requests a site or API allows in a time. For example, 60 requests per minute. Good tools slow down to stay under the limit.

CAPTCHA

A CAPTCHA is a test that checks if you are a human. For example, "click all pictures with a bus". Sites show it when they think a visitor is a bot.

User agent

The user agent is a short text that a browser sends to a site. It says which browser and system you use. Sites may block requests with strange user agents.

HTTP status code

A status code is a number a website sends back for each request. 200 means OK, 404 means not found, 429 means too many requests. Check many links at once with Link Route Audit — Bulk HTTP Status & Redirects.

Redirect

A redirect sends a visitor from one URL to another. For example, from http:// to https://. Codes 301 and 302 mean redirects.

Login wall

A login wall means you must sign in to see the data. Most of our Actors need no login and read public pages. Some, like eBay sold listings, need your own session or an API key.

Terms of service

Terms of service are the rules of a website. Some terms limit scraping or reuse of data. Read them before you collect data. This is not legal advice.

Automation terms

These words help you run tools without clicks.

Cron

Cron is a short code for times. For example, 0 8 * * 1 means "every Monday at 08:00". Apify schedules use cron, and also simple words like @daily.

Integration

An integration connects two apps so data moves between them. For example, Apify and Google Sheets. See Integrations.

Make, Zapier and n8n

These are no-code automation tools. You connect blocks: a trigger, then actions. Each has an Apify app or node to run our tools and load results.

Trigger

A trigger is the event that starts an automation. For example, "a run finished" or "a new row was added".

Monitor

A monitor runs again and again and looks for changes. Our monitor tools mark rows as new, changed or removed since the last run. Example: GitHub Releases Monitor & Scraper.

Baseline

A baseline is the saved "last state" that a monitor compares with. The first run creates the baseline. Later runs show what changed since then.

Deduplication

Deduplication means removing rows that appear twice. For example, the same job on two pages. Our tools try to save each item only once.

Client library

A client library is ready-made code for an API. Apify has official clients for Python and JavaScript. They make it easy to run tools from your code.

Business data terms

These words often come up when you work with web data. Many of them link to a use case page.

Marketing data

Marketing data means facts about your market: leads, prices, reviews, keywords and what people say online. Our Actors can collect this data for you. See all use cases.

Lead generation

Lead generation means finding people or companies that may buy from you. A lead list is a table of names, websites and contacts. See lead lists.

Lead enrichment

Lead enrichment means adding facts to a lead you already have. For example, an email, a phone number, a tech stack or a company size.

Buying signal

A buying signal is a sign that a company may need your product soon. For example, new jobs, new funding or a new tool on its site. See buying signals.

Competitor monitoring

Competitor monitoring means you watch what other companies in your market do. For example, new pages, new prices, new products or new jobs. See competitor watch.

Price monitoring

Price monitoring means you check product prices in shops on a schedule. You see when a price goes up or down. See price tracking.

Review monitoring

Review monitoring means you collect new reviews about your brand from many sites in one place. Then you can reply fast. See review monitoring.

Social listening

Social listening means you track what people say about a brand, a product or a topic on social sites and forums. See social listening.

Content repurposing

Content repurposing means you turn one piece of content into many. For example, a video becomes a blog post, a newsletter and social posts. See content repurposing.

Creator outreach

Creator outreach means you find bloggers, YouTubers or podcasters for a campaign and contact them. See creator outreach.

CRM

A CRM is an app that keeps your contacts, companies and deals. Examples are HubSpot, Pipedrive and Salesforce. Our tools can send new rows to a CRM.

AI and LLM terms

These words come up when you use web data with AI. See also AI agents.

LLM

LLM means large language model. It is an AI model that reads and writes text. ChatGPT, Claude and Gemini are built on LLMs.

AI agent

An AI agent is an AI model that can use tools. It can search, open pages, run Actors and save files. It plans steps to reach a goal.

MCP

MCP means Model Context Protocol. It is an open standard that connects AI apps to tools. The Apify MCP server lets AI apps use Apify Actors, including ours.

RAG

What is RAG? RAG means retrieval-augmented generation. An AI model first finds useful text, then uses it to answer. This gives better, fresher answers. Try RAG Web Search: Google Results as Clean Markdown.

Embedding

An embedding is a list of numbers that shows the meaning of a text. Texts with similar meaning get similar numbers. RAG systems use embeddings to find the right text.

Vector database

A vector database stores embeddings. It can find texts with a similar meaning very fast. RAG apps keep their chunks here.

Chunk

A chunk is a small piece of a long text, like one section of a page. AI apps split long documents into chunks. PDF to Markdown for LLMs & RAG Chunks - No Login returns ready chunks.

Token

A token is a small piece of a word. LLMs count text in tokens. Prices and limits of AI models are often per token. Clean Markdown uses fewer tokens than raw HTML.

Prompt

A prompt is the text you give an AI model. It can be a question or an instruction. Clear prompts with good data give better answers.

Hallucination

A hallucination is when an AI model makes up facts. Fresh, real data from the web helps the model stay correct.

OCR

OCR means optical character recognition. It reads text from images and scanned PDFs. Try PDF & Image to Text OCR for AI Agents - No Login.

Transcript

A transcript is the written text of a video or audio file. It can include times for each line. See YouTube Subtitles & Transcript Scraper API and Video, audio & streaming.

Speech to text

Speech to text turns spoken words into written text. It is how you get a transcript when a video has no subtitles. Try Audio & Podcast Transcriber (Speech to Text).

LLM-ready data

LLM-ready data is clean data that AI models can read easily. It has no menus, ads or cookie banners. Markdown and JSON are the best formats. See AI-ready content & documents.

This web scraping glossary grows with our tools. When we add a new kind of tool, we add its words here too.

Browser extension terms

Browser extension

A browser extension is a small program that adds a feature to your browser. Our browser extensions run on your own computer.

Chrome Web Store

The Chrome Web Store is Google's official shop for Chrome extensions. You install extensions from it with one click.

Manifest V3

Manifest V3 is the current set of rules for Chrome extensions. It limits what an extension can do. This keeps you safer.

Content script

A content script is extension code that runs on one website. For example, Bulk Post Scheduler for X works only on x.com.

Local extension storage

Local extension storage keeps data inside your browser, for example in IndexedDB. It does not go to a server. Bulk Post Scheduler for X keeps your queue there.

Questions

Last updated: 20 October 2026