# XML Sitemap URL Extractor

> Sitemap URL extractor: turn XML sitemaps, sitemap indexes and .xml.gz files into a clean URL list with lastmod and priority. Export to CSV or JSON.

URL: https://cliqtomedia.com/actors/xml-sitemap-urls-extractor/

Category: Search, SEO & ads · Status: live on Apify Store · Apify page: https://apify.com/cliqtomedia/xml-sitemap-urls-extractor?fpr=cliqto

Turn a sitemap into a clean list of page URLs. It reads sitemap indexes and gzip files too. It does not visit the pages.

## How to use it: 3 steps

1. **Make a free Apify account.** Apify is the cloud platform where our tools run. The free plan gives you $5 of usage every month, no credit card needed. You pay Apify, not us.
2. **Paste your input and press Start.** No code needed: you fill in a form (the input). It already has a working example. For example: `{"url": "https://www.sitemaps.org/sitemap.xml"}` in Sitemap URLs. Each start is one run.
3. **Download the results.** When the run ends, the results are in a table (the dataset). Open Output, press Export and pick Excel or CSV. JSON and the API are there too.

## What it does

Read public XML sitemaps, sitemap indexes, and gzip sitemap files and save one deduplicated row per page URL. Each row includes `url`, `lastmod`, `changefreq`, `priority`, and the source `sitemapUrl`.

- reads public HTTP(S) XML sitemaps;
- follows nested `sitemapindex` files up to the depth you set;
- reads gzip sitemap files with bounded decompression;
- saves one row for each unique page URL;
- keeps `lastmod`, `changefreq`, `priority`, and the first sitemap that contained the URL;
- blocks private-network targets, unsafe redirects, XML DTD/entity input, oversized responses, and decompression bombs.

### Good for

- Build a clean URL inventory before a crawler or audit.
- Export sitemap metadata for SEO or migration work.
- Read a large sitemap index without visiting the listed pages.
- Turn `.xml.gz` sitemap files into a normal Dataset.

### Not a good fit

- Bare website domains. Add the direct sitemap URL instead.
- Login-protected or private-network sitemaps.
- Page status checks. The Actor does not request discovered page URLs.
- Image, video, news, or hreflang sitemap extensions.

## Price

Pay per event. Apify bills you, not us.

| What you pay for | Price | When |
|---|---|---|
| Start fee | $0.02 | Charged once per run. This tool uses 1 GB of memory or less by default; more memory costs more. |
| Sitemap URL saved | $0.00027 | One unique page URL saved to the default Dataset. |

- One URL: about $0.02.
- 1,000 URLs: about $1.27 (50 runs of up to 20).
- Every day 100 URLs, for 30 days: about $3.81 a month (5 runs a day).
- 20 URLs once a week (4 runs a month): about $0.10 a month.

Free to try: the Apify free plan gives you $5 of usage every month, no credit card needed.

Prices checked on 24 October 2026. Current price: https://apify.com/cliqtomedia/xml-sitemap-urls-extractor?fpr=cliqto

## Questions

### Can I just add my domain?

No. Add the direct sitemap URL, for example example.com/sitemap.xml.

### Does it open every page in the sitemap?

No. It reads the sitemap files only and saves the listed URLs.

### Does it read .xml.gz files?

Yes. Gzip sitemaps are read and saved as normal rows.

### Does it find a sitemap from a website homepage?

No. Add the direct sitemap URL, such as `https://example.com/sitemap.xml`.

### Does it support sitemap indexes?

Yes. It follows nested indexes up to `maxDepth` and `maxSitemaps`.

### Why are some metadata fields `null`?

They are optional in a sitemap. The Actor keeps `null` when the publisher does not provide a usable value.

## Data fields

- `url` (string): Page URL from the sitemap.
- `lastmod` (string): Last modification value, normalized to ISO 8601 when parseable.
- `changefreq` (string): Publisher-declared change frequency.
- `priority` (number): Publisher-declared priority between 0 and 1.
- `sitemapUrl` (string): First sitemap file that contained this deduplicated URL.

## Input settings

- `startUrls` (array, required): Direct public XML sitemap, sitemap index, or gzip sitemap URLs. Bare domains are not auto-discovered.
- `maxDepth` (integer): Maximum sitemap-index nesting depth. Start URLs are depth 0.
- `maxSitemaps` (integer): Maximum number of unique sitemap files fetched in one run.
- `maxResults` (integer): Stop after this many unique page URLs have been saved.
- `requestTimeoutSecs` (integer): Timeout for one sitemap HTTP request.
- `maxRetries` (integer): Retry temporary 429, 5xx, and transport failures this many times.
- `maxCompressedBytes` (integer): Compressed-byte cap for a gzip sitemap.
- `maxDecompressedBytes` (integer): Maximum sitemap size after decompression.

## Step by step on Apify

1. Open the Input tab.
2. Add one direct public sitemap URL to Sitemap URLs (`startUrls`).
3. Keep the default limits for the first run.
4. Click Start.
5. Open the Dataset and check the `url` and `sitemapUrl` fields.

## Use cases

| Task | Input | Main output | Typical next step |
|---|---|---|---|
| Build an SEO URL inventory | direct sitemap or index | `url`, `lastmod`, `sitemapUrl` | Export to CSV/JSON for analysis |
| Prepare a crawl seed list | direct sitemap or index | deduplicated `url` values | Send URLs to a separate crawler |
| Review publisher sitemap hints | sitemap URL | `lastmod`, `changefreq`, `priority` | Compare in a spreadsheet |
| Read a compressed sitemap | `.xml.gz` URL | normal Dataset rows | Export or call the Dataset API |

## Tips

- Add the direct sitemap link, not the home page.
- Set the index depth to control how deep nested sitemaps go.
- Sort by lastmod in a spreadsheet to find new pages.

## Limits

`partial` means a limit or a failing sitemap prevented a full run. It can contain zero rows, such as when an index depth limit stops all children. Check `RUN_SUMMARY` for `limitReached` and bounded issue details.

- Up to 20 start sitemap URLs per run.
- Up to 4000 unique page results.
- Up to 50 sitemap files when `maxSitemaps` is raised.
- Sitemap-index depth is bounded from 0 to 10.
- Fixed scan limits: 60 seconds of source work, 64 HTTP attempts (including retries and redirects), and 8 MiB of downloaded response bodies across the whole run.
- Each gzip file is capped at 5 MiB compressed and 8 MiB of XML. XML nesting is capped at 64 levels and each file at 50,000 entries.
- A smaller user spending limit can stop results early; summaries count only saved rows.
- Only public HTTP(S) targets on standard web ports are allowed.

## Sample result

```json
{
  "url": "https://www.sitemaps.org/",
  "lastmod": "2016-11-21T00:00:00.000Z",
  "changefreq": null,
  "priority": null,
  "sitemapUrl": "https://www.sitemaps.org/sitemap.xml"
}
```

## Use it from code

Actor ID: `cliqtomedia/xml-sitemap-urls-extractor`

```python
from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")
run = client.actor("cliqtomedia/xml-sitemap-urls-extractor").call(run_input={
    "startUrls": [
        {
            "url": "https://www.sitemaps.org/sitemap.xml"
        }
    ],
    "maxDepth": 5
})
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)
```

```bash
curl -X POST "https://api.apify.com/v2/acts/cliqtomedia~xml-sitemap-urls-extractor/run-sync-get-dataset-items?token=YOUR_APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"startUrls": [{"url": "https://www.sitemaps.org/sitemap.xml"}], "maxDepth": 5}'
```

```python
# pip install langchain-apify
# set APIFY_TOKEN in your environment
from langchain_apify import ApifyActorsTool

tool = ApifyActorsTool("cliqtomedia/xml-sitemap-urls-extractor")
result = tool.invoke({"run_input": {"startUrls": [{"url": "https://www.sitemaps.org/sitemap.xml"}], "maxDepth": 5}})
print(result)
```

run-sync waits up to 300 seconds; for longer runs start a run and use a webhook. LlamaIndex and MCP: https://cliqtomedia.com/actors/ai-agents/

## Use cases with this tool

- [Know what your competitors do](https://cliqtomedia.com/actors/use-cases/competitor-monitoring/): See what your competitors launch, hire, change and advertise. Every week, without opening their sites.
- [SEO data without a big suite](https://cliqtomedia.com/actors/use-cases/seo-research/): Collect keyword ideas and check your pages and your rivals' pages. Search results and trends coming soon.

## Similar tools

- [Google Autocomplete Scraper](https://cliqtomedia.com/actors/google-autocomplete-scraper/)
- [Website JSON-LD Structured Data Scraper](https://cliqtomedia.com/actors/website-jsonld-structured-data-scraper/)
- [Website Metadata & Open Graph Scraper](https://cliqtomedia.com/actors/website-metadata-open-graph-scraper/)
- [Google Ads Transparency Scraper - Ad Text, No Login](https://cliqtomedia.com/actors/google-ads-transparency-scraper/)
- [Google Images Scraper - Image URLs, Sizes & Source Pages](https://cliqtomedia.com/actors/google-images-scraper/)
- [Google News Scraper - Real Article URLs, No Login](https://cliqtomedia.com/actors/google-news-scraper/)