Sitemap URL Extractor

What it does

Get every URL of a website from its XML sitemaps as clean JSON. Give a website (the sitemaps are found through robots.txt, then /sitemap.xml and /sitemap_index.xml) or a sitemap file, and you get one row per page URL with lastmod, changefreq, priority, the sitemap it came from and the image, news, video and hreflang data the sitemap carries. Sitemap indexes are followed, .gz files decompressed, broken or cut-off sitemaps read leniently. Built for SEO audits, crawl planning, content inventories, change tracking and AI agents that need the list of pages before they read them. $0.20 per 1,000 URLs; sites without a sitemap cost nothing.

Price per delivered record

Apify free plan: up to 10 items per run. Paid plans have no per-run cap; set maxTotalChargeUsd to cap spend.

Call it from code, an agent or the Console

This API is private on Apify for now. Email us and we will publish it for you, usually within two working days. The calls below work once it is public. contact@sfoln.com

curl -X POST "https://api.apify.com/v2/actors/sfoln~sitemap-url-extractor/run-sync-get-dataset-items" \
  -H "Authorization: Bearer $APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"siteUrls":["https://apify.com"],"maxUrlsPerSite":10}'

{
  "mcpServers": {
    "apify": {
      "type": "http",
      "url": "https://mcp.apify.com?tools=sfoln/sitemap-url-extractor"
    }
  }
}

POST https://api.apify.com/v2/actors/sfoln~sitemap-url-extractor/runs?maxTotalChargeUsd=1
GET  https://api.apify.com/v2/actor-runs/{runId}            # until status is SUCCEEDED
GET  https://api.apify.com/v2/datasets/{defaultDatasetId}/items?format=json

Authenticate with an Apify API token. MCP clients can instead sign in with OAuth when prompted. Example prompt for an agent: List the first 50 blog URLs from the sitemap of apify.com that changed in the last 30 days.

Input fields

Website or sitemap URLs. Every site gives one dataset row per page URL plus one summary row.

{
  "siteUrls": [
    "https://apify.com"
  ],
  "maxUrlsPerSite": 10
}

One record per input item

Records land in the run's dataset; a synchronous call returns them directly. Each has a status (ok is charged; everything else is free) and a charged flag. The run's OUTPUT record totals items and charges.

[
  {
    "input": "https://www.theguardian.com",
    "recordType": "sitemap-url",
    "position": 1,
    "siteUrl": "https://www.theguardian.com/",
    "loc": "https://www.theguardian.com/science/2026/oct/14/example-article",
    "lastmod": "2026-10-14T21:54:00.000Z",
    "changefreq": "hourly",
    "priority": null,
    "sitemap": "https://www.theguardian.com/sitemaps/news.xml",
    "images": [
      {
        "loc": "https://i.guim.co.uk/img/media/example/master/5494.jpg",
        "title": null,
        "caption": null
      }
    ],
    "news": {
      "publicationName": "The Guardian",
      "language": "en",
      "publicationDate": "2026-10-14T21:51:35.000Z",
      "title": "Example science article",
      "keywords": "Science, Space"
    },
    "videos": [],
    "alternates": [],
    "status": "ok",
    "charged": true,
    "chargedEvent": "sitemap-url",
    "source": "sitemap",
    "fetchedAt": "2026-10-14T10:00:00.000Z"
  },
  {
    "input": "https://www.theguardian.com",
    "inputUrl": "https://www.theguardian.com/",
    "discoveredVia": "robots",
    "sitemapsRead": 2,
    "sitemapsFailed": 0,
    "sitemapsSkipped": 0,
    "duplicateUrls": 3,
    "excludedByPattern": 0,
    "olderThanLastmodSince": 0,
    "recordType": "summary",
    "resultsDelivered": 497,
    "stopReason": "end",
    "status": "ok",
    "charged": false,
    "source": "sitemap",
    "fetchedAt": "2026-10-14T10:00:02.000Z"
  }
]

Questions

How are the sitemaps found?

For a website URL the Actor reads its robots.txt and uses every Sitemap: line; when there is none it tries /sitemap.xml and /sitemap_index.xml. Sitemap indexes are followed up to maxDepth levels and maxSitemaps files, gzip-compressed sitemaps (.xml.gz) are decompressed, and plain-text sitemaps (one URL per line) and RSS/Atom feeds used as sitemaps are read too. You can also pass a sitemap file URL directly. The summary row says how the sitemaps were found (discoveredVia) and how many files were read.

Can I get only some of the URLs?

Yes: includePatterns and excludePatterns keep or drop URLs by glob ("*/blog/*"), substring or regular expression, and lastmodSince keeps only URLs changed after a date or within a period such as "30 days". Filtered URLs are never charged, and with lastmodSince the Actor also skips child sitemaps its index dates earlier, so large sites are read faster.

What about very large sites?

Each sitemap file is read incrementally (up to 50,000 URLs and 50 MB per file, the limits of the sitemap protocol; less on 256 MB runs), so memory stays flat. Raise maxUrlsPerSite and maxSitemaps for full exports; the run stops cleanly at your maximum cost per run.

Related APIs

• rss-feed-reader RSS Feed Reader – Blogs, News & Newsletters$0.50 / 1,000
• website-to-markdown Website to Markdown Crawler (llms.txt)$1.5 / 1,000
• url-to-markdown-summary URL to Markdown & Article Extractor$1.5 / 1,000
• rag-web-search RAG Web Search – Google + LLM-Ready Markdown$1.5 / 1,000
• url-metadata URL Metadata & Open Graph Extractor$1 / 1,000
• onpage-seo-audit SEO Audit API – On-Page Checks for Any URL$4 / 1,000

This page for agents

• /actors/sitemap-url-extractor.json — this API as JSON
• /actors/sitemap-url-extractor.md — this API as Markdown
• /catalog.json — the whole catalogue