# URL to Markdown & Article Extractor (`sfoln/url-to-markdown-summary`)

Turn web pages into clean markdown with the main article only: title, author, publish date, site name, word and token counts. Optional AI summary with key points, keywords and entities. Built for RAG, agents and content pipelines. $1.50 per 1,000 pages; summaries $4 per 1,000.

- Family: Web toolkit for agents
- Categories: AI, Agents, Developer tools
- Availability: on request (private on Apify). Email contact@sfoln.com with subject "Access request: URL to Markdown & Article Extractor (sfoln/url-to-markdown-summary)" before calling; calls fail until it is public.
- JSON: https://www.sfoln.com/actors/url-to-markdown-summary.json
- MCP: https://mcp.apify.com?tools=sfoln/url-to-markdown-summary
- REST (sync): POST https://api.apify.com/v2/actors/sfoln~url-to-markdown-summary/run-sync-get-dataset-items
- REST (async): POST https://api.apify.com/v2/actors/sfoln~url-to-markdown-summary/runs

Turn web pages into clean markdown of their main article, ready for LLMs, RAG pipelines and AI agents. For every URL you get the article as GitHub-flavoured markdown (headline, headings, paragraphs, lists, links and tables; menus, ads, share buttons, cookie banners and footers removed), its metadata (title, author, publish date, site name, canonical URL, Open Graph image), word and token counts and, optionally, a structured AI summary: a short abstract, 3–7 key points, keywords, named entities, the language and the content type. $1.50 per 1,000 pages, summaries $4 per 1,000; broken, missing or non-HTML pages are free.

## Price per delivered record

| Event | Price | Per 1,000 | Charged for |
| --- | --- | --- | --- |
| Markdown page (`url-markdown`) | $0.0015 | $1.5 / 1,000 | Charged once per URL whose page was fetched and its main content converted to readable markdown with metadata. |
| Page summary (`url-summary`), paid plans only | $0.004 | $4 / 1,000 | Charged once per URL for which the AI summary (summary, key points, keywords, entities, language, content type) was generated. |

Pay-per-event, USD, billed by Apify; no platform usage charge. Only records with status `ok` are charged. Cap a run with `maxTotalChargeUsd`. Apify free plan: up to 10 per run.

## Run

```bash
curl -X POST "https://api.apify.com/v2/actors/sfoln~url-to-markdown-summary/run-sync-get-dataset-items" \
  -H "Authorization: Bearer $APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"urls":["https://en.wikipedia.org/wiki/Web_scraping"]}'
```

MCP client configuration:

```json
{
  "mcpServers": {
    "apify": {
      "type": "http",
      "url": "https://mcp.apify.com?tools=sfoln/url-to-markdown-summary"
    }
  }
}
```

Example prompt: Convert https://en.wikipedia.org/wiki/Web_scraping to markdown and give me its key points.

## Input

List web page URLs. Each URL produces one dataset record with markdown, metadata and (optionally) a structured summary.

| Field | Type | Default | Meaning |
| --- | --- | --- | --- |
| `urls` (required) | array | — | URLs: Web page URLs (http/https), one per line. Accounts on the Apify free plan can process up to 10 URLs per run. |
| `summarize` | boolean | `true` | Summarize with AI: Add a structured summary (summary, key points, keywords, entities, language, content type) generated by Mistral AI. Charged per summarized URL (url-summary). |
| `summaryLanguage` | string | — | Summary language: Language for the summary and key points, e.g. "English" or "de". Empty = the page's own language. |
| `maxWords` | integer | `120` | Max summary words: Upper limit for the summary length in words (20–300). |
| `includeMarkdown` | boolean | `true` | Include markdown: Include the page's readable content as markdown (up to 200,000 characters). Turn off if you only need the summary. |
| `extractor` | string | `"article"` | Extractor: article: the page's main article (Mozilla Readability with a safety net, the default). full: every section of the page's main region, as before v2 (documentation and landing pages). One of `"article"`, `"full"`. |
| `maxUrls` | integer | `1000` | Max URLs: Safety cap on how many URLs are processed in this run; URLs beyond it are reported as dropped. |
| `concurrency` | integer | `3` | Concurrency: URLs processed in parallel (1–10). |

Example input:

```json
{
  "urls": [
    "https://en.wikipedia.org/wiki/Web_scraping"
  ]
}
```

## Output

One record per input item with `status` (`ok` is charged; `not_found`, `error`, `skipped_*` are free) and `charged`. The run stores an `OUTPUT` summary record.

| Field | Type | Meaning |
| --- | --- | --- |
| `input` | string | object | Input: The input item this record corresponds to. |
| `status` | string | Status: ok \| not_found \| error \| skipped_invalid \| skipped_budget \| skipped_free_plan |
| `charged` | boolean | Charged: Whether the record's main event was charged. |
| `chargedEvent` | string | null | Charged event: Event charged by the runner for this record, if any. |
| `source` | string | Source: Upstream data source. |
| `fetchedAt` | string | Fetched at: ISO timestamp. |
| `warnings` | array | null | Warnings: Non-fatal problems (pages not reachable, truncation, budget limits). |
| `error` | object | null | Error: { code, message, provider?, status? } when status is not ok. |
| `url` | string | null | URL: Normalized input URL. |
| `finalUrl` | string | null | Final URL: URL after redirects. |
| `title` | string | null | Title: Page <title> (or og:title). |
| `description` | string | null | Description: Meta description / og:description. |
| `canonical` | string | null | Canonical URL: rel=canonical or og:url. |
| `author` | string | null | Author: Author line (Readability byline, meta author or JSON-LD article author). |
| `publishedAt` | string | null | Published at: ISO 8601 (UTC) publication time from the page (article:published_time, JSON-LD datePublished or Readability), when it states a valid one. |
| `siteName` | string | null | Site name: og:site_name or the publisher name. |
| `ogImage` | string | null | Open Graph image: og:image URL. |
| `markdown` | string | null | Markdown: The page's main content as GitHub-flavoured markdown (headline, headings, paragraphs, lists, links, tables), max 200,000 characters. |
| `markdownTruncated` | boolean | null | Markdown truncated: true when the markdown was cut at 200,000 characters. |
| `wordCount` | integer | null | Word count: Words in the extracted markdown. |
| `tokenEstimate` | integer | null | Token estimate: Rough LLM token count of the markdown (about 4 characters per token). |
| `extractor` | string | null | Extractor: readability (Mozilla Readability's article) \| fallback (full main region: used when Readability would drop part of the page, for very large pages, and with extractor full) \| text (a plain-text page). |
| `summary` | string | null | Summary: AI summary (≤ maxWords words). |
| `keyPoints` | array | null | Key points: 3–7 key points. |
| `keywords` | array | null | Keywords: Up to 10 keywords. |
| `entities` | object | null | Entities: { people[], organizations[], places[] } mentioned on the page. |
| `language` | string | null | Language: ISO 639-1 code of the page language (from the summary, else the html lang attribute). |
| `contentType` | string | null | Content type: article \| product \| docs \| other (null without summary). |
| `summaryCharged` | boolean | null | Summary charged: true when url-summary was charged for this URL. |
| `summaryInputTruncated` | boolean | null | Summary input truncated: true when only the first ≈4,000 tokens of the page were summarized. |
| `tokensUsed` | object | null | Tokens used: { input, output } Mistral tokens for the summary. |

Example record:

```json
{
  "input": "https://acme.com/blog/widgets",
  "url": "https://acme.com/blog/widgets",
  "finalUrl": "https://acme.com/blog/widgets",
  "title": "Widgets explained | Acme Blog",
  "description": "All about widgets.",
  "canonical": "https://acme.com/blog/widgets",
  "author": "Jane Example",
  "publishedAt": "2026-09-30T08:00:00.000Z",
  "siteName": "Acme Blog",
  "ogImage": "https://acme.com/img/widgets.png",
  "markdown": "# Widgets explained\n\nWidgets are small mechanical parts ...\n\n- Cheap\n- Durable",
  "markdownTruncated": false,
  "wordCount": 1240,
  "tokenEstimate": 1910,
  "extractor": "readability",
  "summary": "Widgets are small mechanical parts made by Acme Corp in Berlin ...",
  "keyPoints": [
    "Widgets are cheap",
    "They are durable",
    "Acme makes them in Berlin"
  ],
  "keywords": [
    "widgets",
    "Acme",
    "manufacturing"
  ],
  "entities": {
    "people": [
      "Jane Example"
    ],
    "organizations": [
      "Acme Corp"
    ],
    "places": [
      "Berlin"
    ]
  },
  "language": "en",
  "contentType": "article",
  "summaryCharged": true,
  "summaryInputTruncated": false,
  "tokensUsed": {
    "input": 1812,
    "output": 196
  },
  "status": "ok",
  "charged": true,
  "chargedEvent": "url-markdown",
  "source": "website+mistral",
  "fetchedAt": "2026-10-14T10:00:00.000Z"
}
```

## FAQ

**How is the main content found?**

Each page is parsed with Mozilla Readability (the engine of Firefox Reader View) on a full DOM with links made absolute. Its result is used only when it keeps the opening paragraphs and at least 85% of the text of the page's main region; otherwise a second extractor that keeps every section is used (documentation pages, for example; also for very large pages, over 800 KB of HTML, 10,000 elements or 128 levels of nesting, and when extractor is full). Navigation, share buttons, tag clouds and other page chrome are removed, tables become GitHub-flavoured markdown tables, and the markdown starts with the article's headline.

**Which pages do not work?**

Pages that build their content with JavaScript, sit behind a login or a bot check return little or no text: such URLs end as no_text or thin_content errors and are not charged. PDFs and other files are not converted (not_html).


## Related

- [RAG Web Search – Google + LLM-Ready Markdown](https://www.sfoln.com/actors/rag-web-search.md) (`sfoln/rag-web-search`)
- [Website to Markdown Crawler (llms.txt)](https://www.sfoln.com/actors/website-to-markdown.md) (`sfoln/website-to-markdown`)
- [AI Web Scraper – Any URL to Structured JSON](https://www.sfoln.com/actors/ai-web-scraper.md) (`sfoln/ai-web-scraper`)
- [URL Metadata & Open Graph Extractor](https://www.sfoln.com/actors/url-metadata.md) (`sfoln/url-metadata`)
- [Sitemap URL Extractor](https://www.sfoln.com/actors/sitemap-url-extractor.md) (`sfoln/sitemap-url-extractor`)
- [RSS Feed Reader – Blogs, News & Newsletters](https://www.sfoln.com/actors/rss-feed-reader.md) (`sfoln/rss-feed-reader`)
- [Google Search Results Scraper](https://www.sfoln.com/actors/google-search-results.md) (`sfoln/google-search-results`)

Publisher: SFOLN LLC, contact@sfoln.com. Catalogue: https://www.sfoln.com/catalog.json. Agent guide: https://www.sfoln.com/agents.md.