Substack Scraper – Newsletter Posts

sfoln/substack-scraper · News, Social media, AI On request

Scrape any Substack newsletter (or one on its own domain): every post with title, subtitle, date, authors, likes, comments, word count, paywall flag and the full text as markdown for free posts, plus the publication's profile and subscriber count. $1 per 1,000 posts, no start fee.

Request access Run it$1 / 1,000 substack post

About

What it does

Scrape any Substack newsletter — on name.substack.com or its own domain — into clean rows: every post with title, subtitle, date, authors as published, likes, comments, restacks, word count, paywall flag and, for free posts, the full text as markdown. Each newsletter also gets a free publication profile (name, author, description, start date, subscriber count). Built for newsletter research, competitor and creator tracking, content datasets and AI agents.

Price

Price per delivered record

Event	Price	Per 1,000	Charged for
Substack post `post`	$0.001	$1 / 1,000	Charged once per newsletter post delivered (metadata, engagement and, for free posts, the full text as markdown).

Apify free plan: up to 10 items per run. Paid plans have no per-run cap; set maxTotalChargeUsd to cap spend.

Run

Call it from code, an agent or the Console

This API is private on Apify for now. Email us and we will publish it for you, usually within two working days. The calls below work once it is public. contact@sfoln.com

curl · synchronous run, rows in the response

curl -X POST "https://api.apify.com/v2/actors/sfoln~substack-scraper/run-sync-get-dataset-items" \
  -H "Authorization: Bearer $APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"publications":["https://on.substack.com"],"maxPostsPerPublication":5}'

mcp.json · hosted Apify MCP server

{
  "mcpServers": {
    "apify": {
      "type": "http",
      "url": "https://mcp.apify.com?tools=sfoln/substack-scraper"
    }
  }
}

asynchronous run · for large batches

POST https://api.apify.com/v2/actors/sfoln~substack-scraper/runs?maxTotalChargeUsd=1
GET  https://api.apify.com/v2/actor-runs/{runId}            # until status is SUCCEEDED
GET  https://api.apify.com/v2/datasets/{defaultDatasetId}/items?format=json

Authenticate with an Apify API token. MCP clients can instead sign in with OAuth when prompted. Example prompt for an agent: Use substack-scraper to get the 10 newest posts of https://on.substack.com with their full text, then summarize the main announcements.

Input

Input fields

Substack publications, post URLs or writer @handles. Each one delivers its posts (one charged row per post) and one free summary record with the publication profile.

Field	Type	Default	Meaning
`publications` *	`array`	required	Publications, posts or @handles One per line: a Substack name (platformer), a URL (https://name.substack.com or the publication's own domain), a post URL (…/p/<slug>) or a writer's @handle. Accounts on the Apify free plan can process up to 10 publications per run, with up to 20 posts each.
`maxPostsPerPublication`	`integer`	`20`	Max posts per publication Posts to deliver per publication, newest first (or most popular first with sort: top). Range 1–5000.
`sort`	`string`	`"new"`	Sort new (newest first) or top (Substack's most popular posts first). One of `"new"`, `"top"`.
`searchQuery`	`string`	—	Search within the archive Only posts matching this text (Substack's own archive search).
`postedAfter`	`string`	—	Posted after Only posts on or after this date: YYYY-MM-DD, an ISO date-time, or a relative period such as "30 days".
`postedBefore`	`string`	—	Posted before Only posts on or before this date (a YYYY-MM-DD date includes the whole day).
`includeBody`	`boolean`	`true`	Include full text Add the full post as markdown (bodyMarkdown) for posts everyone can read. Paid and subscriber-only posts always come with their public excerpt only.
`includePaywalled`	`boolean`	`true`	Include paid posts Deliver paid and subscriber-only posts too (metadata and the public excerpt, no body). Off: they are left out and not charged.
`includePublicationProfile`	`boolean`	`true`	Include publication profile Add the publication's name, description, author, logo, start date and subscriber count to its free summary record (one extra request).
`maxPublications`	`integer`	`100`	Max publications Safety cap on inputs in this run; the rest are ignored and counted as inputItemsDropped in the OUTPUT record. Range 1–1000.
`concurrency`	`integer`	`2`	Concurrency Publications read in parallel. Range 1–5.
`includeRaw`	`boolean`	`false`	Include raw post objects Attach Substack's post object (without the HTML body) to each row (larger output).

Example input

{
  "publications": [
    "https://on.substack.com"
  ],
  "maxPostsPerPublication": 5
}

Output

One record per input item

Records land in the run's dataset; a synchronous call returns them directly. Each has a status (ok is charged; everything else is free) and a charged flag. The run's OUTPUT record totals items and charges.

Field	Type	Meaning
`input`	`string \| object`	Input The publication, post URL or @handle this record belongs to.
`recordType`	`string`	Record type post (charged) or summary (one per input, with the publication profile; free). e.g. `"post"`
`position`	`integer`	Position Post number within its input in this run (1 = first).
`status`	`string`	Status ok \| not_found \| error \| skipped_invalid \| skipped_budget \| skipped_free_plan \| skipped_unmetered e.g. `"ok"`
`charged`	`boolean`	Charged Whether this record incurred a pay-per-event charge.
`chargedEvent`	`string`	Charged event Pay-per-event event charged for this record (present only when charged).
`chargedCount`	`integer`	Events charged Events actually charged when the charge came back short of the requested count (present only then; a warning explains it).
`source`	`string`	Source Upstream data source.
`fetchedAt`	`string`	Fetched at ISO timestamp.
`error`	`object`	Error { code, message, provider?, status? } when status is not ok. Codes set by the runtime: budget_exhausted, free_plan_limit, unmetered, aborted, charge_failed, record_too_large, invalid_record, dataset_push_failed; upstream codes: rate_limit, timeout, network, server, client, invalid_response, not_found; this Actor: invalid_input, not_substack.
`warnings`	`array`	Warnings Notes about this record, e.g. a cap or the budget that stopped an input early.
`postId`	`integer`	Post ID Substack's post id.
`title`	`string \| null`	Title Post title.
`subtitle`	`string \| null`	Subtitle Post subtitle.
`slug`	`string \| null`	Slug URL slug (…/p/<slug>).
`url`	`string \| null`	URL Canonical post URL.
`publishedAt`	`string \| null`	Published at ISO 8601 UTC.
`updatedAt`	`string \| null`	Updated at Last edit (post URLs only).
`type`	`string \| null`	Type newsletter \| podcast \| thread \| video …
`audience`	`string \| null`	Audience everyone \| only_free \| only_paid \| founding
`isPaywalled`	`boolean`	Paywalled Not readable by everyone (subscriber-only or paid).
`authors`	`array`	Authors Bylines as published: [{ name, handle, profileUrl }].
`likes`	`integer \| null`	Likes All reactions (Substack's likes).
`reactions`	`object`	Reactions Reaction counts by emoji.
`restacks`	`integer \| null`	Restacks Restacks (shares within Substack).
`commentsCount`	`integer \| null`	Comments Comment count.
`wordcount`	`integer \| null`	Word count Words in the post.
`description`	`string \| null`	Description The post's description (social/SEO text).
`excerpt`	`string \| null`	Excerpt The public excerpt from the archive.
`bodyMarkdown`	`string \| null`	Full text (markdown) The full post as markdown: posts everyone can read, with includeBody.
`coverImage`	`string \| null`	Cover image Cover image URL.
`section`	`string \| null`	Section Publication section.
`tags`	`array`	Tags Post tags.
`language`	`string \| null`	Language Post language code.
`podcastUrl`	`string \| null`	Podcast audio Audio URL of a podcast post.
`podcastDurationSecs`	`integer \| null`	Podcast duration Seconds.
`publicationId`	`integer \| null`	Publication ID Substack's publication id.
`publicationHost`	`string`	Publication host Host the post was read from (name.substack.com or the custom domain).
`raw`	`object`	Raw Substack's post object without the HTML body (only with includeRaw).
`publication`	`string`	Publication Summary records: the publication host.
`publicationName`	`string \| null`	Publication name Summary records.
`publicationUrl`	`string`	Publication URL Summary records.
`subdomain`	`string \| null`	Subdomain Summary records: <subdomain>.substack.com.
`customDomain`	`string \| null`	Custom domain Summary records.
`publicationDescription`	`string \| null`	Publication description Summary records: the tagline.
`logoUrl`	`string \| null`	Logo Summary records.
`coverPhotoUrl`	`string \| null`	Cover photo Summary records.
`authorName`	`string \| null`	Author Summary records: the publication's main author as published.
`authorHandle`	`string \| null`	Author handle Summary records.
`createdAt`	`string \| null`	Created at Summary records: when the publication was created.
`firstPostDate`	`string \| null`	First post Summary records.
`subscriberCount`	`integer \| null`	Subscribers Summary records: subscribers when the publication shows the count, else null.
`subscriberCountText`	`string \| null`	Subscribers (as shown) Summary records: e.g. 460,000.
`paidSubscribersRange`	`string \| null`	Paid subscribers Summary records: Substack's band, e.g. Tens of thousands of paid subscribers.
`bestsellerTier`	`integer \| null`	Bestseller tier Summary records: Substack's bestseller badge tier (100, 1000, 10000).
`paymentsEnabled`	`boolean`	Paid subscriptions Summary records: the publication sells subscriptions.
`hasPodcast`	`boolean`	Podcast Summary records.
`podcastFeedUrl`	`string \| null`	Podcast feed Summary records.
`sections`	`array`	Sections Summary records.
`resultsDelivered`	`integer`	Posts delivered Summary records: charged post rows of this input.
`stopReason`	`string`	Stop reason Summary records: limit \| free_plan \| end \| budget \| aborted \| deadline \| empty_pages \| max_pages \| error.

Example record

[
  {
    "input": "synthetic",
    "recordType": "post",
    "position": 1,
    "postId": 5,
    "title": "Synthetic post 5",
    "subtitle": "Subtitle 5",
    "slug": "fifth-post",
    "url": "https://synthetic.substack.com/p/fifth-post",
    "publishedAt": "2026-10-10T10:00:00.000Z",
    "updatedAt": null,
    "type": "newsletter",
    "audience": "everyone",
    "isPaywalled": false,
    "authors": [
      {
        "name": "Sam Synthetic",
        "handle": "samsynthetic",
        "profileUrl": "https://substack.com/@samsynthetic"
      }
    ],
    "likes": 15,
    "reactions": {
      "❤": 15
    },
    "restacks": 5,
    "commentsCount": 10,
    "wordcount": 500,
    "description": "Description 5",
    "excerpt": "Excerpt of post 5…",
    "bodyMarkdown": "Hello **readers**, see [this link](https://example.com/x).[^1]\n\n![A chart](https://substackcdn.example/a.png)\n_Figure 1: growth_\n\n[^1]: A synthetic footnote.",
    "coverImage": "https://substackcdn.example/img/5.png",
    "section": "Essays",
    "tags": [
      "Testing"
    ],
    "language": "en",
    "podcastUrl": null,
    "podcastDurationSecs": null,
    "publicationId": 777,
    "publicationHost": "synthetic.substack.com",
    "status": "ok",
    "charged": true,
    "chargedEvent": "post",
    "source": "substack",
    "fetchedAt": "2026-10-14T12:00:00.000Z"
  },
  {
    "input": "synthetic",
    "publication": "synthetic.substack.com",
    "publicationName": "Synthetic Letter",
    "publicationUrl": "https://synthetic.substack.com",
    "publicationId": 777,
    "subdomain": "synthetic",
    "customDomain": null,
    "publicationDescription": "Notes on synthetic data.",
    "logoUrl": "https://substackcdn.example/logo.png",
    "coverPhotoUrl": null,
    "authorName": "Sam Synthetic",
    "authorHandle": "samsynthetic",
    "language": "en",
    "createdAt": "2021-01-02T03:04:05.000Z",
    "firstPostDate": "2021-01-03T00:00:00.000Z",
    "subscriberCount": 12500,
    "subscriberCountText": "12,500",
    "paidSubscribersRange": "Hundreds of paid subscribers",
    "bestsellerTier": 100,
    "paymentsEnabled": true,
    "hasPodcast": true,
    "podcastFeedUrl": null,
    "sections": [
      "Essays"
    ],
    "recordType": "summary",
    "resultsDelivered": 20,
    "stopReason": "limit",
    "status": "ok",
    "charged": false,
    "source": "substack",
    "fetchedAt": "2026-10-14T12:00:05.000Z"
  }
]

FAQ

Questions

Does it work for newsletters on their own domain?: Yes. Give the custom domain (for example https://www.example-newsletter.com) or the name.substack.com address; the Actor follows Substack's redirect to the custom domain.
Can it read paid posts?: No. Paid and subscriber-only posts come with their metadata and the public excerpt (isPaywalled: true, bodyMarkdown: null). Turn off includePaywalled to leave them out.
How do I get only new posts every day?: Schedule the Actor with postedAfter set to a relative period such as "1 day" and sort set to new: paging stops at the first older post.

Machine-readable

This page for agents

/actors/substack-scraper.json — this API as JSON
/actors/substack-scraper.md — this API as Markdown
/catalog.json — the whole catalogue