# Substack Scraper – Newsletter Posts (`sfoln/substack-scraper`)

Scrape any Substack newsletter (or one on its own domain): every post with title, subtitle, date, authors, likes, comments, word count, paywall flag and the full text as markdown for free posts, plus the publication's profile and subscriber count. $1 per 1,000 posts, no start fee.

- Family: Social media data
- Categories: News, Social media, AI
- Availability: on request (private on Apify). Email contact@sfoln.com with subject "Access request: Substack Scraper – Newsletter Posts (sfoln/substack-scraper)" before calling; calls fail until it is public.
- JSON: https://www.sfoln.com/actors/substack-scraper.json
- MCP: https://mcp.apify.com?tools=sfoln/substack-scraper
- REST (sync): POST https://api.apify.com/v2/actors/sfoln~substack-scraper/run-sync-get-dataset-items
- REST (async): POST https://api.apify.com/v2/actors/sfoln~substack-scraper/runs

Scrape any Substack newsletter — on name.substack.com or its own domain — into clean rows: every post with title, subtitle, date, authors as published, likes, comments, restacks, word count, paywall flag and, for free posts, the full text as markdown. Each newsletter also gets a free publication profile (name, author, description, start date, subscriber count). Built for newsletter research, competitor and creator tracking, content datasets and AI agents.

## Price per delivered record

| Event | Price | Per 1,000 | Charged for |
| --- | --- | --- | --- |
| Substack post (`post`) | $0.001 | $1 / 1,000 | Charged once per newsletter post delivered (metadata, engagement and, for free posts, the full text as markdown). |

Pay-per-event, USD, billed by Apify; no platform usage charge. Only records with status `ok` are charged. Cap a run with `maxTotalChargeUsd`. Apify free plan: up to 10 per run.

## Run

```bash
curl -X POST "https://api.apify.com/v2/actors/sfoln~substack-scraper/run-sync-get-dataset-items" \
  -H "Authorization: Bearer $APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"publications":["https://on.substack.com"],"maxPostsPerPublication":5}'
```

MCP client configuration:

```json
{
  "mcpServers": {
    "apify": {
      "type": "http",
      "url": "https://mcp.apify.com?tools=sfoln/substack-scraper"
    }
  }
}
```

Example prompt: Use substack-scraper to get the 10 newest posts of https://on.substack.com with their full text, then summarize the main announcements.

## Input

Substack publications, post URLs or writer @handles. Each one delivers its posts (one charged row per post) and one free summary record with the publication profile.

| Field | Type | Default | Meaning |
| --- | --- | --- | --- |
| `publications` (required) | array | — | Publications, posts or @handles: One per line: a Substack name (platformer), a URL (https://name.substack.com or the publication's own domain), a post URL (…/p/<slug>) or a writer's @handle. Accounts on the Apify free plan can process up to 10 publications per run, with up to 20 posts each. |
| `maxPostsPerPublication` | integer | `20` | Max posts per publication: Posts to deliver per publication, newest first (or most popular first with sort: top). |
| `sort` | string | `"new"` | Sort: new (newest first) or top (Substack's most popular posts first). One of `"new"`, `"top"`. |
| `searchQuery` | string | — | Search within the archive: Only posts matching this text (Substack's own archive search). |
| `postedAfter` | string | — | Posted after: Only posts on or after this date: YYYY-MM-DD, an ISO date-time, or a relative period such as "30 days". |
| `postedBefore` | string | — | Posted before: Only posts on or before this date (a YYYY-MM-DD date includes the whole day). |
| `includeBody` | boolean | `true` | Include full text: Add the full post as markdown (bodyMarkdown) for posts everyone can read. Paid and subscriber-only posts always come with their public excerpt only. |
| `includePaywalled` | boolean | `true` | Include paid posts: Deliver paid and subscriber-only posts too (metadata and the public excerpt, no body). Off: they are left out and not charged. |
| `includePublicationProfile` | boolean | `true` | Include publication profile: Add the publication's name, description, author, logo, start date and subscriber count to its free summary record (one extra request). |
| `maxPublications` | integer | `100` | Max publications: Safety cap on inputs in this run; the rest are ignored and counted as inputItemsDropped in the OUTPUT record. |
| `concurrency` | integer | `2` | Concurrency: Publications read in parallel. |
| `includeRaw` | boolean | `false` | Include raw post objects: Attach Substack's post object (without the HTML body) to each row (larger output). |

Example input:

```json
{
  "publications": [
    "https://on.substack.com"
  ],
  "maxPostsPerPublication": 5
}
```

## Output

One record per input item with `status` (`ok` is charged; `not_found`, `error`, `skipped_*` are free) and `charged`. The run stores an `OUTPUT` summary record.

| Field | Type | Meaning |
| --- | --- | --- |
| `input` | string | object | Input: The publication, post URL or @handle this record belongs to. |
| `recordType` | string | Record type: post (charged) or summary (one per input, with the publication profile; free). |
| `position` | integer | Position: Post number within its input in this run (1 = first). |
| `status` | string | Status: ok \| not_found \| error \| skipped_invalid \| skipped_budget \| skipped_free_plan \| skipped_unmetered |
| `charged` | boolean | Charged: Whether this record incurred a pay-per-event charge. |
| `chargedEvent` | string | Charged event: Pay-per-event event charged for this record (present only when charged). |
| `chargedCount` | integer | Events charged: Events actually charged when the charge came back short of the requested count (present only then; a warning explains it). |
| `source` | string | Source: Upstream data source. |
| `fetchedAt` | string | Fetched at: ISO timestamp. |
| `error` | object | Error: { code, message, provider?, status? } when status is not ok. Codes set by the runtime: budget_exhausted, free_plan_limit, unmetered, aborted, charge_failed, record_too_large, invalid_record, dataset_push_failed; upstream codes: rate_limit, timeout, network, server, client, invalid_response, not_found; this Actor: invalid_input, not_substack. |
| `warnings` | array | Warnings: Notes about this record, e.g. a cap or the budget that stopped an input early. |
| `postId` | integer | Post ID: Substack's post id. |
| `title` | string | null | Title: Post title. |
| `subtitle` | string | null | Subtitle: Post subtitle. |
| `slug` | string | null | Slug: URL slug (…/p/<slug>). |
| `url` | string | null | URL: Canonical post URL. |
| `publishedAt` | string | null | Published at: ISO 8601 UTC. |
| `updatedAt` | string | null | Updated at: Last edit (post URLs only). |
| `type` | string | null | Type: newsletter \| podcast \| thread \| video … |
| `audience` | string | null | Audience: everyone \| only_free \| only_paid \| founding |
| `isPaywalled` | boolean | Paywalled: Not readable by everyone (subscriber-only or paid). |
| `authors` | array | Authors: Bylines as published: [{ name, handle, profileUrl }]. |
| `likes` | integer | null | Likes: All reactions (Substack's likes). |
| `reactions` | object | Reactions: Reaction counts by emoji. |
| `restacks` | integer | null | Restacks: Restacks (shares within Substack). |
| `commentsCount` | integer | null | Comments: Comment count. |
| `wordcount` | integer | null | Word count: Words in the post. |
| `description` | string | null | Description: The post's description (social/SEO text). |
| `excerpt` | string | null | Excerpt: The public excerpt from the archive. |
| `bodyMarkdown` | string | null | Full text (markdown): The full post as markdown: posts everyone can read, with includeBody. |
| `coverImage` | string | null | Cover image: Cover image URL. |
| `section` | string | null | Section: Publication section. |
| `tags` | array | Tags: Post tags. |
| `language` | string | null | Language: Post language code. |
| `podcastUrl` | string | null | Podcast audio: Audio URL of a podcast post. |
| `podcastDurationSecs` | integer | null | Podcast duration: Seconds. |
| `publicationId` | integer | null | Publication ID: Substack's publication id. |
| `publicationHost` | string | Publication host: Host the post was read from (name.substack.com or the custom domain). |
| `raw` | object | Raw: Substack's post object without the HTML body (only with includeRaw). |
| `publication` | string | Publication: Summary records: the publication host. |
| `publicationName` | string | null | Publication name: Summary records. |
| `publicationUrl` | string | Publication URL: Summary records. |
| `subdomain` | string | null | Subdomain: Summary records: <subdomain>.substack.com. |
| `customDomain` | string | null | Custom domain: Summary records. |
| `publicationDescription` | string | null | Publication description: Summary records: the tagline. |
| `logoUrl` | string | null | Logo: Summary records. |
| `coverPhotoUrl` | string | null | Cover photo: Summary records. |
| `authorName` | string | null | Author: Summary records: the publication's main author as published. |
| `authorHandle` | string | null | Author handle: Summary records. |
| `createdAt` | string | null | Created at: Summary records: when the publication was created. |
| `firstPostDate` | string | null | First post: Summary records. |
| `subscriberCount` | integer | null | Subscribers: Summary records: subscribers when the publication shows the count, else null. |
| `subscriberCountText` | string | null | Subscribers (as shown): Summary records: e.g. 460,000. |
| `paidSubscribersRange` | string | null | Paid subscribers: Summary records: Substack's band, e.g. Tens of thousands of paid subscribers. |
| `bestsellerTier` | integer | null | Bestseller tier: Summary records: Substack's bestseller badge tier (100, 1000, 10000). |
| `paymentsEnabled` | boolean | Paid subscriptions: Summary records: the publication sells subscriptions. |
| `hasPodcast` | boolean | Podcast: Summary records. |
| `podcastFeedUrl` | string | null | Podcast feed: Summary records. |
| `sections` | array | Sections: Summary records. |
| `resultsDelivered` | integer | Posts delivered: Summary records: charged post rows of this input. |
| `stopReason` | string | Stop reason: Summary records: limit \| free_plan \| end \| budget \| aborted \| deadline \| empty_pages \| max_pages \| error. |

Example record:

```json
[
  {
    "input": "synthetic",
    "recordType": "post",
    "position": 1,
    "postId": 5,
    "title": "Synthetic post 5",
    "subtitle": "Subtitle 5",
    "slug": "fifth-post",
    "url": "https://synthetic.substack.com/p/fifth-post",
    "publishedAt": "2026-10-10T10:00:00.000Z",
    "updatedAt": null,
    "type": "newsletter",
    "audience": "everyone",
    "isPaywalled": false,
    "authors": [
      {
        "name": "Sam Synthetic",
        "handle": "samsynthetic",
        "profileUrl": "https://substack.com/@samsynthetic"
      }
    ],
    "likes": 15,
    "reactions": {
      "❤": 15
    },
    "restacks": 5,
    "commentsCount": 10,
    "wordcount": 500,
    "description": "Description 5",
    "excerpt": "Excerpt of post 5…",
    "bodyMarkdown": "Hello **readers**, see [this link](https://example.com/x).[^1]\n\n![A chart](https://substackcdn.example/a.png)\n_Figure 1: growth_\n\n[^1]: A synthetic footnote.",
    "coverImage": "https://substackcdn.example/img/5.png",
    "section": "Essays",
    "tags": [
      "Testing"
    ],
    "language": "en",
    "podcastUrl": null,
    "podcastDurationSecs": null,
    "publicationId": 777,
    "publicationHost": "synthetic.substack.com",
    "status": "ok",
    "charged": true,
    "chargedEvent": "post",
    "source": "substack",
    "fetchedAt": "2026-10-14T12:00:00.000Z"
  },
  {
    "input": "synthetic",
    "publication": "synthetic.substack.com",
    "publicationName": "Synthetic Letter",
    "publicationUrl": "https://synthetic.substack.com",
    "publicationId": 777,
    "subdomain": "synthetic",
    "customDomain": null,
    "publicationDescription": "Notes on synthetic data.",
    "logoUrl": "https://substackcdn.example/logo.png",
    "coverPhotoUrl": null,
    "authorName": "Sam Synthetic",
    "authorHandle": "samsynthetic",
    "language": "en",
    "createdAt": "2021-01-02T03:04:05.000Z",
    "firstPostDate": "2021-01-03T00:00:00.000Z",
    "subscriberCount": 12500,
    "subscriberCountText": "12,500",
    "paidSubscribersRange": "Hundreds of paid subscribers",
    "bestsellerTier": 100,
    "paymentsEnabled": true,
    "hasPodcast": true,
    "podcastFeedUrl": null,
    "sections": [
      "Essays"
    ],
    "recordType": "summary",
    "resultsDelivered": 20,
    "stopReason": "limit",
    "status": "ok",
    "charged": false,
    "source": "substack",
    "fetchedAt": "2026-10-14T12:00:05.000Z"
  }
]
```

## FAQ

**Does it work for newsletters on their own domain?**

Yes. Give the custom domain (for example https://www.example-newsletter.com) or the name.substack.com address; the Actor follows Substack's redirect to the custom domain.

**Can it read paid posts?**

No. Paid and subscriber-only posts come with their metadata and the public excerpt (isPaywalled: true, bodyMarkdown: null). Turn off includePaywalled to leave them out.

**How do I get only new posts every day?**

Schedule the Actor with postedAfter set to a relative period such as "1 day" and sort set to new: paging stops at the first older post.


## Related

- [Hacker News Scraper – Stories & Comments](https://www.sfoln.com/actors/hacker-news-scraper.md) (`sfoln/hacker-news-scraper`)
- [Telegram Channel Scraper – Public Posts](https://www.sfoln.com/actors/telegram-channel-scraper.md) (`sfoln/telegram-channel-scraper`)
- [URL to Markdown & Article Extractor](https://www.sfoln.com/actors/url-to-markdown-summary.md) (`sfoln/url-to-markdown-summary`)
- [Brand Monitoring – X, Reddit, News & Bluesky](https://www.sfoln.com/actors/brand-mentions-monitor.md) (`sfoln/brand-mentions-monitor`)
- [Google News Scraper](https://www.sfoln.com/actors/google-news-search.md) (`sfoln/google-news-search`)
- [Reddit Posts Scraper (Subreddits)](https://www.sfoln.com/actors/reddit-subreddit-posts.md) (`sfoln/reddit-subreddit-posts`)
- [Website Change Monitor – Text Diff Alerts](https://www.sfoln.com/actors/website-change-monitor.md) (`sfoln/website-change-monitor`)

Publisher: SFOLN LLC, contact@sfoln.com. Catalogue: https://www.sfoln.com/catalog.json. Agent guide: https://www.sfoln.com/agents.md.