Hacker News Scraper – Stories & Comments

sfoln/hacker-news-scraper · News, Social media, Developer tools On request

Search Hacker News by keyword, read the top, new, best, Ask HN, Show HN, job and Who is hiring lists, or fetch items by ID with their comment threads. One clean row per story or comment: title, URL, text, author, points, comment count, date. $0.50 per 1,000 rows, no start fee.

Request access Run it$0.50 / 1,000 hacker news item

About

What it does

Get Hacker News stories and comments as clean JSON: search everything ever posted (keywords, type, author, dates, points), read the ranked top, new, best, Ask HN, Show HN and job lists, pull the job posts of the latest "Who is hiring?" thread, or fetch any item by ID with its full comment thread. One row per story or comment with title, link, text, author, points, comment count and date — ready for trend tracking, launch monitoring, recruiting, research datasets and AI agents.

Price

Price per delivered record

Event	Price	Per 1,000	Charged for
Hacker News item `item`	$0.0005	$0.50 / 1,000	Charged once per delivered row: a story, comment, job, poll or Who is hiring post.

Apify free plan: up to 10 items per run. Paid plans have no per-run cap; set maxTotalChargeUsd to cap spend.

Run

Call it from code, an agent or the Console

This API is private on Apify for now. Email us and we will publish it for you, usually within two working days. The calls below work once it is public. contact@sfoln.com

curl · synchronous run, rows in the response

curl -X POST "https://api.apify.com/v2/actors/sfoln~hacker-news-scraper/run-sync-get-dataset-items" \
  -H "Authorization: Bearer $APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"searchQueries":["web scraping"],"lists":["top"],"maxResultsPerInput":10}'

mcp.json · hosted Apify MCP server

{
  "mcpServers": {
    "apify": {
      "type": "http",
      "url": "https://mcp.apify.com?tools=sfoln/hacker-news-scraper"
    }
  }
}

asynchronous run · for large batches

POST https://api.apify.com/v2/actors/sfoln~hacker-news-scraper/runs?maxTotalChargeUsd=1
GET  https://api.apify.com/v2/actor-runs/{runId}            # until status is SUCCEEDED
GET  https://api.apify.com/v2/datasets/{defaultDatasetId}/items?format=json

Authenticate with an Apify API token. MCP clients can instead sign in with OAuth when prompted. Example prompt for an agent: Use hacker-news-scraper to get today's top 10 Hacker News stories with their top comments, then summarize what people are discussing.

Input

Input fields

Give search queries, story lists and/or item IDs. Every search, list and item is one input; each delivers one row per story or comment plus a free summary record. Accounts on the Apify free plan can process up to 10 inputs per run, with up to 20 rows each.

Field	Type	Default	Meaning
`searchQueries`	`array`	—	Search queries Full-text searches over all of Hacker News (Algolia HN Search), one per line. Use * to match everything and select with the filters below only. Accounts on the Apify free plan can process up to 10 inputs per run.
`lists`	`array`	—	Story lists Ranked lists from the official Hacker News API: top (front page), new, best, ask (Ask HN), show (Show HN), job, or whoishiring (the job posts of the latest monthly "Ask HN: Who is hiring?" thread).
`itemIds`	`array`	—	Item IDs or URLs Hacker News item IDs or URLs (https://news.ycombinator.com/item?id=8863), one per line: stories, comments, jobs or polls. Turn on Include comments to get the comment thread too.
`searchTags`	`array`	—	Search: item types Search only these types (any of them). Empty = all types.
`sortBy`	`string`	`"relevance"`	Search: sort by relevance (Algolia ranking; at most 1,000 results per query) or date (newest first; pages as deep as you ask). One of `"relevance"`, `"date"`.
`dateFrom`	`string`	—	Search: from date Only items created on or after this date: YYYY-MM-DD, an ISO date-time, or a relative period such as "7 days".
`dateTo`	`string`	—	Search: to date Only items created on or before this date (a YYYY-MM-DD date includes the whole day), an ISO date-time or a relative period.
`minPoints`	`integer`	—	Search: minimum points Only stories with at least this many points (comments have no public points, so this filters them out). Range 0–….
`minComments`	`integer`	—	Search: minimum comments Only stories with at least this many comments. Range 0–….
`author`	`string`	—	Search: author Only items posted by this Hacker News username.
`maxResultsPerInput`	`integer`	`30`	Max results per search or list Stories (or hits) per search query and per list. A relevance search stops at 1,000; the top and new lists hold 500 stories, the others up to 200. Range 1–5000.
`includeComments`	`boolean`	`false`	Include comments For story lists and item IDs, also deliver the comment thread of each story (one row per comment, in thread order). Each comment is a charged row.
`maxCommentsPerStory`	`integer`	`100`	Max comments per story Comments per story when Include comments is on. The most visible comments come first: all top-level comments in Hacker News order, then their replies. Range 1–5000.
`maxCommentDepth`	`integer`	`10`	Max comment depth Reply levels to follow (1 = only direct replies to the story). Range 1–100.
`maxInputs`	`integer`	`100`	Max inputs Safety cap on searches + lists + items in this run; inputs beyond it are ignored and counted as inputItemsDropped in the OUTPUT record. Range 1–1000.
`concurrency`	`integer`	`3`	Concurrency Inputs processed in parallel. Range 1–10.
`includeRaw`	`boolean`	`false`	Include raw upstream data Attach the unmodified Firebase item or Algolia hit to each row (larger output).

Example input

{
  "searchQueries": [
    "web scraping"
  ],
  "lists": [
    "top"
  ],
  "maxResultsPerInput": 10
}

Output

One record per input item

Records land in the run's dataset; a synchronous call returns them directly. Each has a status (ok is charged; everything else is free) and a charged flag. The run's OUTPUT record totals items and charges.

Field	Type	Meaning
`input`	`string \| object`	Input The search, list or item this record belongs to, e.g. {"type": "search", "query": "rust"}.
`recordType`	`string`	Record type item (a story, comment, job or poll; charged) or summary (one per input; free). e.g. `"item"`
`position`	`integer`	Position Row number within its input (1 = first).
`status`	`string`	Status ok \| not_found \| error \| skipped_invalid \| skipped_budget \| skipped_free_plan \| skipped_unmetered e.g. `"ok"`
`charged`	`boolean`	Charged Whether this record incurred a pay-per-event charge.
`chargedEvent`	`string`	Charged event Pay-per-event event charged for this record (present only when charged).
`chargedCount`	`integer`	Events charged Events actually charged when the charge came back short of the requested count (present only then; a warning explains it).
`source`	`string`	Source Upstream data source.
`fetchedAt`	`string`	Fetched at ISO timestamp.
`error`	`object`	Error { code, message, provider?, status? } when status is not ok. Codes set by the runtime: budget_exhausted, free_plan_limit, unmetered, aborted, charge_failed, record_too_large, invalid_record, dataset_push_failed; upstream codes: rate_limit, timeout, network, server, client, invalid_response, not_found; this Actor: invalid_input, deleted.
`warnings`	`array`	Warnings Notes about this record, e.g. a cap or the budget that stopped an input early.
`id`	`integer`	Item ID Hacker News item id.
`type`	`string`	Type story \| comment \| job \| poll \| pollopt
`title`	`string \| null`	Title Story, job or poll title (null for comments).
`url`	`string \| null`	URL External link of a link story or job.
`domain`	`string \| null`	Domain Host name of the link, without www.
`text`	`string \| null`	Text Plain text of a comment, Ask/Show HN post or job description (paragraphs as blank lines, links written out).
`author`	`string \| null`	Author Hacker News username.
`points`	`integer \| null`	Points Story points (Hacker News does not publish comment scores).
`commentsCount`	`integer \| null`	Comments All comments of a story.
`replyCount`	`integer \| null`	Direct replies Direct replies to the item, when known.
`createdAt`	`string \| null`	Created at ISO 8601 UTC.
`parentId`	`integer \| null`	Parent ID The story or comment a comment replies to.
`storyId`	`integer \| null`	Story ID The story the item belongs to (a story's own id).
`storyTitle`	`string \| null`	Story title Title of that story, when known.
`category`	`string \| null`	Category ask_hn \| show_hn \| launch_hn \| tell_hn, from the title.
`depth`	`integer \| null`	Depth Comment threads: reply level (1 = reply to the story or requested item).
`rank`	`integer \| null`	Rank Story lists: position in the list (1 = first).
`dead`	`boolean`	Dead Hacker News killed (flagged) the item; only an item requested by id is delivered dead.
`hnUrl`	`string`	Hacker News URL https://news.ycombinator.com/item?id=…
`raw`	`object`	Raw The unmodified Firebase item or Algolia hit (only with includeRaw).
`resultsDelivered`	`integer`	Rows delivered Summary records: charged rows this input delivered.
`stopReason`	`string`	Stop reason Summary records: limit \| free_plan \| end \| budget \| aborted \| deadline \| empty_pages \| max_pages \| error.
`query`	`string`	Query Summary of a search: the query.
`sortBy`	`string`	Sort Summary of a search: relevance or date.
`totalHits`	`integer \| null`	Total hits Summary of a search: matches Algolia reports (approximate for large numbers).
`list`	`string`	List Summary of a list: top, new, best, ask, show, job or whoishiring.
`listSize`	`integer`	List size Summary of a list: stories in the list (job posts in the Who is hiring thread).
`threadId`	`integer`	Thread ID Summary of whoishiring: the thread's item id.
`threadTitle`	`string \| null`	Thread title Summary of whoishiring: e.g. Ask HN: Who is hiring? (October 2026).
`itemType`	`string`	Item type Summary of an item: its type.

Example record

[
  {
    "input": {
      "type": "list",
      "list": "top"
    },
    "recordType": "item",
    "position": 1,
    "id": 1001,
    "type": "story",
    "title": "Show HN: A synthetic test story",
    "url": "https://www.example.com/post?id=1",
    "domain": "example.com",
    "text": null,
    "author": "synthetic_poster",
    "points": 321,
    "commentsCount": 5,
    "replyCount": 3,
    "createdAt": "2026-10-13T14:00:00.000Z",
    "parentId": null,
    "storyId": 1001,
    "storyTitle": "Show HN: A synthetic test story",
    "category": "show_hn",
    "depth": null,
    "rank": 1,
    "dead": false,
    "hnUrl": "https://news.ycombinator.com/item?id=1001",
    "status": "ok",
    "charged": true,
    "chargedEvent": "item",
    "source": "hackernews",
    "fetchedAt": "2026-10-14T12:00:00.000Z"
  },
  {
    "input": {
      "type": "list",
      "list": "top"
    },
    "recordType": "item",
    "position": 2,
    "id": 1003,
    "type": "comment",
    "title": null,
    "url": null,
    "domain": null,
    "text": "First top comment, I'd say.\n\nSee https://example.org/a-very-long-path/page",
    "author": "user_alpha",
    "points": null,
    "commentsCount": null,
    "replyCount": 2,
    "createdAt": "2026-10-13T14:01:40.000Z",
    "parentId": 1001,
    "storyId": 1001,
    "storyTitle": "Show HN: A synthetic test story",
    "category": null,
    "depth": 1,
    "rank": null,
    "dead": false,
    "hnUrl": "https://news.ycombinator.com/item?id=1003",
    "status": "ok",
    "charged": true,
    "chargedEvent": "item",
    "source": "hackernews",
    "fetchedAt": "2026-10-14T12:00:00.000Z"
  },
  {
    "input": {
      "type": "list",
      "list": "top"
    },
    "list": "top",
    "listSize": 500,
    "recordType": "summary",
    "resultsDelivered": 2,
    "stopReason": "limit",
    "status": "ok",
    "charged": false,
    "source": "hackernews",
    "fetchedAt": "2026-10-14T12:00:01.000Z"
  }
]

FAQ

Questions

Which Hacker News data sources does it use?: The official Hacker News API (Firebase) for items, ranked lists and comment threads, and Algolia's HN Search API for full-text search. Both are public; no login or API key is needed.
How are comments ordered?: In thread order: each comment is followed by its replies, siblings in Hacker News' own ranking order. When maxCommentsPerStory cuts a thread, the most visible comments are kept: all top-level comments first, then their replies.
Can I get more than 1,000 search results?: Yes, with sortBy set to date: newest-first searches page by date as deep as maxResultsPerInput allows (up to 5,000 per query). Relevance searches stop at 1,000 results, an Algolia limit.

Machine-readable

This page for agents

/actors/hacker-news-scraper.json — this API as JSON
/actors/hacker-news-scraper.md — this API as Markdown
/catalog.json — the whole catalogue