# Hacker News Stories and Comments Scraper

> Hacker News scraper: export Top, New, Best, Ask HN, Show HN or Jobs with comment trees and parent IDs. Plain text and HTML. No API key needed.

URL: https://cliqtomedia.com/actors/hacker-news-stories-comments-scraper/

Category: Social media & communities · Status: live on Apify Store · Apify page: https://apify.com/cliqtomedia/hacker-news-stories-comments-scraper?fpr=cliqto

Export Hacker News stories and comments as rows. Each comment keeps its parent ID, so you can rebuild the thread. No API key is needed.

## How to use it: 3 steps

1. **Make a free Apify account.** Apify is the cloud platform where our tools run. The free plan gives you $5 of usage every month, no credit card needed. You pay Apify, not us.
2. **Paste your input and press Start.** No code needed: you fill in a form (the input). It already has a working example. For example: `[]` in Exact item IDs. Each start is one run.
3. **Download the results.** When the run ends, the results are in a table (the dataset). Open Output, press Export and pick Excel or CSV. JSON and the API are there too.

## What it does

Export Hacker News stories and comments as rows with stable IDs and parent links. Choose Top, New, Best, Ask HN, Show HN or Jobs, or add exact item IDs. No Hacker News API key is needed.





## Price

Pay per event. Apify bills you, not us.

| What you pay for | Price | When |
|---|---|---|
| Start fee | $0.05 | Charged once per run. This tool uses 1 GB of memory or less by default; more memory costs more. |
| Useful HN entity saved | $0.00027 | One live story, job or comment saved to the Dataset. Deleted, dead, unavailable and error rows have no entity fee. |

- One story or comment: about $0.05.
- 1,000 stories and comments: about $0.32.
- Every day 100 stories and comments, for 30 days: about $2.31 a month.

Free to try: the Apify free plan gives you $5 of usage every month, no credit card needed.

Prices checked on 24 October 2026. Current price: https://apify.com/cliqtomedia/hacker-news-stories-comments-scraper?fpr=cliqto

## Questions

### Does it get every comment of a big thread?

Not always. There are caps per root and per run. The tree status tells you if a tree is complete.

### Does it open the linked article?

No. You get the link from Hacker News, not the article text.

### Which lists can I use?

Top, New, Best, Ask HN, Show HN and Jobs.

### Do I need an API key?

No HN API key, cookie or login is needed. You need your Apify account to run the Actor. API clients use an Apify token in an authorization header, never in this Actor's input.

### Does it return every comment?

Only when every reachable child is observed within the caps and `treeStatus` is complete. The new form intentionally uses small limits. Check the summary for each root. `descendants` is the source count, not a completeness check.

### Why are HTML and plain text different?

`textHtml` keeps the source. `text` removes tags, adds block boundaries and decodes entities once. `<` becomes `<`, not an HTML tag. Deleted items can have neither value.

### What do deleted, dead and null mean?

Deleted and dead are source flags. A null response means the item was unavailable at fetch time. It does not prove why. These observations are kept; they are not silently dropped and do not have an entity fee.

### Why is there a start fee?

The scan still makes requests and saves a summary when there are no useful entities. The start fee covers this bounded work. Each fresh run has its own start fee.

### Is it a replacement for another HN scraper?

Use a field mapping before switching. This Actor uses numeric IDs, `url` for the external link and `hnUrl` for HN. Comments are flat rows with stable IDs, not nested message arrays.

## Data fields

- `entityType` (string)
- `id` (integer)
- `type` (string)
- `status` (string)
- `reason` (string)
- `rootId` (integer)
- `storyId` (integer)
- `parentId` (integer)
- `depth` (integer)
- `title` (string)
- `hnUrl` (string)
- `url` (string)
- `author` (string)
- `time` (integer)
- `createdAt` (string)
- `score` (integer)
- `descendants` (integer)
- `kids` (array)
- `textHtml` (string)
- `text` (string)
- `deleted` (boolean)
- `dead` (boolean)
- `retrievedAt` (string)
- `treeStatus` (string)
- `treeReasons` (array)
- `collectedComments` (integer)

## Input settings

- `listMode` (string): Choose an official HN list. Non-empty itemIds take precedence. No source API key is needed.
- `itemIds` (array): Optional positive integer HN IDs, at most 100. Non-empty array replaces the list; [] uses listMode. Duplicates are fetched once.
- `maxStories` (integer): Read the first N distinct IDs in list or input order. Maximum 100. Start with 3.
- `includeComments` (boolean): Return bounded replies as separate rows. Source descendants is not the number collected. Turn off for stories only.
- `maxCommentsPerStory` (integer): Limit all descendant observations per root, including deleted, unavailable and error nodes. Zero means zero comments, never unlimited.
- `maxDepth` (integer): Root is depth 0, direct reply is 1. Zero fetches roots only. A limit marks the tree partial when children remain.
- `maxNodes` (integer): Total distinct roots and descendants reserved for fetch. At most 1000. Other request, time and byte caps may stop sooner.

## Step by step on Apify

1. Open Input in Apify Console.
2. Keep the new form as it is: Top stories, 3 roots, up to 5 comments per root and depth 2.
3. Keep memory at 256 MB and timeout at 300 seconds. Set a maximum cost of $0.15 for this small example.
4. Click Start.
5. Open Dataset. Look for a `story` row and `comment` rows linked by `rootId` and `parentId`.
6. Open Key-value store → RUN_SUMMARY. A current busy story can be `partial` because the example uses small caps.

## Use cases

| Goal | Input choice | Fields to use |
|---|---|---|
| Track current tech discussions | `top`, `new` or `best` | `id`, `title`, `score`, `descendants`, `retrievedAt` |
| Study project feedback | `show` or exact `itemIds` | `rootId`, `parentId`, `depth`, `text`, `treeStatus` |
| Build a public discussion corpus | Exact IDs, bounded comments | `id`, `textHtml`, `text`, `deleted`, `dead`, summary |
| Export HN jobs | `job`, comments off | `title`, `url`, `author`, `createdAt` |

## Tips

- Add exact item IDs to export one known discussion.
- Check the tree status to see if a comment tree is complete.
- Use the plain text field for AI tools and the HTML field for display.

## Limits



- At most 100 roots, 200 descendant observations per root, depth 10 and 1,000 total reserved nodes. A live Cloud export reached 1,000 observed nodes within the configured limits; this is a cap, not a guaranteed amount.
- Only the six official lists are supported. Lists can be smaller than `maxStories` and can include jobs. Their order can change while the run is active.
- Source work has a 90-second deadline, at most 1,300 HTTP attempts, concurrency 4, an 8-second request timeout, a 2 MiB response cap and an 8 MiB total response cap. Temporary transport, 429 and 5xx errors are retried up to three attempts within these caps.
- Set platform timeout to 300 seconds. A longer finite timeout also works but does not increase the 90-second source deadline. A smaller timeout reduces the source window to leave time for saved output.
- `comment_cap`, `depth_cap`, `node_cap`, `request_cap`, `byte_cap`, `response_byte_cap`, `deadline` and `charge_cap` show a limit. `unavailable_child`, `missing_parent`, `parent_mismatch`, `unexpected_child_type` and `duplicate_or_cycle` show an observed source/graph issue.
- `treeStatus` describes what was fetched. If the charge cap stops saving rows, use run-level `status=partial` to see that the export is incomplete.
- Original HTML is data. Do not render `textHtml`, titles or external URLs as trusted HTML in your own app. Use escaped text for display. The Dataset table displays plain text.
- Scores and comments are live. The API is not an atomic snapshot. Dead/deleted/null states may change after a run. A hard platform kill can prevent final KVS writes; rows already saved remain in the Dataset.

## Sample result

```json
[
  {
    "entityType": "story",
    "id": 8863,
    "type": "story",
    "rootId": 8863,
    "storyId": 8863,
    "parentId": null,
    "depth": 0,
    "title": "My YC app: Dropbox - Throw away your USB drive",
    "hnUrl": "https://news.ycombinator.com/item?id=8863",
    "url": "http://www.getdropbox.com/u/2/screencast.html",
    "author": "example_user",
    "time": 1175714200,
    "score": 104,
    "descendants": 71,
    "status": "complete",
    "treeStatus": "complete",
    "collectedComments": 72
  },
  {
    "entityType": "comment",
    "id": 8917,
    "rootId": 8863,
    "storyId": 8863,
    "parentId": 8863,
    "depth": 1,
    "status": "complete"
  }
]
```

## Use it from code

Actor ID: `cliqtomedia/hacker-news-stories-comments-scraper`

```python
from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")
run = client.actor("cliqtomedia/hacker-news-stories-comments-scraper").call(run_input={
    "listMode": "top",
    "itemIds": []
})
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)
```

```bash
curl -X POST "https://api.apify.com/v2/acts/cliqtomedia~hacker-news-stories-comments-scraper/run-sync-get-dataset-items?token=YOUR_APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"listMode": "top", "itemIds": []}'
```

```python
# pip install langchain-apify
# set APIFY_TOKEN in your environment
from langchain_apify import ApifyActorsTool

tool = ApifyActorsTool("cliqtomedia/hacker-news-stories-comments-scraper")
result = tool.invoke({"run_input": {"listMode": "top", "itemIds": []}})
print(result)
```

run-sync waits up to 300 seconds; for longer runs start a run and use a webhook. LlamaIndex and MCP: https://cliqtomedia.com/actors/ai-agents/

## Use cases with this tool

- [Hear what people say about you](https://cliqtomedia.com/actors/use-cases/social-listening/): Find what people say about your brand and your topic on Bluesky, Hacker News, YouTube and more.

## Similar tools

- [RSS & Atom Feed Reader / Monitor](https://cliqtomedia.com/actors/rss-atom-feed-reader-monitor/)
- [Link Route Audit — Bulk HTTP Status & Redirects](https://cliqtomedia.com/actors/bulk-url-status-redirect-checker/)
- [Substack Newsletter Scraper](https://cliqtomedia.com/actors/substack-newsletter-scraper/)
- [Bilibili Scraper: Videos, Comments & Creators, No Login](https://cliqtomedia.com/actors/bilibili-scraper/)
- [Bluesky Scraper — Posts, Profiles, Feeds & Threads, No Login](https://cliqtomedia.com/actors/bluesky-scraper/)
- [Medium Article Scraper: Tags, Authors & Publications](https://cliqtomedia.com/actors/medium-articles-scraper/)