Stack Overflow Scraper, No Login

site	stackoverflow
questionId	77823456
title	Apify: weird costs for using key-value stores

How it will work: 3 steps

Follow us on ApifyThe tool is built and waits in the queue for the Apify Store. Apify is the cloud platform where our tools run.
Paste your input and press StartNo code needed: you fill in a form (the input). It already has a working example. For example: apify in Tags. Each start is one run.
Download the resultsWhen the run ends, the results are in a table (the dataset). Open Output, press Export and pick Excel or CSV. JSON and the API are there too.

This tool is an Apify Actor: a small program that runs on Apify. You can also run it on a schedule, for example every Monday. New to it? Read how it works, step by step and the 8 words you need first.

Made by Cliqto Media, an independent digital media company. We made this tool. About us

What it does

Stack Overflow & Stack Exchange Q&A Scraper exports questions with their answers from Stack Overflow and the other Stack Exchange sites (Super User, Server Fault, Ask Ubuntu, Mathematics and 170+ more). You pick a site and a tag, a search query or a list of question links, and get one table row per question with the title, body, tags, votes, author and the answers inside the same row.

Good for

Watch new questions about your product tag on Stack Overflow.
Build a question-and-answer dataset in Markdown for search, RAG or an LLM.
Save a list of known threads with all answers and votes.
Find the most-voted questions on a topic on any Stack Exchange site.

Not a good fit

No comments, user profiles, badges or chat. Only questions and their answers.
No private or deleted content. Deleted questions come back as not_found rows.
Answer links (/a/<id>) are not looked up. Use the question link instead.
The data is live: votes, views and answers can change between runs.

Who uses it

Product teams

Watch new questions about your product tag.

AI builders

Build a Q&A dataset in Markdown for search, RAG or an LLM.

Researchers

Find the most-voted questions on a topic on any Stack Exchange site.

Price

Prices will be shown after release.

Questions

Do I need a Stack Overflow account or API key?

No. It uses the official public API. The key field is optional and only raises the quota.

Are answers charged separately?

No. Answers are inside the question row. One question row is one charge.

Which sites work?

Stack Overflow, Super User, Server Fault, Ask Ubuntu, Mathematics and many more public sites.

Does it get comments or user profiles?

No. It gets questions and their answers only.

Are the numbers live?

Yes. Each run reads the API at that moment; fetchedAt shows when. Votes, views and answers change over time.

Is personal data included?

Only public profile data shown on the site: the author's user ID, display name, reputation and profile link. No email, location or other profile details.

Details

Step by step on Apify

Open the Input tab.
Keep Site at the prefilled stackoverflow and Tags at the prefilled apify, or type your own tag.
Keep Max questions at the prefilled 10. Mode stays at the default tag.
Click Start.
Open the Questions view of the output and check title, score and answers.

Use cases

Task	Input	You get	Key fields	Next step
Follow questions about your product	`tags: ["your-tag"]`, `sort: newest`, scheduled	question rows	`title`, `link`, `createdAt`, `isAnswered`	Send unanswered ones to your support channel
Build an LLM or RAG dataset	`tags`, `sort: votes`, `bodyFormat: markdown`	rows with answers	`body`, `answers[].body`, `answers[].isAccepted`, `attribution`	Load into a vector store, keep the credit line
Research a topic	`mode: search`, `query`, `fromDate`	question rows	`score`, `viewCount`, `tags`	Count questions per month or per tag
Save known threads	`mode: question`, `questions`	one row per thread	`answers`, `acceptedAnswerId`, `lastActivityAt`	Archive or compare over time

Tips for better results

Use your own tag on a daily schedule to see new questions.
Add a free Stack Apps key to raise the daily quota.
Use question links when you need known threads with answers.

Limits and good to know

Up to 1,000 questions per run. We tested 1,000 questions with 16,522 answers in one run: COMPLETE in 150–220 seconds
Without an API key, the Stack Exchange API gives about 300 requests per day per IP address. One request reads up to 100 questions with answers, so normal runs use a few requests. RUN_SUMMARY shows the remaining quota
The API asks clients to wait between some requests. The Actor follows this, so big runs take minutes, not seconds
If the daily quota runs out, the run stops as CAPPED with reason QUOTA_LIMIT, keeps saved rows and suggests an apiKey. If nothing was saved yet, the run is ERROR
The Actor stops reading at about 270 seconds so it can save the results within the 300-second timeout (CAPPED, DEADLINE)

For developers

Sample result

One result from a real test run, in JSON. Real values change from run to run. In Excel or CSV each field is a column.

one result · JSON

{
  "site": "stackoverflow",
  "questionId": 77823456,
  "title": "Apify: weird costs for using key-value stores",
  "link": "https://stackoverflow.com/questions/77823456/apify-weird-costs-for-using-key-value-stores",
  "tags": ["node.js", "apify"],
  "score": 0,
  "upVotes": 1,
  "downVotes": 1,
  "viewCount": 213,
  "answerCount": 1,
  "isAnswered": true,
  "acceptedAnswerId": null,
  "commentCount": null,
  "bountyAmount": null,
  "isClosed": false,
  "closedReason": null,
  "closedAt": null,
  "createdAt": "2024-01-16T04:20:59.000Z",
  "lastActivityAt": "2024-01-16T13:03:25.000Z",
  "lastEditedAt": null,
  "author": {
    "userId": 2486638,
    "displayName": "Alvein",
    "reputation": 217,
    "userType": "registered",
    "profileUrl": "https://stackoverflow.com/users/2486638/alvein"
  },
  "bodyFormat": "markdown",
  "body": "I'm working with Node.js but I think my question is language-agnostic.\n\nIn my Apify actor, First I do a few long scrapes…",
  "answers": [
    {
      "answerId": 77825927,
      "link": "https://stackoverflow.com/questions/77823456/apify-weird-costs-for-using-key-value-stores/77825927#77825927",
      "score": 1,
      "upVotes": 1,
      "downVotes": 0,
      "isAccepted": false,
      "commentCount": 1,
      "createdAt": "2024-01-16T13:03:25.000Z",
      "lastActivityAt": "2024-01-16T13:03:25.000Z",
      "lastEditedAt": null,
      "author": {
        "userId": 9898465,
        "displayName": "Lukáš Křivka",
        "reputation": 983,
        "userType": "registered",
        "profileUrl": "https://stackoverflow.com/users/9898465/luk%c3%a1%c5%a1-k%c5%99ivka"
      },
      "body": "Apify doesn't charge for how big the writes are. Apify charges for number of writes and then for time based storage…",
      "license": "CC BY-SA 4.0",
      "licenseUrl": "https://creativecommons.org/licenses/by-sa/4.0/"
    }
  ],
  "answersIncluded": 1,
  "answersComplete": true,
  "license": "CC BY-SA 4.0",
  "licenseUrl": "https://creativecommons.org/licenses/by-sa/4.0/",
  "attribution": "\"Apify: weird costs for using key-value stores\" by Alvein (https://stackoverflow.com/users/2486638/alvein), https://stackoverflow.com/questions/77823456/apify-weird-costs-for-using-key-value-stores, licensed under CC BY-SA 4.0",
  "rank": 8
}

Data fields (37)

Field	Type	Meaning
`site`	string	Site as requested, e.g. stackoverflow or math.stackexchange.com.
`questionId`	integer	Question ID on that site.
`title`	string	Question title, HTML entities decoded.
`link`	string	Question URL.
`tags`	array	Question tags.
`score`	integer	Score (up minus down votes).
`upVotes`	integer	Up votes.
`downVotes`	integer	Down votes.
`viewCount`	integer	Views.
`answerCount`	integer	Total answers on the site (0 = none, null = unknown).
`isAnswered`	boolean	true when the question has an accepted or upvoted answer (Stack Exchange definition).
`acceptedAnswerId`	integer	ID of the accepted answer, null if none.
`commentCount`	integer	Comments on the question.
`bountyAmount`	integer	Open bounty in reputation, null if none.
`isClosed`	boolean	true if the question is closed.
`closedReason`	string	Reason the question was closed.
`closedAt`	string	When it was closed, ISO 8601 UTC.
`createdAt`	string	Created, ISO 8601 UTC.
`lastActivityAt`	string	Last activity, ISO 8601 UTC.
`lastEditedAt`	string	Last edit, ISO 8601 UTC; null if never edited.
`author`	object	Public author profile: userId, displayName, reputation, userType, profileUrl. null for deleted users.
`bodyFormat`	string	Format of body fields in this row.
`body`	string	Question body in bodyFormat.
`answers`	array	Answers (accepted first, then score). null when includeAnswers is off.
`answersIncluded`	integer	Answers in this row; null when answers are off.
`answersComplete`	boolean	true when every answer of the question is included.
`license`	string	Question licence from the API, e.g. CC BY-SA 4.0.
`licenseUrl`	string	Creative Commons licence URL.
`attribution`	string	Ready-made CC BY-SA attribution line: title, author, link, licence.
`rank`	integer	Position in tag/search results (1 = first); null in question mode.
`input`	string	Your question ID or URL as given (question mode).
`inputs`	array	not_found and error rows: every question ID or URL this row covers (one row per batch with the same outcome). null on question rows.
`status`	string	Row result. complete and partial rows are charged; not_found and error rows are free.
`statusReason`	string	Plain-English reason for partial, not_found or error rows.
`errorCode`	string	Machine code for error rows, e.g. INVALID_INPUT, QUOTA_EXHAUSTED.
`sourceUrl`	string	Official API request that produced the row (no key).
`fetchedAt`	string	When the row was fetched, ISO 8601 UTC.

Input settings (13)

The main fields of the input form. With the API you send them as JSON.

Setting	Name in the API	Type	Example value
Mode	`mode`	string	`"tag"`
Site	`site`	string	`"stackoverflow"`
Tags	`tags`	array	`["apify"]`
Search query	`query`	string
Question IDs or URLs	`questions`	array
Sort	`sort`	string	`"newest"`
Max questions	`maxQuestions`	integer	`10`
Include answers	`includeAnswers`	boolean	`true`
Max answers per question	`maxAnswersPerQuestion`	integer	`30`
Body format	`bodyFormat`	string	`"markdown"`
Created from	`fromDate`	string
Created until	`toDate`	string
Stack Exchange API key (optional)	`apiKey`	string

Stack Overflow & Stack Exchange Q&A Scraper

How it will work: 3 steps

What it does

Good for

Not a good fit

Who uses it

Product teams

AI builders

Researchers

Price

Questions

Details

For developers

Sample result

Similar tools

Substack Newsletter Scraper

Hacker News Stories & Comments Scraper

Bilibili Scraper: Videos, Comments & Creators, No Login

Bluesky Scraper — Posts, Profiles, Feeds & Threads, No Login

Medium Article Scraper: Tags, Authors & Publications

Pinterest Search Scraper: Pins & Boards, No Login

Start on Apify