# Google Scholar Search Scraper

> Google Scholar scraper: get title, authors, venue, year, cited-by count and PDF link for your queries. Filter by year. No login or API key.

URL: https://cliqtomedia.com/actors/google-scholar-scraper/

Category: Science & research data · Status: coming soon · Apify profile: https://apify.com/cliqtomedia?fpr=cliqto

Search Google Scholar and get one row per article. Each row has title, authors, year, snippet and cited-by count. It adds a PDF link when shown. You need no Google login.

## How to use it: 3 steps

1. **Follow us on Apify.** The tool is built and waits in the queue for the Apify Store. Apify is the cloud platform where our tools run.
2. **Paste your input and press Start.** No code needed: you fill in a form (the input). It already has a working example. For example: `web scraping` in Search queries. Each start is one run.
3. **Download the results.** When the run ends, the results are in a table (the dataset). Open Output, press Export and pick Excel or CSV. JSON and the API are there too.

## What it does

Search Google Scholar with your own queries and get one Dataset row per article: title, link, authors, venue, year, snippet, cited-by count and a direct PDF or HTML link when Scholar shows one. Add a year range, skip [CITATION] entries or sort by date. No Google login, account or API key is needed.



### Good for

- Literature review: the top articles on a topic, with authors, venue, year and citation counts in one spreadsheet.
- Finding open full texts: `pdfUrl` gives the direct [PDF] or [HTML] link from the right column of Scholar.
- Comparing topics or periods: several queries in one run, each with the same year range.
- Feeding papers to an AI agent or a RAG pipeline: titles, snippets and links as clean JSON.

### Not a good fit

- More than 100 articles per query. Scholar cuts deeper pages for automated traffic; split a big topic into narrower queries or year ranges.
- Author profile pages, "Cited by" lists, BibTeX/citation export or full abstracts. The Actor returns the snippet Scholar shows, not the abstract.
- Downloading PDFs. The Actor returns links; it does not fetch or store files.
- Pages behind a CAPTCHA. The Actor never solves CAPTCHAs. When Scholar blocks every proxy IP, the query gets an `error` row and is not charged.

## Price

Prices will be shown after release.

## Questions

### Do I need a Google account or an API key?

No. It reads public Scholar pages. You only need an Apify account.

### Why is citedBy or year empty for some rows?

Scholar does not show them for every result. Empty means not shown, not zero.

### Can I get every article on a topic?

No. It returns up to 100 articles per query. Use narrower queries or year ranges to get more.

### Does it download PDFs?

No. It gives you the links only. It does not save files.

### Are the results the same as in my browser?

Not always. Scholar answers different IPs and sessions differently: when we ran the same query several times, the order and some of the articles changed between runs. `position` is Scholar's order at the time of the run, so compare by `title` or `url`, not by position.

## Data fields

- `query` (string): Your search query.
- `position` (integer): Place of the article among the articles returned for this query, continuing across pages (page 2 normally starts at 11). Articles Scholar repeats from earlier pages are skipped, so later positions can move up. Null on not_found/error rows.
- `page` (integer): Scholar result page (1 = first page).
- `resultId` (string): Scholar's own id of the result (data-cid), stable across runs; useful for de-duplication.
- `type` (string): Kind Scholar marks before the title: article (no label), pdf, html, book or citation (known only from reference lists).
- `title` (string): Article title.
- `url` (string): Link of the title (publisher page or repository). Null for [CITATION] entries without a link.
- `authors` (array): Author names as Scholar prints them (initials and surname). Scholar may shorten long lists, see authorsTruncated.
- `authorsTruncated` (boolean): True when Scholar cut the author list with "…".
- `authorProfiles` (array): Authors with a Google Scholar profile: [{name, profileId, profileUrl}].
- `venue` (string): Journal, conference or book as Scholar prints it; may be shortened with "…". Null when Scholar shows none.
- `year` (integer): Publication year from the result line. Null when Scholar shows no year (not 0).
- `source` (string): Domain or publisher Scholar shows at the end of the result line, e.g. "nature.com".
- `snippet` (string): Text excerpt Scholar shows under the title.
- `addedAgo` (string): Only with sortByDate: how long ago Scholar added the article, as Scholar prints it (e.g. "2 days ago"); removed from the snippet. Null otherwise.
- `citedBy` (integer): Number from "Cited by N". 0 when Scholar shows the result's link bar without a "Cited by" link (no known citations); null when the link bar is missing.
- `citedByUrl` (string): Scholar page listing the citing articles.
- `versions` (integer): Number from "All N versions"; 1 when the link bar has no versions link; null when the link bar is missing.
- `versionsUrl` (string): Scholar page listing all versions.
- `relatedUrl` (string): Scholar "Related articles" search.
- `pdfUrl` (string): Direct full-text link from the right column ([PDF] or [HTML]); null when Scholar shows none.
- `fullTextType` (string): Label Scholar shows for that link: usually PDF or HTML, sometimes another file type such as PS or DOC.
- `searchUrl` (string): Scholar results page this row came from.
- `status` (string): complete: an article row. not_found: Scholar found nothing for the query and filters (no article charge; the query event applies). error: the query or a later page failed (free), see statusReason.
- `statusReason` (string): Why the row is not_found or error.
- `errorCode` (string): Machine-readable error code on error rows, e.g. SOURCE_BLOCKED, DEADLINE_REACHED.
- `fetchedAt` (string): When the page was fetched (ISO 8601, UTC).

## Input settings

- `queries` (array, required): What to search on Google Scholar, one query per line, e.g. "web scraping". Scholar operators such as author:"J Smith", source:Nature or "exact phrase" work. Up to 200 queries; duplicates are removed.
- `maxResults` (integer): How many articles to return for each query, up to 100 (Scholar shows 10 per page). You pay per article returned plus a small fee per answered query. Stops earlier when Scholar has no more results. Need more on one topic? Add narrower queries or split by year range.
- `yearFrom` (integer): Only articles published in this year or later (Scholar's "Since" / custom range filter). Leave empty for any year.
- `yearTo` (integer): Only articles published in this year or earlier. Leave empty for any year.
- `includeCitations` (boolean): Keep [CITATION] entries (works Scholar knows only from other papers' reference lists, without an online copy). Turn off to get only results with a link.
- `sortByDate` (boolean): Newest first, as Scholar's "Sort by date" does: Scholar then shows only articles added in the last year, so year filters and totals differ. Default: sort by relevance.
- `proxyConfiguration` (object): Apify datacenter proxy by default. A page Scholar refuses or shortens on datacenter IPs is retried through Apify residential proxy (US). Choose your own proxy group or URLs to turn this off. The Actor never solves CAPTCHAs.

## Step by step on Apify

1. Open the Input tab. The form is prefilled with the query `web scraping` and Max results per query = 10.
2. Click Start.
3. Open the Results view in the Dataset. Check `title`, `authors`, `year`, `citedBy` and `pdfUrl`.
4. Open the `RUN_SUMMARY` record in the Key-value store to see the run status and counts.

## Use cases

| Task | Input | You get | Key fields | Next step |
|---|---|---|---|---|
| Top papers on a topic | `queries` + `maxResults` | Article | `title`, `authors`, `year`, `citedBy` | Export CSV or Excel |
| Papers from a fixed period | `yearFrom` + `yearTo` | Article | `year`, `venue`, `url` | Screening spreadsheet |
| Open full texts | `includeCitations: false` | Article | `pdfUrl`, `fullTextType`, `source` | Download with your own tool |
| Newest papers | `sortByDate: true` | Article | `addedAgo`, `title`, `url` | Read via API |
| Papers by one author or journal | `queries` with `author:` or `source:` | Article | `authorProfiles`, `venue`, `citedBy` | Join on `resultId` |

## Tips

- Use a year range to focus on new papers.
- Use narrow queries or year ranges to get more than 100 articles.
- Use the pdfUrl field to find open full texts fast.

## Limits

This Actor is independent of Google. It reads public Google Scholar results pages, the same pages you see in a browser without login. Articles belong to their authors and publishers; check their terms before you reuse texts or PDFs.

- Up to 200 queries per run and 100 articles per query. In our tests 200 queries × 1 article took about 9 minutes with a 600-second timeout, so raise the timeout for big lists.
- Speed: a 10-article run usually takes under 10 seconds. In our tests 3 queries × 50 articles took about 30 seconds, and 5 queries × 100 articles took 2–3 minutes at 256 MB.
- Deep pages: Scholar often repeats or empties pages after about 70 articles of one query. In our tests 1–2 of 5 queries × 100 ended with a free `error` row after 72–92 articles; the other queries reached 100.
- The default run timeout is 300 seconds. The Actor stops source work 15 seconds before the timeout and keeps what it has: the run ends SUCCEEDED with status `CAPPED` and reason `DEADLINE`. For many queries, raise the timeout in the run options.
- Default memory is 256 MB; more memory does not add results.
- Results are in English Scholar (`hl=en`) with Scholar's default search scope (articles, no patents).

## Sample result

```json
{
  "query": "web scraping",
  "position": 1,
  "page": 1,
  "resultId": "vzUZDE4wNWMJ",
  "type": "pdf",
  "title": "Web scraping or web crawling: State of art, techniques, approaches and application.",
  "url": "http://i-csrs.org/Volumes/ijasca/2021.3.11.pdf",
  "authors": ["Author One"],
  "authorsTruncated": false,
  "authorProfiles": [
    { "name": "Author One", "profileId": "AUTHOR_ID_1", "profileUrl": "https://scholar.google.com/citations?user=AUTHOR_ID_1&hl=en" }
  ],
  "venue": "International Journal of Advances in Soft Computing & …",
  "year": 2021,
  "source": "i-csrs.org",
  "snippet": "… are required to implement web scrapping properly such as … into what web scraping is, how it works, web scraping stages, …",
  "addedAgo": null,
  "citedBy": 679,
  "versions": 4,
  "pdfUrl": "http://i-csrs.org/Volumes/ijasca/2021.3.11.pdf",
  "fullTextType": "PDF",
  "searchUrl": "https://scholar.google.com/scholar?q=web+scraping&hl=en&as_sdt=0%2C5",
  "status": "complete",
  "statusReason": null,
  "errorCode": null,
  "fetchedAt": "2026-10-17T12:32:46.110Z"
}
```

## Similar tools

- [DOI Metadata & Bibliography Enrichment](https://cliqtomedia.com/actors/doi-bibliography-enrichment/)
- [Google Patent Details Scraper](https://cliqtomedia.com/actors/google-patents-scraper/)
- [ClinicalTrials.gov Studies Search & Monitor](https://cliqtomedia.com/actors/clinicaltrials-gov-studies-monitor/)
- [DataCite Research Dataset Search](https://cliqtomedia.com/actors/datacite-dataset-search/)
- [Europe PMC Literature Search & Monitor](https://cliqtomedia.com/actors/europe-pmc-literature-search/)
- [GBIF Biodiversity Occurrence Search](https://cliqtomedia.com/actors/gbif-occurrence-search/)