Docs
Open Lodol
DocumentationIntegrationsAPI

ScrapeGraphAI

App · AI Agents · 15 actions

Turn any web page into something a workflow can use. Read one page as text, links, images or a picture of it, pull named facts out of it in plain language, search the web and get the results already read, crawl a whole site, or watch a page and be told what changed.

Actions

ActionWhat it does
Change a watched pageChange how often a page is checked, what is captured, or whether it is running at all.
Check a crawlSee how far a crawl has got and which pages it has reached.
Check your credit balanceSee how many credits are left, which plan the account is on, and how many crawls and watched pages it has room for.
Delete a crawlPermanently delete a crawl and every page it stored.
Extract data from a pageSay what you want from a page in plain language and get it back as data a later step can use, such as every product name and price, or a company's address and phone number.
Get a page's recent checksRead the recent checks on a watched page, newest first, each saying whether the page changed and what changed.
Get crawled pagesCollect the pages a crawl read, each with the content it was asked for.
List watched pagesList every page this account is watching, with how often each one is checked and whether it is running.
Resume a crawlPut a stopped crawl back to work from the links it had left to follow.
Scrape a pageRead one web page and bring back its text, its HTML, its links, its images, a summary, a picture of it, or its branding.
Search the webSearch the web and get the top results back with each page already read, so there is no second step to fetch them.
Start a crawlWalk a whole site from one address, following its links and reading every page.
Stop a crawlStop a crawl that is still running.
Stop watching a pageStop watching a page and delete every check it recorded.
Watch a page for changesHave ScrapeGraphAI check a page on a schedule and record what changed each time.

Change a watched page

Change how often a page is checked, what is captured, or whether it is running at all. Pausing keeps everything it has already recorded.

Inputs

Watched pagewatch_idtextRequired
The watch's id, as given back by 'Watch a page for changes' or 'List watched pages'.
Running or pausedstatustext
Pause to stop the checks without losing the page's history. Leave empty to keep it as it is.One of: active (Running), paused.
How often to checkfrequencytext
Leave empty to keep the current schedule. Changing only the time or the day works too.One of: every_10_minutes (Every 10 minutes), every_30_minutes (Every 30 minutes), hourly (Every hour), every_6_hours (Every 6 hours), daily (Every day), weekly (Every week), monthly (Every month).
At what timetime_of_daytime
Leave empty to keep the current time. Ignored when the page is checked more than once a day. Times are UTC.
On which dayday_of_weektext
Leave empty to keep the current day. Used only when the page is checked every week.One of: monday, tuesday, wednesday, thursday, friday, saturday, sunday.
What to bring backformatslist
Leave empty to keep capturing what it captures now.One or more of: markdown (The text, as markdown), html (The HTML), links (Every link on the page), images (Every image on the page), summary (A short summary), screenshot (A picture of the page (2 credits)), branding (Brand colours, fonts and logos (25 credits)).
How much of the page to keeppage_modetext
Applies to the text and the HTML, and only takes effect when you also change what to bring back.One of: normal (Everything on the page (recommended)), reader (Just the main article), prune (The page without its menus and footers).

Outputs

created_attext
When it was set up.
nametext
What this watch is called.
scheduletext
When it checks, in words, such as 'Every day at 09:00 UTC'.
statustext
'active' while it is checking, 'paused' while it is not.
updated_attext
When it last changed.
urltext
The page being watched.
watch_idtext
The id to use on the other watch actions.
Example output
{
  "created_at": "2026-04-23T11:11:37.487Z",
  "name": "Pricing page",
  "schedule": "Every day at 09:00 UTC",
  "status": "active",
  "updated_at": "2026-04-23T11:11:37.487Z",
  "url": "https://example.com/pricing",
  "watch_id": "f0b171db-b288-4380-9cad-1bcac916348e"
}

Check a crawl

See how far a crawl has got and which pages it has reached. Check 'is_finished' to know whether to wait again. Use 'Get crawled pages' for the content.

Inputs

Crawlcrawl_idtextRequired
The crawl's id, as given back by 'Start a crawl'.

Outputs

crawl_idtext
The crawl.
is_finishedtrue/false
True once the crawl has stopped, however it ended.
pageslist
Each page reached, with its address and title.
pages_finishednumber
How many it has read.
pages_foundnumber
How many pages the crawl knows about.
statustext
Where the crawl has got to.
Example output
{
  "crawl_id": "79694e03-f2ea-43f2-93cc-7c6fc26f999a",
  "is_finished": true,
  "pages": [
    {
      "depth": 1,
      "status": "completed",
      "title": "Hello",
      "url": "https://example.com/blog/hello"
    }
  ],
  "pages_finished": 3,
  "pages_found": 3,
  "status": "completed"
}

Check your credit balance

See how many credits are left, which plan the account is on, and how many crawls and watched pages it has room for. Costs nothing to run.

Inputs

None.

Outputs

crawl_limitnumber
How many crawls the plan allows at once.
crawls_runningnumber
How many crawls are running right now.
credits_remainingnumber
Credits left to spend.
credits_usednumber
Credits spent this billing period.
plantext
The plan this account is on.
watch_limitnumber
How many watched pages the plan allows.
watches_activenumber
How many pages are being watched.
Example output
{
  "crawl_limit": 50,
  "crawls_running": 0,
  "credits_remaining": 750000,
  "credits_used": 287,
  "plan": "Pro Plan",
  "watch_limit": 100,
  "watches_active": 1
}

Delete a crawl

Permanently delete a crawl and every page it stored. Collect anything you need with 'Get crawled pages' first, because this cannot be undone.

Inputs

Crawlcrawl_idtextRequired
The crawl's id, as given back by 'Start a crawl'.

Outputs

crawl_idtext
The crawl.
deletedtrue/false
True once the crawl has been deleted.
Example output
{
  "crawl_id": "79694e03-f2ea-43f2-93cc-7c6fc26f999a",
  "deleted": true
}

Extract data from a page

Say what you want from a page in plain language and get it back as data a later step can use, such as every product name and price, or a company's address and phone number.

Inputs

Page address or contentsourcetextRequired
A page address to read, such as https://example.com/pricing. You can also paste the page's text or HTML from an earlier step to work on that instead of fetching it again.
What to pull outprompttextRequired
Describe the facts you want, such as 'every job listing with its title, location and salary range'.
Result shaperesult_shapeobject
Optional. A JSON schema pinning the exact fields and types to come back, so every run returns the same shape and a later step can rely on it. Leave empty and the fields follow what you asked for.
How much of the page to keeppage_modetext
Applies to the text and the HTML. Keeping just the article gives a much cleaner result on a news story or a blog post.One of: normal (Everything on the page (recommended)), reader (Just the main article), prune (The page without its menus and footers). Default: normal.
How to load the pageload_modetext
Automatic suits almost every page. Choose 'Run the page's scripts' for a page that builds itself after loading and comes back empty otherwise.One of: auto (Automatic (recommended)), fast (Quickly, without running scripts), js (Run the page's scripts). Default: auto.
Get past bot protectionget_past_blockstrue/false
Load the page through a residential connection so a site that blocks automated visits lets it through. Try this when a page comes back empty or refused. Costs 5 extra credits.Default: false.
Load the page fromload_from_countrytext
Visit the page as if from this country, for a site that shows different prices or content by region. Leave empty to let ScrapeGraphAI choose.One of: ar (Argentina), au (Australia), at (Austria), be (Belgium), br (Brazil), ca (Canada), cl (Chile), cn (China), co (Colombia), cz (Czechia), and 37 more.
Scroll down this many timesscroll_countnumber
For a page that loads more as you scroll, such as an endless feed. Leave empty not to scroll.From 0 to 100.
Wait after the page loadswait_secondsnumber
Seconds to wait before reading, for a page that fills itself in a moment after it loads. Leave empty not to wait.From 0 to 30.

Outputs

completion_tokensnumber
How much was written back.
dataobject
What was pulled out of the page.
prompt_tokensnumber
How much of the page was read.
rawtext
The unparsed answer, when there is one.
request_idtext
ScrapeGraphAI's id for this extraction.
Example output
{
  "completion_tokens": 92,
  "data": {
    "title": "Example Domain"
  },
  "prompt_tokens": 361,
  "request_id": "8c34fc03-17be-4fcc-a7ce-6ebcab23ad43"
}

Get a page's recent checks

Read the recent checks on a watched page, newest first, each saying whether the page changed and what changed. Run this on a schedule to act on a change.

Inputs

Watched pagewatch_idtextRequired
The watch's id, as given back by 'Watch a page for changes' or 'List watched pages'.
Start fromstart_attext
Leave empty for the most recent checks. To go further back, use the 'next_start_at' from the previous run.

Outputs

changed_countnumber
How many of them found the page had changed.
check_countnumber
How many checks came back.
checkslist
One entry per check, newest first.
has_moretrue/false
True when there are older checks to read.
next_start_attext
Where the next batch begins, when there is one.
watch_idtext
The watched page.
Example output
{
  "changed_count": 1,
  "check_count": 1,
  "checks": [
    {
      "changed": true,
      "changes": {},
      "check_id": "fb7ada6e-97d2-4e66-9fee-ebf598a5d16a",
      "checked_at": "2026-04-23T11:11:37.619Z",
      "status": "completed",
      "took_seconds": 1.4
    }
  ],
  "has_more": false,
  "next_start_at": null,
  "watch_id": "f0b171db-b288-4380-9cad-1bcac916348e"
}

Get crawled pages

Collect the pages a crawl read, each with the content it was asked for. Comes back in batches: if 'has_more' is true, run this again with 'next_start_at'.

Inputs

Crawlcrawl_idtextRequired
The crawl's id, as given back by 'Start a crawl'.
How many pages to collectpage_countnumber
How many to return in this batch.From 1 to 100. Default: 50.
Start fromstart_atnumber
Leave empty to start at the first page. To continue, use the 'next_start_at' from the previous run.From 0 to 1000. Default: 0.

Outputs

crawl_idtext
The crawl.
has_moretrue/false
True when there are more pages to collect.
next_start_atnumber
Where the next batch begins, when there is one.
page_countnumber
How many pages are in this batch.
pageslist
One entry per page, with its content.
Example output
{
  "crawl_id": "79694e03-f2ea-43f2-93cc-7c6fc26f999a",
  "has_more": false,
  "next_start_at": null,
  "page_count": 1,
  "pages": [
    {
      "depth": 1,
      "links": [
        "https://example.com/blog"
      ],
      "markdown": "# Hello\n\nOur first post.",
      "parent_url": "https://example.com/blog",
      "status": "completed",
      "title": "Hello",
      "url": "https://example.com/blog/hello"
    }
  ]
}

List watched pages

List every page this account is watching, with how often each one is checked and whether it is running.

Inputs

None.

Outputs

watch_countnumber
How many pages are being watched.
watcheslist
One entry per watched page.
Example output
{
  "watch_count": 1,
  "watches": [
    {
      "created_at": "2026-04-23T11:11:37.487Z",
      "name": "Pricing page",
      "schedule": "Every day at 09:00 UTC",
      "status": "active",
      "updated_at": "2026-04-23T11:11:37.487Z",
      "url": "https://example.com/pricing",
      "watch_id": "f0b171db-b288-4380-9cad-1bcac916348e"
    }
  ]
}

Resume a crawl

Put a stopped crawl back to work from the links it had left to follow.

Inputs

Crawlcrawl_idtextRequired
The crawl's id, as given back by 'Start a crawl'.

Outputs

crawl_idtext
The crawl.
resumedtrue/false
True once the crawl is running again.
Example output
{
  "crawl_id": "79694e03-f2ea-43f2-93cc-7c6fc26f999a",
  "resumed": true
}

Scrape a page

Read one web page and bring back its text, its HTML, its links, its images, a summary, a picture of it, or its branding. Handles pages that need a browser to load.

Inputs

Page addressurltextRequired
The page to read, such as https://example.com/pricing.
What to bring backformatslist
Tick everything you want from the page. The text as markdown suits almost every workflow. To pull out named facts instead, use 'Extract data from a page'.One or more of: markdown (The text, as markdown), html (The HTML), links (Every link on the page), images (Every image on the page), summary (A short summary), screenshot (A picture of the page (2 credits)), branding (Brand colours, fonts and logos (25 credits)). Default: ["markdown"].
How much of the page to keeppage_modetext
Applies to the text and the HTML. Keeping just the article gives a much cleaner result on a news story or a blog post.One of: normal (Everything on the page (recommended)), reader (Just the main article), prune (The page without its menus and footers). Default: normal.
How to load the pageload_modetext
Automatic suits almost every page. Choose 'Run the page's scripts' for a page that builds itself after loading and comes back empty otherwise.One of: auto (Automatic (recommended)), fast (Quickly, without running scripts), js (Run the page's scripts). Default: auto.
Get past bot protectionget_past_blockstrue/false
Load the page through a residential connection so a site that blocks automated visits lets it through. Try this when a page comes back empty or refused. Costs 5 extra credits.Default: false.
Load the page fromload_from_countrytext
Visit the page as if from this country, for a site that shows different prices or content by region. Leave empty to let ScrapeGraphAI choose.One of: ar (Argentina), au (Australia), at (Austria), be (Belgium), br (Brazil), ca (Canada), cl (Chile), cn (China), co (Colombia), cz (Czechia), and 37 more.
Scroll down this many timesscroll_countnumber
For a page that loads more as you scroll, such as an endless feed. Leave empty not to scroll.From 0 to 100.
Wait after the page loadswait_secondsnumber
Seconds to wait before reading, for a page that fills itself in a moment after it loads. Leave empty not to wait.From 0 to 30.

Outputs

brandingobject
The site's brand colours, fonts and logos.
content_typetext
What kind of file the page turned out to be.
htmltext
The page's HTML.
imageslist
Every image address on the page.
linkslist
Every link on the page.
markdowntext
The page's text, as markdown.
request_idtext
ScrapeGraphAI's id for this read.
screenshot_urltext
A link to the picture of the page. It expires after an hour.
summarytext
A short summary of the page.
urltext
The page that was read.
Example output
{
  "content_type": "text/html",
  "markdown": "# Example Domain\n\nThis domain is for use in documentation examples.",
  "request_id": "7bc67b9e-e539-4d7f-b378-ceb4d86910bc",
  "url": "https://example.com"
}

Search the web

Search the web and get the top results back with each page already read, so there is no second step to fetch them. Add a question and it answers across all of them at once.

Inputs

What to search forquerytextRequired
What you want, in plain language, such as 'best project management tools for small agencies'.
How many resultsresult_countnumber
How many pages to find and read. Each one costs credits, so start small.From 1 to 20. Default: 3.
What to pull out of the resultsprompttext
Optional. Describe what you want gathered across all the pages, such as 'each tool's name, price and free trial length'. Leave empty to get the pages only.
Result shaperesult_shapeobject
Optional. A JSON schema pinning the exact fields and types to come back, so every run returns the same shape and a later step can rely on it. Leave empty and the fields follow what you asked for.
How to bring back each pagepage_formattext
Markdown reads well and is much smaller than HTML.One of: markdown (As markdown (recommended)), html (As HTML). Default: markdown.
Published withinpublished_withintext
Only use results from this recently. Leave empty for any time.One of: past_hour (The past hour), past_24_hours (The past 24 hours), past_week (The past week), past_month (The past month), past_year (The past year).
Search as if fromresults_for_countrytext
Get the results someone in this country would see. Leave empty to search without a location.One of: ar (Argentina), au (Australia), at (Austria), be (Belgium), br (Brazil), ca (Canada), cl (Chile), cn (China), co (Colombia), cz (Czechia), and 37 more.
How to load the pageload_modetext
Automatic suits almost every page. Choose 'Run the page's scripts' for a page that builds itself after loading and comes back empty otherwise.One of: auto (Automatic (recommended)), fast (Quickly, without running scripts), js (Run the page's scripts). Default: auto.
Get past bot protectionget_past_blockstrue/false
Load the page through a residential connection so a site that blocks automated visits lets it through. Try this when a page comes back empty or refused. Costs 5 extra credits.Default: false.
Wait after the page loadswait_secondsnumber
Seconds to wait before reading, for a page that fills itself in a moment after it loads. Leave empty not to wait.From 0 to 30.

Outputs

completion_tokensnumber
How much was written back.
dataobject
What was pulled out across all the pages.
pages_readnumber
How many of them could be read.
pages_requestednumber
How many pages were asked for.
prompt_tokensnumber
How much of the pages was read.
request_idtext
ScrapeGraphAI's id for this search.
result_countnumber
How many pages came back.
resultslist
The pages that were found, best first, each with its content.
Example output
{
  "pages_read": 3,
  "pages_requested": 3,
  "request_id": "27783ace-462a-45d2-9ff1-cdf3a22adf87",
  "result_count": 1,
  "results": [
    {
      "content": "## Simple, transparent pricing ...",
      "title": "Pricing - ScrapeGraphAI",
      "url": "https://scrapegraphai.com/pricing"
    }
  ]
}

Start a crawl

Walk a whole site from one address, following its links and reading every page. The pages are not ready when this step finishes: follow it with a Wait step and 'Check a crawl'.

Inputs

Where to starturltextRequired
The first page to read, such as https://example.com/blog. Its links are followed from there.
What to bring backformatslist
Tick everything you want from the page. The text as markdown suits almost every workflow. To pull out named facts instead, use 'Extract data from a page'.One or more of: markdown (The text, as markdown), html (The HTML), links (Every link on the page), images (Every image on the page), summary (A short summary), screenshot (A picture of the page (2 credits)), branding (Brand colours, fonts and logos (25 credits)). Default: ["markdown"].
How much of the page to keeppage_modetext
Applies to the text and the HTML. Keeping just the article gives a much cleaner result on a news story or a blog post.One of: normal (Everything on the page (recommended)), reader (Just the main article), prune (The page without its menus and footers). Default: normal.
Most pages to readmax_pagesnumber
The crawl stops here. Every page costs credits, so keep it as low as the job allows.From 1 to 1000. Default: 50.
How many links deep to gomax_depthnumber
1 reads only the starting page, 2 also reads the pages it links to, and so on.From 1 to 10. Default: 2.
Most links to follow from each pagemax_links_per_pagenumber
Stops one link-heavy page from filling the whole crawl. Leave empty for ScrapeGraphAI's 10.From 1 to 100.
Follow links to other websitesfollow_external_linkstrue/false
Off by default, so the crawl stays on the site it started from.Default: false.
Only read addresses matchinginclude_patternslist
Address patterns to stay inside, one per entry, such as /blog/*. Leave empty to read everything the crawl reaches.
Skip addresses matchingexclude_patternslist
Address patterns to leave out, one per entry, such as /admin/*.
How to load the pageload_modetext
Automatic suits almost every page. Choose 'Run the page's scripts' for a page that builds itself after loading and comes back empty otherwise.One of: auto (Automatic (recommended)), fast (Quickly, without running scripts), js (Run the page's scripts). Default: auto.
Get past bot protectionget_past_blockstrue/false
Load the page through a residential connection so a site that blocks automated visits lets it through. Try this when a page comes back empty or refused. Costs 5 extra credits.Default: false.
Load the page fromload_from_countrytext
Visit the page as if from this country, for a site that shows different prices or content by region. Leave empty to let ScrapeGraphAI choose.One of: ar (Argentina), au (Australia), at (Austria), be (Belgium), br (Brazil), ca (Canada), cl (Chile), cn (China), co (Colombia), cz (Czechia), and 37 more.
Scroll down this many timesscroll_countnumber
For a page that loads more as you scroll, such as an endless feed. Leave empty not to scroll.From 0 to 100.
Wait after the page loadswait_secondsnumber
Seconds to wait before reading, for a page that fills itself in a moment after it loads. Leave empty not to wait.From 0 to 30.

Outputs

crawl_idtext
The id to use on the other crawl actions.
is_finishedtrue/false
True once the crawl has stopped, however it ended.
pages_finishednumber
How many it has read so far.
pages_foundnumber
How many pages the crawl knows about so far.
statustext
Where the crawl has got to.
urltext
Where the crawl started.
Example output
{
  "crawl_id": "79694e03-f2ea-43f2-93cc-7c6fc26f999a",
  "is_finished": false,
  "pages_finished": 0,
  "pages_found": 3,
  "status": "running",
  "url": "https://example.com/blog"
}

Stop a crawl

Stop a crawl that is still running. The pages it already read stay available, and 'Resume a crawl' picks it back up.

Inputs

Crawlcrawl_idtextRequired
The crawl's id, as given back by 'Start a crawl'.

Outputs

crawl_idtext
The crawl.
stoppedtrue/false
True once the crawl has been stopped.
Example output
{
  "crawl_id": "79694e03-f2ea-43f2-93cc-7c6fc26f999a",
  "stopped": true
}

Stop watching a page

Stop watching a page and delete every check it recorded. To stop the checks but keep the history, use 'Change a watched page' and pause it instead.

Inputs

Watched pagewatch_idtextRequired
The watch's id, as given back by 'Watch a page for changes' or 'List watched pages'.

Outputs

deletedtrue/false
True once the watch has been deleted.
watch_idtext
The watched page.
Example output
{
  "deleted": true,
  "watch_id": "f0b171db-b288-4380-9cad-1bcac916348e"
}

Watch a page for changes

Have ScrapeGraphAI check a page on a schedule and record what changed each time. Set this up once, then read the changes with 'Get a page's recent checks'.

Inputs

Page addressurltextRequired
The page to keep an eye on, such as a competitor's pricing page.
What to call itnametextRequired
A name you will recognise later, such as 'Acme pricing'.
How often to checkfrequencytext
Checking more often uses credits faster.One of: every_10_minutes (Every 10 minutes), every_30_minutes (Every 30 minutes), hourly (Every hour), every_6_hours (Every 6 hours), daily (Every day), weekly (Every week), monthly (Every month). Default: daily.
At what timetime_of_daytime
Used by the daily, weekly and monthly choices, and ignored by the rest. Times are UTC.Default: 09:00.
On which dayday_of_weektext
Used by the weekly choice, and ignored by the rest.One of: monday, tuesday, wednesday, thursday, friday, saturday, sunday. Default: monday.
What to bring backformatslist
Tick everything you want from the page. The text as markdown suits almost every workflow. To pull out named facts instead, use 'Extract data from a page'.One or more of: markdown (The text, as markdown), html (The HTML), links (Every link on the page), images (Every image on the page), summary (A short summary), screenshot (A picture of the page (2 credits)), branding (Brand colours, fonts and logos (25 credits)). Default: ["markdown"].
How much of the page to keeppage_modetext
Applies to the text and the HTML. Keeping just the article gives a much cleaner result on a news story or a blog post.One of: normal (Everything on the page (recommended)), reader (Just the main article), prune (The page without its menus and footers). Default: normal.
How to load the pageload_modetext
Automatic suits almost every page. Choose 'Run the page's scripts' for a page that builds itself after loading and comes back empty otherwise.One of: auto (Automatic (recommended)), fast (Quickly, without running scripts), js (Run the page's scripts). Default: auto.
Get past bot protectionget_past_blockstrue/false
Load the page through a residential connection so a site that blocks automated visits lets it through. Try this when a page comes back empty or refused. Costs 5 extra credits.Default: false.
Load the page fromload_from_countrytext
Visit the page as if from this country, for a site that shows different prices or content by region. Leave empty to let ScrapeGraphAI choose.One of: ar (Argentina), au (Australia), at (Austria), be (Belgium), br (Brazil), ca (Canada), cl (Chile), cn (China), co (Colombia), cz (Czechia), and 37 more.
Scroll down this many timesscroll_countnumber
For a page that loads more as you scroll, such as an endless feed. Leave empty not to scroll.From 0 to 100.
Wait after the page loadswait_secondsnumber
Seconds to wait before reading, for a page that fills itself in a moment after it loads. Leave empty not to wait.From 0 to 30.

Outputs

created_attext
When it was set up.
nametext
What this watch is called.
scheduletext
When it checks, in words, such as 'Every day at 09:00 UTC'.
statustext
'active' while it is checking, 'paused' while it is not.
updated_attext
When it last changed.
urltext
The page being watched.
watch_idtext
The id to use on the other watch actions.
Example output
{
  "created_at": "2026-04-23T11:11:37.487Z",
  "name": "Pricing page",
  "schedule": "Every day at 09:00 UTC",
  "status": "active",
  "updated_at": "2026-04-23T11:11:37.487Z",
  "url": "https://example.com/pricing",
  "watch_id": "f0b171db-b288-4380-9cad-1bcac916348e"
}

ScanMalware

Previous Page

Scrapeless

Next Page

On this page

ActionsChange a watched pageCheck a crawlCheck your credit balanceDelete a crawlExtract data from a pageGet a page's recent checksGet crawled pagesList watched pagesResume a crawlScrape a pageSearch the webStart a crawlStop a crawlStop watching a pageWatch a page for changes