Bright Data
App · AI Agents · 7 actions
Get structured data out of a website without writing a scraper. Bright Data maintains ready-made scrapers for Amazon, LinkedIn, Instagram, TikTok, YouTube, Google Maps and thousands of other sites: point one at some pages and get back proper fields - price, stock, rating, job history - either in the step or from a job that finishes later.
Actions
| Action | What it does |
|---|---|
| Check a collection job | See whether a collection job has finished and whether it worked. |
| Collect data from a site | Point one of Bright Data's site scrapers at a handful of pages and get structured records back in this step - a product with its price, stock and rating; a profile with its job history. |
| Get a scraper's fields | Show what one site's scraper needs and what it gives back: the inputs it takes, which of them are required, an example you can paste straight into a collection step, and every field on a collected record. |
| Get collection job results | Download the records a collection job produced. |
| List collection jobs | List the collection jobs run for one site, newest first, with each one's status and how many records it holds. |
| List site scrapers | List the sites Bright Data already knows how to collect structured data from - thousands of them, covering Amazon, LinkedIn, Instagram, TikTok, YouTube, Google Maps and more. |
| Start a collection job | Set a site scraper running over a large batch without waiting for it. |
Check a collection job
See whether a collection job has finished and whether it worked. Answers with 'finished' and 'succeeded' as well as Bright Data's own status word, so a workflow can branch on them directly - wait and check again while it is running, download the results once it is ready, and report the reason if it failed.
Inputs
- Job
job_idtextRequired - The collection job to look at. This is the id 'Start a collection job' returned - or the one 'Collect data from a site' hands back when the collection ran long enough that it stopped waiting.
Outputs
dataset_idtext- The scraper it ran
error_messagetext- Why it failed, when it did
finishedtrue/false- True once the job has stopped, whether or not it worked
job_idtext- The job this describes
record_countnumber- How many records it has collected
statustext- starting, running, ready, failed or canceled
succeededtrue/false- True only when the job finished with results to download
Example output
{
"dataset_id": "gd_l7q7dkf244hwjntr0",
"error_message": null,
"finished": true,
"job_id": "s_m4x7enmven8djfqak",
"record_count": 2,
"status": "ready",
"succeeded": true
}Collect data from a site
Point one of Bright Data's site scrapers at a handful of pages and get structured records back in this step - a product with its price, stock and rating; a profile with its job history. Reach for this when the workflow needs the data now and there are tens of pages, not thousands. Bright Data allows a minute; a run that takes longer keeps going and hands back a job to pick up with 'Check a collection job'.
Inputs
- Site
dataset_idtextRequired - Which of Bright Data's scrapers to use - one per site, such as 'Amazon - Products' or 'LinkedIn - People profiles'. Pick one from the list, or paste a scraper id if it is newer than this list.
- Pages to collect
urlslist - The page addresses to collect from, one per entry - product pages, profiles, listings. Each must start with http:// or https://. This is what almost every scraper takes; up to 1000 at a time.
- Other inputs
inputslist - Use this instead of 'Pages to collect' for a scraper that takes something other than a page address - a search keyword, a category. Each entry is a set of fields, such as {"keyword": "running shoes"}. Run 'Get a scraper's fields' to see what a scraper accepts.
- How to collect
collection_methodtext - How this scraper should run. Leave it empty to collect the pages you name above, which is what most workflows want. The other choices discover pages first - from a keyword, a category or a best-sellers list - and then collect what they find; each one takes its own inputs, which 'Get a scraper's fields' will show you.
- Only return these fields
fieldslist - Names of the fields to keep on each record, one per entry. Leave empty to get everything the scraper collects, which is often a hundred fields or more. 'Get a scraper's fields' lists the names.
- Most records per input
limit_per_inputnumber - Caps how many records each entry above can produce. Matters for a scraper pointed at a listing or a search, where one input can produce thousands. Leave empty for no cap.From 1 to 100000.
- Include pages that failed
include_errorstrue/false - Turn this on to get a row for every input the scraper could not collect, alongside the ones it could. Useful when the run has to account for every address it was given.
Outputs
dataset_idtext- The scraper that ran
finishedtrue/false- True when the records are in this step's result. False when the collection ran long and became a job instead
input_countnumber- How many inputs this step sent
job_idtext- The job to check when 'finished' is false, and empty otherwise
record_countnumber- How many records came back
recordslist- The collected records
Example output
{
"dataset_id": "gd_l7q7dkf244hwjntr0",
"finished": true,
"input_count": 2,
"job_id": null,
"record_count": 2,
"records": [
{
"availability": "In Stock",
"brand": "Aurora",
"currency": "USD",
"final_price": 449.0,
"rating": 4.6,
"reviews_count": 1284,
"title": "Aurora Bar 9 Espresso Machine",
"url": "https://www.example-shop.com/dp/B0EXAMPLE1"
},
{
"availability": "In Stock",
"brand": "Aurora",
"currency": "USD",
"final_price": 199.0,
"rating": 4.4,
"reviews_count": 803,
"title": "Aurora Grind Pro Burr Grinder",
"url": "https://www.example-shop.com/dp/B0EXAMPLE2"
}
]
}Get a scraper's fields
Show what one site's scraper needs and what it gives back: the inputs it takes, which of them are required, an example you can paste straight into a collection step, and every field on a collected record. A scraper can often be run more than one way - Amazon's takes a product address, or a search keyword - and this describes one of them and lists the rest.
Inputs
- Site
dataset_idtextRequired - Which of Bright Data's scrapers to use - one per site, such as 'Amazon - Products' or 'LinkedIn - People profiles'. Pick one from the list, or paste a scraper id if it is newer than this list.
- How to collect
collection_methodtext - Which way of running this scraper to describe. Leave it empty for the usual one - collecting the pages you name.
Outputs
collection_methodtext- Which way of running it this describes
collection_methodslist- Every way this scraper can be run
configuration_urltext- This scraper's page in the Bright Data control panel
dataset_idtext- The scraper this describes
domaintext- The site it collects from
example_inputobject- One ready-made input, straight from Bright Data. Paste it into 'Other inputs' on a collection step to try the scraper out
field_countnumber- How many fields that is
fieldslist- Every field a collected record can have
inputslist- What this way of running it accepts as input
nametext- What the scraper collects
required_inputslist- The inputs it will not run without
Example output
{
"collection_method": "collect_by_url",
"collection_methods": [
"collect_by_url",
"discover_by_keyword",
"discover_by_category_url"
],
"configuration_url": "https://brightdata.com/cp/scrapers/gd_l7q7dkf244hwjntr0/pdp/configuration",
"dataset_id": "gd_l7q7dkf244hwjntr0",
"domain": "amazon.com",
"example_input": {
"url": "https://www.amazon.com/dp/B0CRMZHDG8",
"zipcode": "94107"
},
"field_count": 3,
"fields": [
{
"description": "Product title",
"name": "title",
"type": "text"
},
{
"description": "Product brand",
"name": "brand",
"type": "text"
},
{
"description": "The price shown on the page, after any discount",
"name": "final_price",
"type": "price"
}
],
"inputs": [
{
"description": "The product page to collect",
"name": "url",
"required": true,
"type": "url"
},
{
"description": "Postcode to price and stock the product for",
"name": "zipcode",
"required": false,
"type": "text"
}
],
"name": "Amazon products",
"required_inputs": [
"url"
]
}Get collection job results
Download the records a collection job produced. Asking before it has finished is not an error: the step comes back with 'ready' set to false and no records, so a workflow can check again on a schedule without a failed run in between.
Inputs
- Job
job_idtextRequired - The collection job to look at. This is the id 'Start a collection job' returned - or the one 'Collect data from a site' hands back when the collection ran long enough that it stopped waiting.
Outputs
job_idtext- The job these records came from
readytrue/false- True when the job had finished and the records are here
record_countnumber- How many records came back
recordslist- The collected records
statustext- Where the job is, when it was not ready yet
Example output
{
"job_id": "s_m4x7enmven8djfqak",
"ready": true,
"record_count": 2,
"records": [
{
"availability": "In Stock",
"brand": "Aurora",
"currency": "USD",
"final_price": 449.0,
"rating": 4.6,
"reviews_count": 1284,
"title": "Aurora Bar 9 Espresso Machine",
"url": "https://www.example-shop.com/dp/B0EXAMPLE1"
},
{
"availability": "In Stock",
"brand": "Aurora",
"currency": "USD",
"final_price": 199.0,
"rating": 4.4,
"reviews_count": 803,
"title": "Aurora Grind Pro Burr Grinder",
"url": "https://www.example-shop.com/dp/B0EXAMPLE2"
}
],
"status": "ready"
}List collection jobs
List the collection jobs run for one site, newest first, with each one's status and how many records it holds. Use it to pick up a job an earlier run started, to find yesterday's results, or to check how a scheduled collection has been doing. Narrow it by status or by the days it ran.
Inputs
- Site
dataset_idtextRequired - Which of Bright Data's scrapers to use - one per site, such as 'Amazon - Products' or 'LinkedIn - People profiles'. Pick one from the list, or paste a scraper id if it is newer than this list.
- Only jobs that are
statustext - Keep only the jobs in this state. Leave empty for all of them.One of:
starting,running,ready,failed,canceled. - Started on or after
from_datedate - Only jobs started on this day or later. Leave empty for no earliest day.
- Started on or before
to_datedate - Only jobs started on this day or earlier. Leave empty for no latest day.
- How many to return
limitnumber - The most jobs to return, newest first. Defaults to 50.From 1 to 5000. Default:
50.
Outputs
dataset_idtext- The scraper these jobs ran
job_countnumber- How many matched
jobslist- The matching jobs
Example output
{
"dataset_id": "gd_l7q7dkf244hwjntr0",
"job_count": 2,
"jobs": [
{
"created": "2026-09-14T08:12:44.201Z",
"dataset_id": "gd_l7q7dkf244hwjntr0",
"finished": true,
"job_id": "s_m4x7enmven8djfqak",
"record_count": 2,
"started_by": "API",
"status": "ready"
},
{
"created": "2026-09-15T06:03:10.888Z",
"dataset_id": "gd_l7q7dkf244hwjntr0",
"finished": false,
"job_id": "s_m4x7f1q2wc93hdl0p",
"record_count": 0,
"started_by": "API",
"status": "running"
}
]
}List site scrapers
List the sites Bright Data already knows how to collect structured data from - thousands of them, covering Amazon, LinkedIn, Instagram, TikTok, YouTube, Google Maps and more. Filter by name to find one. You do not need this to build a workflow, because every collection step offers the same list as a dropdown; it is here for a workflow that has to look a site up while it runs.
Inputs
- Filter by name
searchtext - Keep only the scrapers whose name, site or id contains this - 'amazon', 'linkedin.com', 'profile'. Leave empty for all of them.
Outputs
scraper_countnumber- How many matched
scraperslist- The matching scrapers, in name order
searchtext- The filter that was applied, if any
Example output
{
"scraper_count": 2,
"scrapers": [
{
"collection_methods": [
"collect_by_url",
"discover_by_keyword",
"discover_by_category_url"
],
"dataset_id": "gd_l7q7dkf244hwjntr0",
"description": "eCommerce product data for amazon.com and its categories",
"domain": "amazon.com",
"name": "Amazon products",
"record_count": 300000000
},
{
"collection_methods": [
"collect_by_url",
"discover_by_keyword"
],
"dataset_id": "gd_l1viktl72bvl7bjuj0",
"description": "Public profile data: roles, employers and education",
"domain": "linkedin.com",
"name": "LinkedIn people profiles",
"record_count": 690000000
}
],
"search": null
}Start a collection job
Set a site scraper running over a large batch without waiting for it. Returns a job id straight away; the collection keeps going on Bright Data's side, and 'Check a collection job' and 'Get collection job results' pick it up - in this workflow or in a scheduled one that runs later. Use this for hundreds or thousands of pages, where waiting in a step is not an option.
Inputs
- Site
dataset_idtextRequired - Which of Bright Data's scrapers to use - one per site, such as 'Amazon - Products' or 'LinkedIn - People profiles'. Pick one from the list, or paste a scraper id if it is newer than this list.
- Pages to collect
urlslist - The page addresses to collect from, one per entry - product pages, profiles, listings. Each must start with http:// or https://. This is what almost every scraper takes; up to 1000 at a time.
- Other inputs
inputslist - Use this instead of 'Pages to collect' for a scraper that takes something other than a page address - a search keyword, a category. Each entry is a set of fields, such as {"keyword": "running shoes"}. Run 'Get a scraper's fields' to see what a scraper accepts.
- How to collect
collection_methodtext - How this scraper should run. Leave it empty to collect the pages you name above, which is what most workflows want. The other choices discover pages first - from a keyword, a category or a best-sellers list - and then collect what they find; each one takes its own inputs, which 'Get a scraper's fields' will show you.
- Only return these fields
fieldslist - Names of the fields to keep on each record, one per entry. Leave empty to get everything the scraper collects, which is often a hundred fields or more. 'Get a scraper's fields' lists the names.
- Most records per input
limit_per_inputnumber - Caps how many records each entry above can produce. Matters for a scraper pointed at a listing or a search, where one input can produce thousands. Leave empty for no cap.From 1 to 100000.
- Include pages that failed
include_errorstrue/false - Turn this on to get a row for every input the scraper could not collect, alongside the ones it could. Useful when the run has to account for every address it was given.
Outputs
dataset_idtext- The scraper it is running
input_countnumber- How many inputs it was given
job_idtext- The new job's id. Every other job action takes this
statustext- Always 'starting' - the job has just begun
Example output
{
"dataset_id": "gd_l7q7dkf244hwjntr0",
"input_count": 2,
"job_id": "s_m4x7enmven8djfqak",
"status": "starting"
}