# Bright Data

> Get structured data out of a website without writing a scraper. Bright Data maintains ready-made scrapers for Amazon, LinkedIn, Instagram, TikTok, YouTube, Google Maps and thousands of other sites: point one at some pages and get back proper fields - price, stock, rating, job history - either in the step or from a job that finishes later.

App · AI Agents · 7 actions. Web page: https://docs.lodol.com/docs/integrations/bright-data

## Actions

### Check a collection job

See whether a collection job has finished and whether it worked. Answers with 'finished' and 'succeeded' as well as Bright Data's own status word, so a workflow can branch on them directly - wait and check again while it is running, download the results once it is ready, and report the reason if it failed.

**Inputs**

- Job (`job_id`, text, required): The collection job to look at. This is the id 'Start a collection job' returned - or the one 'Collect data from a site' hands back when the collection ran long enough that it stopped waiting.

**Outputs**

- `dataset_id` (text): The scraper it ran
- `error_message` (text): Why it failed, when it did
- `finished` (true/false): True once the job has stopped, whether or not it worked
- `job_id` (text): The job this describes
- `record_count` (number): How many records it has collected
- `status` (text): starting, running, ready, failed or canceled
- `succeeded` (true/false): True only when the job finished with results to download

**Example output**

```json
{
  "dataset_id": "gd_l7q7dkf244hwjntr0",
  "error_message": null,
  "finished": true,
  "job_id": "s_m4x7enmven8djfqak",
  "record_count": 2,
  "status": "ready",
  "succeeded": true
}
```

### Collect data from a site

Point one of Bright Data's site scrapers at a handful of pages and get structured records back in this step - a product with its price, stock and rating; a profile with its job history. Reach for this when the workflow needs the data now and there are tens of pages, not thousands. Bright Data allows a minute; a run that takes longer keeps going and hands back a job to pick up with 'Check a collection job'.

**Inputs**

- Site (`dataset_id`, text, required): Which of Bright Data's scrapers to use - one per site, such as 'Amazon - Products' or 'LinkedIn - People profiles'. Pick one from the list, or paste a scraper id if it is newer than this list.
- Pages to collect (`urls`, list): The page addresses to collect from, one per entry - product pages, profiles, listings. Each must start with http:// or https://. This is what almost every scraper takes; up to 1000 at a time.
- Other inputs (`inputs`, list): Use this instead of 'Pages to collect' for a scraper that takes something other than a page address - a search keyword, a category. Each entry is a set of fields, such as {"keyword": "running shoes"}. Run 'Get a scraper's fields' to see what a scraper accepts.
- How to collect (`collection_method`, text): How this scraper should run. Leave it empty to collect the pages you name above, which is what most workflows want. The other choices discover pages first - from a keyword, a category or a best-sellers list - and then collect what they find; each one takes its own inputs, which 'Get a scraper's fields' will show you.
- Only return these fields (`fields`, list): Names of the fields to keep on each record, one per entry. Leave empty to get everything the scraper collects, which is often a hundred fields or more. 'Get a scraper's fields' lists the names.
- Most records per input (`limit_per_input`, number): Caps how many records each entry above can produce. Matters for a scraper pointed at a listing or a search, where one input can produce thousands. Leave empty for no cap. From 1 to 100000.
- Include pages that failed (`include_errors`, true/false): Turn this on to get a row for every input the scraper could not collect, alongside the ones it could. Useful when the run has to account for every address it was given.

**Outputs**

- `dataset_id` (text): The scraper that ran
- `finished` (true/false): True when the records are in this step's result. False when the collection ran long and became a job instead
- `input_count` (number): How many inputs this step sent
- `job_id` (text): The job to check when 'finished' is false, and empty otherwise
- `record_count` (number): How many records came back
- `records` (list): The collected records

**Example output**

```json
{
  "dataset_id": "gd_l7q7dkf244hwjntr0",
  "finished": true,
  "input_count": 2,
  "job_id": null,
  "record_count": 2,
  "records": [
    {
      "availability": "In Stock",
      "brand": "Aurora",
      "currency": "USD",
      "final_price": 449.0,
      "rating": 4.6,
      "reviews_count": 1284,
      "title": "Aurora Bar 9 Espresso Machine",
      "url": "https://www.example-shop.com/dp/B0EXAMPLE1"
    },
    {
      "availability": "In Stock",
      "brand": "Aurora",
      "currency": "USD",
      "final_price": 199.0,
      "rating": 4.4,
      "reviews_count": 803,
      "title": "Aurora Grind Pro Burr Grinder",
      "url": "https://www.example-shop.com/dp/B0EXAMPLE2"
    }
  ]
}
```

### Get a scraper's fields

Show what one site's scraper needs and what it gives back: the inputs it takes, which of them are required, an example you can paste straight into a collection step, and every field on a collected record. A scraper can often be run more than one way - Amazon's takes a product address, or a search keyword - and this describes one of them and lists the rest.

**Inputs**

- Site (`dataset_id`, text, required): Which of Bright Data's scrapers to use - one per site, such as 'Amazon - Products' or 'LinkedIn - People profiles'. Pick one from the list, or paste a scraper id if it is newer than this list.
- How to collect (`collection_method`, text): Which way of running this scraper to describe. Leave it empty for the usual one - collecting the pages you name.

**Outputs**

- `collection_method` (text): Which way of running it this describes
- `collection_methods` (list): Every way this scraper can be run
- `configuration_url` (text): This scraper's page in the Bright Data control panel
- `dataset_id` (text): The scraper this describes
- `domain` (text): The site it collects from
- `example_input` (object): One ready-made input, straight from Bright Data. Paste it into 'Other inputs' on a collection step to try the scraper out
- `field_count` (number): How many fields that is
- `fields` (list): Every field a collected record can have
- `inputs` (list): What this way of running it accepts as input
- `name` (text): What the scraper collects
- `required_inputs` (list): The inputs it will not run without

**Example output**

```json
{
  "collection_method": "collect_by_url",
  "collection_methods": [
    "collect_by_url",
    "discover_by_keyword",
    "discover_by_category_url"
  ],
  "configuration_url": "https://brightdata.com/cp/scrapers/gd_l7q7dkf244hwjntr0/pdp/configuration",
  "dataset_id": "gd_l7q7dkf244hwjntr0",
  "domain": "amazon.com",
  "example_input": {
    "url": "https://www.amazon.com/dp/B0CRMZHDG8",
    "zipcode": "94107"
  },
  "field_count": 3,
  "fields": [
    {
      "description": "Product title",
      "name": "title",
      "type": "text"
    },
    {
      "description": "Product brand",
      "name": "brand",
      "type": "text"
    },
    {
      "description": "The price shown on the page, after any discount",
      "name": "final_price",
      "type": "price"
    }
  ],
  "inputs": [
    {
      "description": "The product page to collect",
      "name": "url",
      "required": true,
      "type": "url"
    },
    {
      "description": "Postcode to price and stock the product for",
      "name": "zipcode",
      "required": false,
      "type": "text"
    }
  ],
  "name": "Amazon products",
  "required_inputs": [
    "url"
  ]
}
```

### Get collection job results

Download the records a collection job produced. Asking before it has finished is not an error: the step comes back with 'ready' set to false and no records, so a workflow can check again on a schedule without a failed run in between.

**Inputs**

- Job (`job_id`, text, required): The collection job to look at. This is the id 'Start a collection job' returned - or the one 'Collect data from a site' hands back when the collection ran long enough that it stopped waiting.

**Outputs**

- `job_id` (text): The job these records came from
- `ready` (true/false): True when the job had finished and the records are here
- `record_count` (number): How many records came back
- `records` (list): The collected records
- `status` (text): Where the job is, when it was not ready yet

**Example output**

```json
{
  "job_id": "s_m4x7enmven8djfqak",
  "ready": true,
  "record_count": 2,
  "records": [
    {
      "availability": "In Stock",
      "brand": "Aurora",
      "currency": "USD",
      "final_price": 449.0,
      "rating": 4.6,
      "reviews_count": 1284,
      "title": "Aurora Bar 9 Espresso Machine",
      "url": "https://www.example-shop.com/dp/B0EXAMPLE1"
    },
    {
      "availability": "In Stock",
      "brand": "Aurora",
      "currency": "USD",
      "final_price": 199.0,
      "rating": 4.4,
      "reviews_count": 803,
      "title": "Aurora Grind Pro Burr Grinder",
      "url": "https://www.example-shop.com/dp/B0EXAMPLE2"
    }
  ],
  "status": "ready"
}
```

### List collection jobs

List the collection jobs run for one site, newest first, with each one's status and how many records it holds. Use it to pick up a job an earlier run started, to find yesterday's results, or to check how a scheduled collection has been doing. Narrow it by status or by the days it ran.

**Inputs**

- Site (`dataset_id`, text, required): Which of Bright Data's scrapers to use - one per site, such as 'Amazon - Products' or 'LinkedIn - People profiles'. Pick one from the list, or paste a scraper id if it is newer than this list.
- Only jobs that are (`status`, text): Keep only the jobs in this state. Leave empty for all of them. One of: `starting`, `running`, `ready`, `failed`, `canceled`.
- Started on or after (`from_date`, date): Only jobs started on this day or later. Leave empty for no earliest day.
- Started on or before (`to_date`, date): Only jobs started on this day or earlier. Leave empty for no latest day.
- How many to return (`limit`, number): The most jobs to return, newest first. Defaults to 50. From 1 to 5000. Default: `50`.

**Outputs**

- `dataset_id` (text): The scraper these jobs ran
- `job_count` (number): How many matched
- `jobs` (list): The matching jobs

**Example output**

```json
{
  "dataset_id": "gd_l7q7dkf244hwjntr0",
  "job_count": 2,
  "jobs": [
    {
      "created": "2026-09-14T08:12:44.201Z",
      "dataset_id": "gd_l7q7dkf244hwjntr0",
      "finished": true,
      "job_id": "s_m4x7enmven8djfqak",
      "record_count": 2,
      "started_by": "API",
      "status": "ready"
    },
    {
      "created": "2026-09-15T06:03:10.888Z",
      "dataset_id": "gd_l7q7dkf244hwjntr0",
      "finished": false,
      "job_id": "s_m4x7f1q2wc93hdl0p",
      "record_count": 0,
      "started_by": "API",
      "status": "running"
    }
  ]
}
```

### List site scrapers

List the sites Bright Data already knows how to collect structured data from - thousands of them, covering Amazon, LinkedIn, Instagram, TikTok, YouTube, Google Maps and more. Filter by name to find one. You do not need this to build a workflow, because every collection step offers the same list as a dropdown; it is here for a workflow that has to look a site up while it runs.

**Inputs**

- Filter by name (`search`, text): Keep only the scrapers whose name, site or id contains this - 'amazon', 'linkedin.com', 'profile'. Leave empty for all of them.

**Outputs**

- `scraper_count` (number): How many matched
- `scrapers` (list): The matching scrapers, in name order
- `search` (text): The filter that was applied, if any

**Example output**

```json
{
  "scraper_count": 2,
  "scrapers": [
    {
      "collection_methods": [
        "collect_by_url",
        "discover_by_keyword",
        "discover_by_category_url"
      ],
      "dataset_id": "gd_l7q7dkf244hwjntr0",
      "description": "eCommerce product data for amazon.com and its categories",
      "domain": "amazon.com",
      "name": "Amazon products",
      "record_count": 300000000
    },
    {
      "collection_methods": [
        "collect_by_url",
        "discover_by_keyword"
      ],
      "dataset_id": "gd_l1viktl72bvl7bjuj0",
      "description": "Public profile data: roles, employers and education",
      "domain": "linkedin.com",
      "name": "LinkedIn people profiles",
      "record_count": 690000000
    }
  ],
  "search": null
}
```

### Start a collection job

Set a site scraper running over a large batch without waiting for it. Returns a job id straight away; the collection keeps going on Bright Data's side, and 'Check a collection job' and 'Get collection job results' pick it up - in this workflow or in a scheduled one that runs later. Use this for hundreds or thousands of pages, where waiting in a step is not an option.

**Inputs**

- Site (`dataset_id`, text, required): Which of Bright Data's scrapers to use - one per site, such as 'Amazon - Products' or 'LinkedIn - People profiles'. Pick one from the list, or paste a scraper id if it is newer than this list.
- Pages to collect (`urls`, list): The page addresses to collect from, one per entry - product pages, profiles, listings. Each must start with http:// or https://. This is what almost every scraper takes; up to 1000 at a time.
- Other inputs (`inputs`, list): Use this instead of 'Pages to collect' for a scraper that takes something other than a page address - a search keyword, a category. Each entry is a set of fields, such as {"keyword": "running shoes"}. Run 'Get a scraper's fields' to see what a scraper accepts.
- How to collect (`collection_method`, text): How this scraper should run. Leave it empty to collect the pages you name above, which is what most workflows want. The other choices discover pages first - from a keyword, a category or a best-sellers list - and then collect what they find; each one takes its own inputs, which 'Get a scraper's fields' will show you.
- Only return these fields (`fields`, list): Names of the fields to keep on each record, one per entry. Leave empty to get everything the scraper collects, which is often a hundred fields or more. 'Get a scraper's fields' lists the names.
- Most records per input (`limit_per_input`, number): Caps how many records each entry above can produce. Matters for a scraper pointed at a listing or a search, where one input can produce thousands. Leave empty for no cap. From 1 to 100000.
- Include pages that failed (`include_errors`, true/false): Turn this on to get a row for every input the scraper could not collect, alongside the ones it could. Useful when the run has to account for every address it was given.

**Outputs**

- `dataset_id` (text): The scraper it is running
- `input_count` (number): How many inputs it was given
- `job_id` (text): The new job's id. Every other job action takes this
- `status` (text): Always 'starting' - the job has just begun

**Example output**

```json
{
  "dataset_id": "gd_l7q7dkf244hwjntr0",
  "input_count": 2,
  "job_id": "s_m4x7enmven8djfqak",
  "status": "starting"
}
```