# Spider

> Scrape, crawl, search and transform public web content with Spider Cloud.

App · 10 actions. Web page: https://docs.lodol.com/docs/integrations/spider

## Actions

### Collect links on the Unlimited plan

Fetch publicly accessible web content with your Spider account. Uses Spider credits or plan allowance.

**Inputs**

- Url (`url`, text, required): Url.
- Return format (`return_format`, text): Return format. One of: `raw`, `markdown`, `commonmark`, `text`, `xml`, `empty`.
- Request (`request`, text): Request. One of: `smart`, `http`, `browser`.
- Metadata (`metadata`, true/false): Metadata.

**Outputs**: Response.

- `data` (list): Data.

**Example output**

```json
{
  "data": []
}
```

### Collect website links

Fetch publicly accessible web content with your Spider account. Uses Spider credits or plan allowance.

**Inputs**

- Url (`url`, text, required): Url.
- Return format (`return_format`, text): Return format. One of: `raw`, `markdown`, `commonmark`, `text`, `xml`, `empty`.
- Request (`request`, text): Request. One of: `smart`, `http`, `browser`.
- Metadata (`metadata`, true/false): Metadata.

**Outputs**: Response.

- `data` (list): Data.

**Example output**

```json
{
  "data": []
}
```

### Convert HTML

Convert existing HTML to the chosen content format. Uses Spider credits.

**Inputs**

- Data (`data`, list, required): Data.
- Return format (`return_format`, text): Return format. One of: `raw`, `markdown`, `commonmark`, `text`, `xml`, `empty`.
- Readability (`readability`, true/false): Readability.
- Clean (`clean`, true/false): Clean.
- Clean full (`clean_full`, true/false): Clean full.

**Outputs**: Response.

- `data` (object): Data.

**Example output**

```json
{
  "data": {
    "content": [],
    "error": "example",
    "status": 1
  }
}
```

### Crawl a website

Fetch publicly accessible web content with your Spider account. Uses Spider credits or plan allowance.

**Inputs**

- Url (`url`, text, required): Url.
- Limit (`limit`, number): Limit.
- Return format (`return_format`, text): Return format. One of: `raw`, `markdown`, `commonmark`, `text`, `xml`, `empty`.
- Request (`request`, text): Request. One of: `smart`, `http`, `browser`.
- Depth (`depth`, number): Depth.
- Subdomains (`subdomains`, true/false): Subdomains.
- Metadata (`metadata`, true/false): Metadata.

**Outputs**: Response.

- `data` (list): Data.

**Example output**

```json
{
  "data": []
}
```

### Crawl a website on the Unlimited plan

Fetch publicly accessible web content with your Spider account. Uses Spider credits or plan allowance.

**Inputs**

- Url (`url`, text, required): Url.
- Limit (`limit`, number): Limit.
- Return format (`return_format`, text): Return format. One of: `raw`, `markdown`, `commonmark`, `text`, `xml`, `empty`.
- Request (`request`, text): Request. One of: `smart`, `http`, `browser`.
- Depth (`depth`, number): Depth.
- Subdomains (`subdomains`, true/false): Subdomains.
- Metadata (`metadata`, true/false): Metadata.

**Outputs**: Response.

- `data` (list): Data.

**Example output**

```json
{
  "data": []
}
```

### Get credit balance

Read the current Spider credit balance.

**Inputs**: None.

**Outputs**: Response.

- `data` (object): Data.

**Example output**

```json
{
  "data": {
    "data": {
      "created_at": "example",
      "credits": 1.0,
      "id": "example",
      "updated_at": "example",
      "user_id": "example"
    }
  }
}
```

### Scrape a page

Fetch publicly accessible web content with your Spider account. Uses Spider credits or plan allowance.

**Inputs**

- Url (`url`, text, required): Url.
- Return format (`return_format`, text): Return format. One of: `raw`, `markdown`, `commonmark`, `text`, `xml`, `empty`.
- Request (`request`, text): Request. One of: `smart`, `http`, `browser`.
- Metadata (`metadata`, true/false): Metadata.

**Outputs**: Response.

- `data` (list): Data.

**Example output**

```json
{
  "data": []
}
```

### Scrape a page on the Unlimited plan

Fetch publicly accessible web content with your Spider account. Uses Spider credits or plan allowance.

**Inputs**

- Url (`url`, text, required): Url.
- Return format (`return_format`, text): Return format. One of: `raw`, `markdown`, `commonmark`, `text`, `xml`, `empty`.
- Request (`request`, text): Request. One of: `smart`, `http`, `browser`.
- Metadata (`metadata`, true/false): Metadata.

**Outputs**: Response.

- `data` (list): Data.

**Example output**

```json
{
  "data": []
}
```

### Search the web

Search public web pages by keyword. Uses Spider credits.

**Inputs**

- Search (`search`, text, required): Search.
- Search limit (`search_limit`, number): Search limit. From 1 to 100.

**Outputs**: Response.

- `data` (object): Data.

**Example output**

```json
{
  "data": {
    "content": []
  }
}
```

### Take a screenshot

Capture a publicly accessible page as an image. Uses Spider credits.

**Inputs**

- Url (`url`, text, required): Url.

**Outputs**: Response.

- `data` (list): Data.

**Example output**

```json
{
  "data": []
}
```
