DeepInfra

App · 20 actions

Run AI models and inspect model, deployment and inference batch information.

Actions

ActionWhat it does
Decode token IDsTurn token IDs back into readable text for a selected model.
Generate a chat responseGenerate a response using a DeepInfra chat model.
Generate an imageCreate an image using a DeepInfra image model.
Generate text embeddingsTurn text into vectors using a DeepInfra embedding model.
Get a voiceGet a voice from your DeepInfra connection.
Get account detailsGet account details from your DeepInfra connection.
Get account GPU poolGet account GPU pool from your DeepInfra connection.
Get account rate limitsGet account rate limits from your DeepInfra connection.
Get an inference batchGet an inference batch from your DeepInfra connection.
Get deployment detailsGet deployment details from your DeepInfra connection.
Get deployment statisticsGet deployment statistics from your DeepInfra connection.
List available voicesList available voices from your DeepInfra connection.
List deploymentsList deployments from your DeepInfra connection.
List featured modelsList featured models from your DeepInfra connection.
List GPU pool typesList GPU pool types from your DeepInfra connection.
List hardwareList hardware from your DeepInfra connection.
List inference batchesList inference batches from your DeepInfra connection.
List inference modelsList inference models from your DeepInfra connection.
List public modelsList public models from your DeepInfra connection.
Tokenize textConvert text to token IDs for a selected DeepInfra model.

Decode token IDs

Turn token IDs back into readable text for a selected model.

Inputs

ModelmodeltextRequired
Model.
TokenstokenslistRequired
Tokens.

Outputs

Response.

dataobject
Data.
Example output
{
  "data": {
    "prompt": "example"
  }
}

Generate a chat response

Generate a response using a DeepInfra chat model. Uses inference credits.

Inputs

ModelmodeltextRequired
Model.
MessagesmessageslistRequired
Messages.
Temperaturetemperaturenumber
Temperature.From 0 to 2.
Max tokensmax_tokensnumber
Max tokens.At least 1.

Outputs

Response.

dataobject
Data.
Example output
{
  "data": {
    "choices": [],
    "id": "example",
    "model": "example",
    "usage": {
      "completion_tokens": 1,
      "prompt_tokens": 1,
      "total_tokens": 1
    }
  }
}

Generate an image

Create an image using a DeepInfra image model. Uses inference credits.

Inputs

ModelmodeltextRequired
Model.
Nnnumber
N.From 1.0 to 4.0.
Response formatresponse_formattext
Response format.One of: b64_json (B64 json), url.
Sizesizetext
Size.
PromptprompttextRequired
Prompt.

Outputs

Response.

dataobject
Data.
Example output
{
  "data": {
    "created": 1,
    "data": []
  }
}

Generate text embeddings

Turn text into vectors using a DeepInfra embedding model. Uses inference credits.

Inputs

ModelmodeltextRequired
Model.
InputinputtextRequired
Input.
Encoding formatencoding_formattext
Encoding format.One of: float, base64.

Outputs

Response.

dataobject
Data.
Example output
{
  "data": {
    "data": [],
    "model": "example"
  }
}

Get a voice

Get a voice from your DeepInfra connection.

Inputs

Voicevoice_idtextRequired
Voice id.

Outputs

Response.

dataobject
Data.
Example output
{
  "data": {
    "created_at": 1,
    "description": "example",
    "name": "example",
    "updated_at": 1,
    "user_id": "example",
    "voice_id": "example"
  }
}

Get account details

Get account details from your DeepInfra connection.

Inputs

None.

Outputs

Response.

dataobject
Data.
Example output
{
  "data": {
    "account_setup": false,
    "can_access_agents": false,
    "can_access_sandboxes": false,
    "checklist": {
      "billing_address": false,
      "billing_address_info": {
        "city": "example",
        "country": "example",
        "line1": "example",
        "line2": "example",
        "name": "example",
        "postal_code": "example",
        "state": "example"
      },
      "billing_type": "example",
      "email": false,
      "last_checked": 1,
      "limit": 1.0,
      "overdue_invoices": 1.0,
      "payment_collection": "example",
      "payment_method": false,
      "payment_method_info": {
        "card": {
          "brand": "example",
          "last4": "example",
          "wallet": "example"
        },
        "cashapp": {
          "cashtag": "example"
        },
        "type": "example",
        "us_bank_account": {
          "bank_name": "example",
          "last4": "example"
        }
      },
      "payment_method_required": false,
      "recent": 1.0,
      "scoped_credits": [],
      "stripe_balance": 1.0,
      "suspend_reason": "balance",
      "suspended": false,
      "topup": false,
      "topup_amount": 1,
      "topup_failed": false,
      "topup_threshold": 1
    },
    "company": "example",
    "country": "example",
    "display_name": "example",
    "email": "example",
    "email_verified": false,
    "first_name": "example",
    "google_linked": false,
    "is_admin": false,
    "is_business_account": false,
    "is_team_account": false,
    "is_team_owner": false,
    "is_team_upgrade_enabled": false,
    "last_name": "example",
    "mfa_available": false,
    "mfa_enabled": false,
    "name": "example",
    "picture": "example",
    "provider": "example",
    "require_email_verified": false,
    "root_uid": "example",
    "show_signup_form": false,
    "team_display_name": "example",
    "team_role": "example",
    "title": "example",
    "uid": "example",
    "vercel_connection": {
      "installation_id": "example",
      "team_id": "example",
      "user_id": "example"
    },
    "website": "example"
  }
}

Get account GPU pool

Get account GPU pool from your DeepInfra connection.

Inputs

None.

Outputs

Response.

dataobject
Data.
Example output
{
  "data": {
    "last_rejection": {
      "created_at": 1,
      "id": "example",
      "reason": "example",
      "requested_max": {},
      "resolution_reason": "example",
      "resolved_at": 1
    },
    "max_limits": {},
    "min_limits": {},
    "pending_request": {
      "created_at": 1,
      "id": "example",
      "reason": "example",
      "requested_max": {}
    }
  }
}

Get account rate limits

Get account rate limits from your DeepInfra connection.

Inputs

None.

Outputs

Response.

dataobject
Data.
Example output
{
  "data": {}
}

Get an inference batch

Get an inference batch from your DeepInfra connection.

Inputs

Batch idbatch_idtextRequired
Batch id.

Outputs

Response.

dataobject
Data.
Example output
{
  "data": {
    "cancelled_at": 1,
    "cancelling_at": 1,
    "completed_at": 1,
    "completion_window": "example",
    "created_at": 1,
    "endpoint": "example",
    "error_file_id": "example",
    "errors": {
      "data": [],
      "object": "example"
    },
    "expired_at": 1,
    "expires_at": 1,
    "failed_at": 1,
    "finalizing_at": 1,
    "id": "example",
    "in_progress_at": 1,
    "input_file_id": "example",
    "metadata": {},
    "model": "example",
    "object": "example",
    "output_file_id": "example",
    "request_counts": {
      "completed": 1,
      "failed": 1,
      "total": 1
    },
    "status": "example",
    "usage": {
      "input_tokens": 1,
      "input_tokens_details": {
        "cached_tokens": 1,
        "image_tokens": 1
      },
      "output_tokens": 1,
      "output_tokens_details": {
        "reasoning_tokens": 1
      },
      "total_tokens": 1
    }
  }
}

Get deployment details

Get deployment details from your DeepInfra connection.

Inputs

Deploydeploy_idtextRequired
Deploy id.

Outputs

Response.

dataobject
Data.
Example output
{
  "data": {
    "config": {
      "gpu": "L4-24GB",
      "max_batch_size": 1,
      "num_gpus": 1,
      "weights": {
        "repo": "example",
        "revision": "example",
        "token": "example"
      }
    },
    "created_at": "example",
    "deploy_id": "example",
    "extra_args": [],
    "fail_reason": "example",
    "instances": {
      "pending": 1,
      "running": 1,
      "warming": 1
    },
    "model_name": "example",
    "rollout": {
      "booting": 1,
      "outdated": 1,
      "up_to_date": 1
    },
    "settings": {
      "max_instances": 1,
      "min_instances": 1
    },
    "standard_args": {
      "enable_prefix_caching": false,
      "gpu_memory_fraction": 1.0,
      "kv_cache_dtype": "auto",
      "max_concurrent_requests": 1,
      "max_context_size": 1,
      "max_images_per_prompt": 1,
      "max_prefill_tokens": 1,
      "quantization": "fp8"
    },
    "status": "example",
    "task": "example",
    "type": "legacy",
    "update_error": "example",
    "updated_at": "example",
    "version": "example"
  }
}

Get deployment statistics

Get deployment statistics from your DeepInfra connection.

Inputs

Deploydeploy_idtextRequired
Deploy id.

Outputs

Response.

dataobject
Data.
Example output
{
  "data": {
    "avg95_time": 1.0,
    "avg_time": 1.0,
    "errors": 1,
    "input_tokens": 1,
    "output_tokens": 1,
    "requests": 1,
    "total_amount": 1,
    "total_time": 1,
    "total_tokens": 1
  }
}

List available voices

List available voices from your DeepInfra connection.

Inputs

None.

Outputs

Response.

dataobject
Data.
Example output
{
  "data": {
    "voices": []
  }
}

List deployments

List deployments from your DeepInfra connection.

Inputs

Statusstatustext
Status.

Outputs

Response.

datalist
Data.
Example output
{
  "data": []
}

List featured models from your DeepInfra connection.

Inputs

None.

Outputs

Response.

datalist
Data.
Example output
{
  "data": []
}

List GPU pool types

List GPU pool types from your DeepInfra connection.

Inputs

None.

Outputs

Response.

dataobject
Data.
Example output
{
  "data": {
    "gpu_types": []
  }
}

List hardware

List hardware from your DeepInfra connection.

Inputs

None.

Outputs

Response.

dataobject
Data.
Example output
{
  "data": {
    "hardware": []
  }
}

List inference batches

List inference batches from your DeepInfra connection.

Inputs

Afteraftertext
After.
Limitlimitnumber
Limit.

Outputs

Response.

dataobject
Data.
Example output
{
  "data": {
    "data": [],
    "has_more": false,
    "last_id": "example"
  }
}

List inference models

List inference models from your DeepInfra connection.

Inputs

Sort bysort_bytext
Sort by.
Filterfiltertext
Filter.

Outputs

Response.

dataobject
Data.
Example output
{
  "data": {
    "data": [],
    "object": "example"
  }
}

List public models

List public models from your DeepInfra connection.

Inputs

None.

Outputs

Response.

datalist
Data.
Example output
{
  "data": []
}

Tokenize text

Convert text to token IDs for a selected DeepInfra model.

Inputs

ModelmodeltextRequired
Model.
PromptprompttextRequired
Prompt.
Return token strsreturn_token_strstrue/false
Return token strs.

Outputs

Response.

dataobject
Data.
Example output
{
  "data": {
    "count": 1,
    "max_model_len": 1,
    "token_strs": [],
    "tokens": []
  }
}

On this page