# SambaNova Cloud

> Discover SambaNova models and generate text, responses and embeddings.

App · 8 actions. Web page: https://docs.lodol.com/docs/integrations/sambanova-cloud

## Actions

### Chat

Chat.

**Inputs**

- Model (`model`, text, required): The model ID to use (e.g. gpt-oss-120b). See available [models](https://docs.sambanova.ai/docs/en/models/sambacloud-models)
- Max tokens (`max_tokens`, number): The maximum number of tokens that can be generated in the chat completion. The total length of input tokens and generated tokens is limited by the model's context length.
- Max completion tokens (`max_completion_tokens`, number): The maximum number of tokens that can be generated in the chat completion. The total length of input tokens and generated tokens is limited by the model's context length.
- Temperature (`temperature`, number): What sampling temperature to use, determines the degree of randomness in the response. between 0 and 2, Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic. Is recommended altering this, top_p or top_k but not more than one of these. From 0 to 2. Default: `0.7`.
- Top p (`top_p`, number): Cumulative probability for token choices. An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered. Is recommended altering this, top_k or temperature but not more than one of these. From 0 to 1.
- Top k (`top_k`, number): Amount limit of token choices. An alternative to sampling with temperature, the model considers the results of the first K tokens with higher probability. So 10 means only the first 10 tokens with higher probability are considered. Is recommended altering this, top_p or temperature but not more than one of these. From 1 to 100.
- Presence penalty (`presence_penalty`, number): Number between -2.0 and 2.0. Positive values penalize new tokens based on whether they appear in the text so far, increasing the model's likelihood to talk about new topics. Not currently implemented; accepted for API compatibility. From -2 to 2. Default: `0`.
- Frequency penalty (`frequency_penalty`, number): Number between -2.0 and 2.0. Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model's likelihood to repeat the same line verbatim. Not currently implemented; accepted for API compatibility. From -2 to 2. Default: `0`.
- Repetition penalty (`repetition_penalty`, number): Penalizes repeated tokens. A value of 1.0 applies no penalty; values above 1.0 discourage repetition. Only supported for some models (currently MiniMax and gpt-oss models); silently ignored on models that do not support it. From 1 to 2.
- Stop (`stop`, text): Sequences where the API will stop generating tokens. The returned text will not contain the stop sequence.
- Stream options include usage (`stream_options_include_usage`, true/false): Whether to include the usage metrics in a final chunk or not
- Response format json schema (`response_format_json_schema`, object): A JSON Schema definition the model's structured output. Follows standard JSON Schema syntax.
- Response format type (`response_format_type`, text): Response format type. One of: `json_schema` (Json schema).
- Reasoning effort (`reasoning_effort`, text): Value specifying the amount of reasoning the model is allowed to do, increasing it will increase the number of output reasoning tokens generated by the model, but will improve quality of the responses. allowed values are 'low', 'medium', 'high'. One of: `low`, `medium`, `high`.
- Tool choice (`tool_choice`, text): Controls which (if any) tool is called by the model. `none` means the model will not call any tool and instead generates a message. `auto` means the model can pick between generating a message or calling one or more tools. `required` means the model must call one or more tools. Specifying a particular tool via `{"type": "function", "function": {"name": "my_function"}}` forces the model to call that tool. One of: `none`, `auto`, `required`.
- Parallel tool calls (`parallel_tool_calls`, true/false): Whether to enable parallel function calling during tool use.
- Tools (`tools`, list): A list of tools the model may call. Use this to provide a list of functions the model may generate JSON inputs for.
- Chat template kwargs enable thinking (`chat_template_kwargs_enable_thinking`, true/false): Enables the model's internal reasoning or "thinking" mode, if supported by the chat template (deepseek models).
- Logprobs (`logprobs`, true/false): Whether to return log probabilities of the output tokens or not. If true, returns the log probabilities of each output token returned in the `content` of `message`. Default: `false`.
- Top logprobs (`top_logprobs`, number): An integer between 0 and 20 specifying the number of most likely tokens to return at each token position, each with an associated log probability. `logprobs` must be set to `true` if this parameter is used. From 0 to 20.
- N (`n`, number): How many completions to generate for each prompt. **Note:** Because this parameter generates many completions, it can quickly consume your token quota. Use carefully and ensure that you have reasonable settings for `max_tokens`. From 1 to 8. Default: `1`.
- Logit bias (`logit_bias`, object): Modify the likelihood of specified tokens appearing in the generation. Accepts a JSON object that maps tokens (specified by their token ID in the model tokenizer) to an associated bias value from -100 to 100. Mathematically, the bias is added to the logits generated by the model prior to sampling. Values between -1 and 1 should decrease or increase likelihood of selection; values like -100 or 100 should result in a ban or exclusive selection of the relevant token.
- Seed (`seed`, number): If specified, our system will make a best effort to sample deterministically, such that repeated requests with the same `seed` and parameters should return the same result. Determinism is not guaranteed, and you should refer to the `system_fingerprint` response parameter to monitor changes in the backend.
- Prompt (`prompt`, text, required): The message to send to the model.

**Outputs**

- `data` (object): chat completion response returned by the model

**Example output**

```json
{
  "data": {
    "choices": [],
    "created": 1.0,
    "id": "example",
    "model": "example",
    "object": "chat.completion",
    "system_fingerprint": "example",
    "usage": {
      "acceptance_rate": 1.0,
      "completion_tokens": 1,
      "completion_tokens_after_first_per_sec": 1.0,
      "completion_tokens_after_first_per_sec_first_ten": 1.0,
      "completion_tokens_after_first_per_sec_graph": 1.0,
      "completion_tokens_details": {
        "reasoning_tokens": 1
      },
      "completion_tokens_per_sec": 1.0,
      "end_time": 1.0,
      "is_last_response": true,
      "prompt_tokens": 1,
      "prompt_tokens_details": {
        "cached_tokens": 1
      },
      "start_time": 1.0,
      "stop_reason": "example",
      "time_to_first_token": 1.0,
      "time_to_first_token_graph": 1.0,
      "total_latency": 1.0,
      "total_tokens": 1,
      "total_tokens_per_sec": 1.0
    }
  }
}
```

### Complete

Complete.

**Inputs**

- Model (`model`, text, required): The model ID to use (e.g. gpt-oss-120b). See available [models](https://docs.sambanova.ai/docs/en/models/sambacloud-models)
- Prompt (`prompt`, text, required): Prompt to send to the model.
- Max tokens (`max_tokens`, number): The maximum number of tokens that can be generated in the chat completion. The total length of input tokens and generated tokens is limited by the model's context length.
- Max completion tokens (`max_completion_tokens`, number): The maximum number of tokens that can be generated in the chat completion. The total length of input tokens and generated tokens is limited by the model's context length.
- Temperature (`temperature`, number): What sampling temperature to use, determines the degree of randomness in the response. between 0 and 2, Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic. Is recommended altering this, top_p or top_k but not more than one of these. From 0 to 2. Default: `0.7`.
- Top p (`top_p`, number): Cumulative probability for token choices. An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered. Is recommended altering this, top_k or temperature but not more than one of these. From 0 to 1.
- Top k (`top_k`, number): Amount limit of token choices. An alternative to sampling with temperature, the model considers the results of the first K tokens with higher probability. So 10 means only the first 10 tokens with higher probability are considered. Is recommended altering this, top_p or temperature but not more than one of these. From 1 to 100.
- Presence penalty (`presence_penalty`, number): Number between -2.0 and 2.0. Positive values penalize new tokens based on whether they appear in the text so far, increasing the model's likelihood to talk about new topics. Not currently implemented; accepted for API compatibility. From -2 to 2. Default: `0`.
- Frequency penalty (`frequency_penalty`, number): Number between -2.0 and 2.0. Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model's likelihood to repeat the same line verbatim. Not currently implemented; accepted for API compatibility. From -2 to 2. Default: `0`.
- Repetition penalty (`repetition_penalty`, number): Penalizes repeated tokens. A value of 1.0 applies no penalty; values above 1.0 discourage repetition. Only supported for some models (currently MiniMax and gpt-oss models); silently ignored on models that do not support it. From 1 to 2.
- Stop (`stop`, text): Sequences where the API will stop generating tokens. The returned text will not contain the stop sequence.
- Stream options include usage (`stream_options_include_usage`, true/false): Whether to include the usage metrics in a final chunk or not
- Logprobs (`logprobs`, true/false): This is not yet supported by our models in completions API. Whether to return log probabilities of the output tokens or not. If true, returns the log probabilities of each output token returned in the `content` of `message`. Default: `false`.
- Top logprobs (`top_logprobs`, number): This is not yet supported by our models in completions API. An integer between 0 and 20 specifying the number of most likely tokens to return at each token position, each with an associated log probability. `logprobs` must be set to `true` if this parameter is used. From 0 to 20.
- N (`n`, number): How many completions to generate for each prompt. **Note:** Because this parameter generates many completions, it can quickly consume your token quota. Use carefully and ensure that you have reasonable settings for `max_tokens`. From 1 to 1. Default: `1`.
- Logit bias (`logit_bias`, object): Modify the likelihood of specified tokens appearing in the generation. Accepts a JSON object that maps tokens (specified by their token ID in the model tokenizer) to an associated bias value from -100 to 100. Mathematically, the bias is added to the logits generated by the model prior to sampling. Values between -1 and 1 should decrease or increase likelihood of selection; values like -100 or 100 should result in a ban or exclusive selection of the relevant token.
- Seed (`seed`, number): If specified, our system will make a best effort to sample deterministically, such that repeated requests with the same `seed` and parameters should return the same result. Determinism is not guaranteed, and you should refer to the `system_fingerprint` response parameter to monitor changes in the backend.

**Outputs**

- `data` (object): ompletion response returned by the model

**Example output**

```json
{
  "data": {
    "choices": [],
    "created": 1.0,
    "id": "example",
    "model": "example",
    "object": "chat.completion",
    "system_fingerprint": "example",
    "usage": {
      "acceptance_rate": 1.0,
      "completion_tokens": 1,
      "completion_tokens_after_first_per_sec": 1.0,
      "completion_tokens_after_first_per_sec_first_ten": 1.0,
      "completion_tokens_after_first_per_sec_graph": 1.0,
      "completion_tokens_details": {
        "reasoning_tokens": 1
      },
      "completion_tokens_per_sec": 1.0,
      "end_time": 1.0,
      "is_last_response": true,
      "prompt_tokens": 1,
      "prompt_tokens_details": {
        "cached_tokens": 1
      },
      "start_time": 1.0,
      "stop_reason": "example",
      "time_to_first_token": 1.0,
      "time_to_first_token_graph": 1.0,
      "total_latency": 1.0,
      "total_tokens": 1,
      "total_tokens_per_sec": 1.0
    }
  }
}
```

### Count message tokens

Count message tokens.

**Inputs**

- Model (`model`, text, required): Model identifier.
- System (`system`, text): System prompt for the conversation. Accepts either a single string (most common) or an array of text blocks (used when individual segments need `cache_control` markers). Multiple text blocks are joined with newlines and prepended to the conversation as a `role: system` message.
- Thinking type (`thinking_type`, text): Thinking type. One of: `disabled`.
- Tools (`tools`, list): Tool definitions the model may call.
- Tool choice type (`tool_choice_type`, text): Tool choice type. One of: `auto`.
- Tool choice disable parallel tool use (`tool_choice_disable_parallel_tool_use`, true/false): Silently dropped.
- Prompt (`prompt`, text, required): The message to send to the model.

**Outputs**

- `data` (object): Token count for the supplied prompt.

**Example output**

```json
{
  "data": {
    "input_tokens": 1
  }
}
```

### Create embeddings

Create embeddings.

**Inputs**

- Model (`model`, text, required): The model ID to use See available [models](https://docs.sambanova.ai/docs/en/models/sambacloud-models)
- Input (`input`, text, required): Input text to embed. to embed multiple inputs in a single request, pass an array of strings. The input must not exceed the max input tokens for the model
- Encoding format (`encoding_format`, text): The format to return the embeddings in. Can be either `float` or `base64`. Omitted from the request when not set. One of: `float`, `base64`. Default: `float`.

**Outputs**

- `data` (object): Embeddings response returned by the model

**Example output**

```json
{
  "data": {
    "data": [],
    "model": "example",
    "object": "list",
    "usage": {
      "acceptance_rate": 1.0,
      "completion_tokens": 1,
      "completion_tokens_after_first_per_sec": 1.0,
      "completion_tokens_after_first_per_sec_first_ten": 1.0,
      "completion_tokens_after_first_per_sec_graph": 1.0,
      "completion_tokens_details": {
        "reasoning_tokens": 1
      },
      "completion_tokens_per_sec": 1.0,
      "end_time": 1.0,
      "is_last_response": true,
      "prompt_tokens": 1,
      "prompt_tokens_details": {
        "cached_tokens": 1
      },
      "start_time": 1.0,
      "stop_reason": "example",
      "time_to_first_token": 1.0,
      "time_to_first_token_graph": 1.0,
      "total_latency": 1.0,
      "total_tokens": 1,
      "total_tokens_per_sec": 1.0
    }
  }
}
```

### Create message

Create message.

**Inputs**

- Model (`model`, text, required): The model ID to use (e.g. gpt-oss-120b). See available [models](https://docs.sambanova.ai/docs/en/models/sambacloud-models)
- Max tokens (`max_tokens`, number, required): Maximum number of tokens to generate. The combined input + output token count is bounded by the model's context window. At least 1.
- System (`system`, text): System prompt for the conversation. Accepts either a single string (most common) or an array of text blocks (used when individual segments need `cache_control` markers). Multiple text blocks are joined with newlines and prepended to the conversation as a `role: system` message.
- Temperature (`temperature`, number): Sampling temperature in `[0.0, 2.0]`. Higher values produce more random output, lower values more deterministic. Adjust only one of `temperature`, `top_p`, `top_k`. From 0 to 2.
- Top p (`top_p`, number): Nucleus sampling. Considers tokens with cumulative probability mass up to `top_p`. From 0 to 1.
- Top k (`top_k`, number): Top-k sampling. Considers only the K most likely tokens at each step. Set to 0 to disable. At least 0.
- Stop sequences (`stop_sequences`, list): Custom strings that, when generated, cause the model to stop.
- Metadata user id (`metadata_user_id`, text): External identifier for the end-user making the request. Mapped internally to the Chat Completions `user` field.
- Thinking type (`thinking_type`, text): Thinking type. One of: `disabled`.
- Tools (`tools`, list): Tool definitions the model may call.
- Tool choice type (`tool_choice_type`, text): Tool choice type. One of: `auto`.
- Tool choice disable parallel tool use (`tool_choice_disable_parallel_tool_use`, true/false): Silently dropped.
- Service tier (`service_tier`, text): Service-tier preference. **In v1**: silently dropped. One of: `auto`, `standard_only` (Standard only).
- Container (`container`, text): Existing code-execution container ID to reuse. **In v1**: silently dropped
- Prompt (`prompt`, text, required): The message to send to the model.

**Outputs**

- `data` (object): Non-streaming response from `POST /messages`. Wire-compatible with the official Anthropic Messages API.

**Example output**

```json
{
  "data": {
    "container": {
      "expires_at": "example",
      "id": "example"
    },
    "content": [],
    "id": "example",
    "model": "example",
    "role": "assistant",
    "stop_details": {
      "category": "cyber",
      "type": "refusal"
    },
    "stop_reason": "end_turn",
    "stop_sequence": "example",
    "type": "message",
    "usage": {
      "cache_creation": {},
      "cache_creation_input_tokens": 1,
      "cache_read_input_tokens": 1,
      "inference_geo": "example",
      "input_tokens": 1,
      "output_tokens": 1,
      "server_tool_use": {},
      "service_tier": "example"
    }
  }
}
```

### Create response

Create response.

**Inputs**

- Model (`model`, text, required): The model ID to use (e.g. gpt-oss-120b). See available [models](https://docs.sambanova.ai/docs/en/models/sambacloud-models)
- Input (`input`, text, required): Text input to the model, or a structured list of input items representing the full conversation turn. A plain string is equivalent to a single user message.
- Instructions (`instructions`, text): Inserts a system (or developer) message as the first item in the model's context. Equivalent to a system-role message prepended to input[].
- Max output tokens (`max_output_tokens`, number): Upper bound on the number of tokens the model may generate, including visible output tokens and reasoning tokens.
- Temperature (`temperature`, number): Controls randomness in generation. Range: 0 to 2. It is recommended to alter this, top_p, or top_k but not more than one at a time. From 0 to 2. Default: `0.7`.
- Top p (`top_p`, number): Nucleus sampling cutoff. Range: 0 to 1. It is recommended to alter this, temperature, or top_k but not more than one at a time. From 0 to 1. Default: `1`.
- Top k (`top_k`, number): Limits sampling to the top K most probable tokens. It is recommended to alter this, top_p, or temperature but not more than one at a time. From 1 to 100.
- Top logprobs (`top_logprobs`, number): Number of top log-probability entries to return per output token. Null means log probabilities are not returned.
- Frequency penalty (`frequency_penalty`, number): Number between -2.0 and 2.0. Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model's likelihood to repeat the same line verbatim. Not currently implemented; accepted for API compatibility and echoed in the response. From -2 to 2. Default: `0`.
- Presence penalty (`presence_penalty`, number): Number between -2.0 and 2.0. Positive values penalize new tokens based on whether they appear in the text so far, increasing the model's likelihood to talk about new topics. Not currently implemented; accepted for API compatibility and echoed in the response. From -2 to 2. Default: `0`.
- Tools (`tools`, list): Tools available to the model. Only type: "function" is supported; all other tool types are filtered server-side.
- Parallel tool calls (`parallel_tool_calls`, true/false): Whether the model may issue multiple tool calls in parallel within one turn. Default: `true`.
- Max tool calls (`max_tool_calls`, number): Maximum number of tool calls the model may make in a single response turn. Not currently implemented; accepted for API compatibility.
- Tool choice (`tool_choice`, text): Controls which tool (if any) the model calls. String values: "none" disables tool calls; "auto" lets the model decide; "required" forces at least one tool call. To force a specific function, provide a ResponseNamedToolChoice object. One of: `none`, `auto`, `required`.
- Text format (`text_format`, object): Controls the format of the model's output for the Responses API text.format field. "text" returns plain text (default). "json_object" returns a raw JSON object. "json_schema" enforces structured output matching a provided JSON Schema definition (fields are flat , no json_schema wrapper key).
- Reasoning effort (`reasoning_effort`, text): Reasoning effort level. "low" is faster with less depth; "high" is deeper at higher token cost. One of: `low`, `medium`, `high`. Default: `medium`.
- User (`user`, text): Included for API compatibility, but only echoed back in response
- Metadata (`metadata`, object): Included for API compatibility, but not supported
- Store (`store`, true/false): SambaNova is stateless - this field is accepted for API compatibility but has no effect. Always echoed back as false. Default: `false`.
- Truncation (`truncation`, text): Accepted for API compatibility and echoed in the response. Context truncation behavior is not currently configurable via this field in SambaNova. One of: `auto`, `disabled`. Default: `disabled`.
- Previous response id (`previous_response_id`, text): Not supported. SambaNova is stateless and does not maintain server-side conversation state. Accepted for API compatibility but ignored; clients must supply the full conversation history in input[].
- Background (`background`, true/false): Accepted for API compatibility and echoed back in the response. Has no effect on server behavior.
- Service tier (`service_tier`, text): Accepted for API compatibility and echoed back in the response. Has no effect on server behavior.

**Outputs**

- `data` (object): A response object returned by POST /responses (non-streaming). Contains the model's output items, echoed input parameters, lifecycle metadata, and token usage.

**Example output**

```json
{
  "data": {
    "background": false,
    "completed_at": 1,
    "created_at": 1,
    "error": {
      "code": "example",
      "message": "example"
    },
    "frequency_penalty": 1.0,
    "id": "example",
    "incomplete_details": {
      "reason": "example"
    },
    "instructions": "example",
    "max_output_tokens": 1,
    "max_tool_calls": 1,
    "metadata": {},
    "model": "example",
    "object": "response",
    "output": [],
    "parallel_tool_calls": false,
    "presence_penalty": 1.0,
    "previous_response_id": "example",
    "reasoning": {
      "effort": "low",
      "summary": "example"
    },
    "service_tier": "example",
    "status": "completed",
    "store": false,
    "temperature": 1.0,
    "text": {
      "format": {
        "type": "text"
      }
    },
    "tool_choice": "none",
    "tools": [],
    "top_k": 1,
    "top_logprobs": 1,
    "top_p": 1.0,
    "truncation": "auto",
    "usage": {
      "acceptance_rate": 1.0,
      "end_time": 1.0,
      "input_tokens": 1,
      "input_tokens_details": {
        "cached_tokens": 1
      },
      "is_last_response": false,
      "output_tokens": 1,
      "output_tokens_after_first_per_sec": 1.0,
      "output_tokens_details": {
        "reasoning_tokens": 1
      },
      "output_tokens_per_sec": 1.0,
      "start_time": 1.0,
      "time_to_first_token": 1.0,
      "total_latency": 1.0,
      "total_tokens": 1,
      "total_tokens_per_sec": 1.0
    },
    "user": "example"
  }
}
```

### Get model

Get model.

**Inputs**

- Model (`model_id`, text, required): model id to get metadata

**Outputs**

- `data` (object): model metadata

**Example output**

```json
{
  "data": {
    "context_length": 1,
    "id": "example",
    "max_completion_tokens": 1,
    "object": "model",
    "owned_by": "example",
    "pricing": {
      "completion": 1.0,
      "duration_per_hour": 1.0,
      "prompt": 1.0
    },
    "sn_metadata": {}
  }
}
```

### List models

List models. Returns one page; use the continuation fields for more results.

**Inputs**: None.

**Outputs**

- `data` (object): object containing available models and metadata

**Example output**

```json
{
  "data": {
    "data": [],
    "object": "list"
  }
}
```
