Custom LLM

The Custom LLM connection lets Siesta AI use a self-hosted or third-party model endpoint that implements the OpenAI-compatible Chat Completions API. Models are declared for each connection instead of being selected from Siesta AI's shared model catalog.

Custom LLM is a model connection for agents. It is not a tool connection, REST function definition, or data source.

Requirements

• An absolute OpenAI-compatible base URL that includes the API version path, for example https://llm.example.com/v1.
• Support for Chat Completions at the configured endpoint.
• An API key. Siesta AI requires one for Custom LLM inference even if the endpoint can otherwise accept unauthenticated requests.
• Network access from the Siesta AI backend to the endpoint.
• At least one model declared on the connection before an agent can select it.

To load the endpoint's advertised models automatically, it must also support the OpenAI-compatible models endpoint at <Base URL>/models, for example https://llm.example.com/v1/models.

Add the Connection

1. Open Connections.
2. Click Add Connection and select Custom LLM.
3. Enter a recognizable Name, for example Private LLM - PROD.
4. Enter the OpenAI-compatible Base URL, including /v1 when required by the provider.
5. Enter the ApiKey and choose the appropriate private or shared access policy.
6. Add and review at least one model definition, then save the connection.
7. Open an agent and select the Custom LLM connection and one of its declared models.

Current app availability

The backend supports model discovery and per-connection model definitions, but some current Siesta AI App versions do not yet show the model controls in the connection form. If the Models section is not available in your deployment, the visible form cannot complete a usable Custom LLM setup. Contact your Siesta AI administrator before using the connection in production.

Declare Models

Use Load models where the control is available to read model IDs from <Base URL>/models. A model whose name matches the Siesta AI catalog receives best-effort capability defaults. These values are editable hints, not a compatibility guarantee. An unknown model can still be declared manually.

Field	How Siesta AI uses it	Recommendation
Name	Sends this exact model identifier with Chat Completions requests and makes it selectable by agents.	Copy the ID returned by the endpoint. Names must be non-blank and unique within the connection, ignoring letter case.
Context Window	Defines the context size used by automatic conversation compaction.	Enter the provider's documented token limit. 0 means unknown and disables automatic compaction.
Supports Reasoning	Enables best-effort reasoning options and recovery of reasoning content for the declared model.	Enable only when the model and gateway have been tested with the expected reasoning behavior.
Function Invocation	Declares that the model supports function or tool invocation.	Disable it for models that cannot reliably produce compatible tool calls, and verify agent tools in a pilot.

Do not infer capabilities from a model name alone. Gateways can expose the same model ID with different context limits or feature support.

When editing a connection, model names remain scoped to that connection. Siesta AI blocks removal of a declared model while one or more agents still use it; reassign those agents first.

Verify the Connection

Before production use:

1. Confirm that model discovery returns the expected model IDs, or enter the exact ID manually.
2. Create a test agent with the Custom LLM connection and declared model.
3. Run a short chat without tools.
4. If enabled, test reasoning and function invocation separately.
5. Test a conversation that approaches the configured context window and verify the expected compaction behavior.
6. Review provider-side logs, latency, rate limits, and error responses.

Successful model discovery proves only that the models endpoint is reachable. It does not guarantee that Chat Completions, reasoning, tools, or the configured context window work correctly.

Security and Operations

• Siesta AI stores the API key through its backend secret store and does not treat it as a regular connection field.
• Use a narrowly scoped key and separate credentials for development, staging, and production.
• Use shared access only when the endpoint and credential are approved for the intended teams.
• Restrict network access to trusted Siesta AI backend egress where the provider supports allowlisting.
• Monitor endpoint availability, rate limits, latency, token usage, and provider cost independently of Siesta AI.
• Rotate a compromised or expired key at the provider and update the connection before resuming agent use.

Troubleshooting

Symptom	Likely cause	What to check
Base URL is rejected	The value is not an absolute URL or points to the wrong API root.	Use an https:// URL and include the provider's OpenAI-compatible version path, commonly /v1.
Models cannot be loaded	<Base URL>/models is unavailable, blocked, or requires different authentication.	Test backend network access, TLS, the API key, and the provider's models endpoint. You can declare a known model ID manually where model controls are available.
The agent reports that the model does not exist	The declared name differs from the ID accepted by the endpoint.	Copy the exact model ID, including punctuation and version suffixes.
Chat starts but tool calls fail	The gateway or model does not implement compatible function invocation.	Disable Function Invocation or fix the provider's Chat Completions tool-call support.
Reasoning is missing or malformed	Reasoning fields are not standardized across compatible gateways.	Disable Supports Reasoning or confirm the gateway's behavior with a pilot request.
Long conversations fail unexpectedly	The declared context window is higher than the endpoint's real limit, or it is 0.	Enter the documented limit and retest compaction before production use.
A model cannot be removed	An agent still references it.	Reassign every affected agent to another model, then update the connection.

Technical Notes

• Custom LLM uses the OpenAI-compatible Chat Completions API, not the OpenAI Responses API.
• Model discovery uses the supplied base URL and can use the supplied API key. Model invocation requires both, and discovery can fail independently of chat.
• Reasoning support is best-effort because compatible gateways do not expose it consistently.
• Siesta AI does not confirm image generation, transcription, vision, or other OpenAI API features for Custom LLM connections.
• Custom LLM has no catalog models of its own; every usable model is declared on the individual connection.

Summary

Custom LLM connects Siesta AI agents to an OpenAI-compatible model endpoint that you operate or choose. Configure the versioned base URL and API key, declare and test each model's capabilities, and treat discovery defaults as hints rather than proof of compatibility.