Docs

Deploy an OpenAI-compatible model

Deploy an open model with one command and call it with any OpenAI client and your Klio key.

For agents: this page as markdown

Deploy a model

klio model list
klio model deploy qwen3-14b

klio model list shows the models you can deploy. klio model deploy qwen3-14b deploys it on an L40S behind an OpenAI-compatible API on your own address, shows progress while it starts, and prints the base URL and how to call it.

Call it

Use the printed base URL with any OpenAI client and a key from Keys in the console. Requests without a key are refused.

import os
from openai import OpenAI

client = OpenAI(base_url="https://<your-deployment-address>/v1", api_key=os.environ["KLIO_TOKEN"])
reply = client.chat.completions.create(
    model="qwen3-14b",
    messages=[{"role": "user", "content": "Hello"}],
)
print(reply.choices[0].message.content)
curl https://<your-deployment-address>/v1/chat/completions \
  -H "Authorization: Bearer $KLIO_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"model": "qwen3-14b", "messages": [{"role": "user", "content": "Hello"}]}'

Starting, stopping and cost

  • From no machine, the first start has taken about 12 minutes, and that can vary. --no-wait returns once it is deployed; --timeout sets how many minutes to wait.
  • It stops by itself after 5 minutes without requests, and --keep-warm sets how many seconds it stays up after the last one. You pay while it runs.
  • The next request starts it again and waits while it loads.
  • klio deployment stop <deployment-id> removes it; the deploy prints the command with its id.

© 2026 Klio