# Klio Compute docs > Serverless GPUs for Python. Run a function on a GPU with one decorator, deploy endpoints, task queues, schedules and sandboxes, or deploy an open model behind an OpenAI-compatible API. It scales to zero, and you pay per second from prepaid credit. # Quick start > Install the Klio Compute SDK, sign in, and run a Python function in the cloud. Canonical page: https://klio.tech/docs/quickstart ## Install ```bash pip install klio-compute ``` Install needs Python 3.10 to 3.13 on your machine. Your code runs on Python 3.12 in Klio Compute. The newest release is on PyPI and is also named in [latest.txt](https://klio.tech/sdk/latest.txt). ## Sign in ```bash klio login ``` This opens the sign-in page in your browser and creates an account if you don't have one. On a machine without a browser, such as a server, a container or an agent's shell, print a link and a short code to use from any device instead. Over SSH and in CI this happens by itself. ```bash klio login --browserless ``` In CI, set `KLIO_TOKEN` to a key from Keys in the console instead of signing in. Your sign-in is stored in `~/.klio/config.ini`. `klio doctor` checks your connection and credentials. ## Run your first function Save this as app.py, then run python app.py: ```python from klio_compute import function, Image @function(image=Image(python_version="python3.12"), cpu=1, memory="1Gi") def hello(): return "Hello from Klio Compute" if __name__ == "__main__": print(hello.remote()) ``` `.remote()` runs the function in a container in Klio Compute and returns its result. The container stops when the call is done, and you pay for the seconds it ran. ## Keep it running ```bash klio deploy app.py:hello ``` This deploys `hello` as an app you can see, call and stop in the console. Next: [run it on a GPU](https://klio.tech/docs/gpus.md), [put it behind an address](https://klio.tech/docs/endpoints.md) or [deploy an open model](https://klio.tech/docs/models.md). ## Free credit A new workspace gets $5 of free credit, which expires after 30 days. Until a card is on file you can use $1 of it; then add a card in the console to keep going. See [pricing](https://klio.tech/docs/pricing.md). # GPUs > Run a function on a GPU, see live GPU prices, fall back between GPU types, and use 2, 4 or 8 GPUs in one container. Canonical page: https://klio.tech/docs/gpus ## Run a function on a GPU Name the GPU type in `gpu=`. The container starts on that GPU, runs the call and stops. ```python from klio_compute import function, Image image = Image(python_version="python3.12", python_packages=["torch"]) @function(gpu="L40S", image=image) def device_name() -> str: import torch return torch.cuda.get_device_name(0) if __name__ == "__main__": print(device_name.remote()) ``` Before a GPU workload starts (`klio run`, `klio deploy`, `.remote()`, `klio model deploy`), the command line prints what that GPU costs right now, once per GPU. `klio run` and `klio deploy` also take `--gpu`, for example `--gpu L40S`. ## GPU types and prices ```bash klio gpus klio gpus --json ``` `klio gpus` lists every GPU type, its price per GPU-hour right now and whether it can start now. Prices follow the market and are checked every few minutes; a container keeps the price it started at for as long as it runs. The same list is on the [pricing page](https://klio.tech/docs/pricing.md). Table: The names gpu= accepts, with memory per GPU | gpu= | Memory | | --- | --- | | `A10` | 24 GB | | `A16` | 16 GB | | `A30` | 24 GB | | `A40` | 48 GB | | `A100` | 40 GB | | `A100-80` | 80 GB | | `A4000` | 16 GB | | `A5000` | 24 GB | | `A6000` | 48 GB | | `B200` | 180 GB | | `B300` | 288 GB | | `H100` | 80 GB | | `H200` | 141 GB | | `L4` | 24 GB | | `L40` | 48 GB | | `L40S` | 48 GB | | `RTX4000Ada` | 20 GB | | `RTX4090` | 24 GB | | `RTX6000` | 24 GB | | `RTX6000Ada` | 48 GB | | `RTXPro6000` | 96 GB | | `V100` | 16 GB | | `V100-32` | 32 GB | ## GPU fallbacks Give a list of GPU types and the workload uses whichever of them can start: ```python @function(gpu=["L40S", "A100-80"], image=image) def generate(prompt: str) -> str: ... ``` If none of the GPUs you asked for can start right now, the workload stops before anything starts, and nothing is charged. With a list, the price of each GPU is printed. ## Several GPUs in one container A container can have 1, 2, 4 or 8 GPUs of one type, all on the same machine, so they can talk to each other directly (for example with NCCL or tensor parallelism). ```python from klio_compute import function @function(gpu="L4", gpu_count=2) def train(): import torch return torch.cuda.device_count() # 2 ``` - Any other count stops before anything starts, and nothing is charged. - Several GPUs need a paid workspace. On the free tier a container runs one GPU at a time. - Each GPU count has its own price per GPU-hour, paid for each of the container's GPUs. `klio gpus` lists the counts that can start right now. - With the Modal-compatible API, write `gpu="L4:2"`. # Endpoints > Deploy a function as an HTTPS endpoint that scales to zero when idle and is called with your key. Canonical page: https://klio.tech/docs/endpoints ## Deploy an endpoint ```python from klio_compute import endpoint, Image @endpoint(gpu="L4", image=Image(python_version="python3.12"), keep_warm_seconds=60) def predict(**inputs): return {"echo": inputs} ``` ```bash klio deploy app.py:predict ``` The deploy prints the endpoint's address. Requests need a key: send one from Keys in the console as `Authorization: Bearer`. ## Call it ```bash curl -X POST https:// \ -H "Authorization: Bearer $KLIO_TOKEN" \ -H "Content-Type: application/json" \ -d '{"text": "hello"}' ``` `klio curl` sends the same request with the key you signed in with, and only ever to an endpoint's own address. ## Scale to zero An endpoint's containers stop after `keep_warm_seconds` without requests, and nothing is billed while none run. The next request starts one again, which takes longer than a warm request. Raise `keep_warm_seconds` to keep it ready for longer. ## Manage deployments ```bash klio deployment list klio deployment stop klio app list klio app logs ``` Every deployment is also in the console, with its logs, usage and cost. # Task queues > Queue work for a function that runs it in the background, one task at a time per container. Canonical page: https://klio.tech/docs/task-queues ## Queue work ```python from klio_compute import task_queue, Image @task_queue(image=Image(python_version="python3.12"), gpu="L4") def transcribe(path: str): ... if __name__ == "__main__": task = transcribe.put("clips/intro.wav") print(task.id) ``` `.put()` adds a task with the arguments you pass and returns it straight away; a container picks it up and runs it. `klio deploy app.py:transcribe` keeps the queue deployed. ```bash klio task list klio task stop ``` # Schedules > Run a function on a cron schedule in UTC. Canonical page: https://klio.tech/docs/schedules ## Run on a schedule ```python from klio_compute import schedule @schedule(when="0 * * * *") def hourly_report(): ... ``` ```bash klio deploy app.py:hourly_report ``` `when` is a cron expression in UTC, to the minute. `@hourly`, `@daily`, `@weekly`, `@monthly` and `@yearly` work too. The schedule runs once it is deployed, and each run is billed like any other container. With the Modal-compatible API, `modal.Cron` and `modal.Period` work. A `Cron` in another time zone starts a short container often enough to check its local clock, and that time is billed. # Sandboxes > Start an isolated container, run commands or code in it, read and write its files, then terminate it. Canonical page: https://klio.tech/docs/sandboxes ## Create a sandbox A sandbox is a container you drive from your code: useful for running code an LLM wrote or a player uploaded, apart from everything else. Every customer's work runs sandboxed, apart from other customers and from the machine. ```python from klio_compute import Sandbox, Image sandbox = Sandbox(cpu=1, memory="1Gi", image=Image(python_version="python3.12")).create() try: result = sandbox.process.run_code("print(2 + 2)") print(result.result) finally: sandbox.terminate() ``` `sandbox.process.exec(...)` runs a command, and `sandbox.fs` uploads and downloads files. A sandbox can have a GPU with `gpu=`. You pay for the seconds it runs; `terminate()` stops it. ## With the Modal-compatible API ```python import klio_compute.compat as modal sandbox = modal.Sandbox.create(image=modal.Image.debian_slim(), timeout=300) process = sandbox.exec("bash", "-c", "echo hello") print(process.stdout.read()) sandbox.terminate() ``` `Sandbox.create`, `exec`, `filesystem.write_text`, `filesystem.read_text` and `terminate` work. Options that can't be honored stop with a clear message. # Volumes > Keep files such as model weights in a named volume that every container of your workspace can mount. Canonical page: https://klio.tech/docs/volumes ## Mount a volume ```python from klio_compute import function, Volume weights = Volume(name="weights", mount_path="./weights") @function(volumes=[weights], gpu="L40S") def load(): with open("./weights/model.bin", "rb") as f: return len(f.read()) ``` A volume is created the first time it is used and keeps its files between runs. ## Move files from the command line ```bash klio volume create weights klio volume put weights ./model.bin klio volume ls weights klio volume get weights model.bin ./model.bin klio volume list ``` `klio volume cp` copies within a volume, and `klio volume delete` removes one. With the Modal-compatible API, `Volume.reload()` right after a commit in another container waits until the data is readable there, which can take a moment. # Secrets > Store API keys and passwords once, and give them to the functions that need them as environment variables. Canonical page: https://klio.tech/docs/secrets ## Create a secret ```bash klio secret create HF_TOKEN hf_your_token klio secret list ``` `NAME VALUE` stores one value. `KEY=VALUE` pairs make a group, as Modal does, and `--from-dotenv .env` reads them from a file; `--force` replaces a group that exists. `klio secret delete` removes one. ## Use it in a function ```python import os from klio_compute import function @function(secrets=["HF_TOKEN"]) def download(): token = os.environ["HF_TOKEN"] ... ``` Each secret named in `secrets=` is set as an environment variable in the container. With the Modal-compatible API, `modal.Secret.from_name("group")` gives a function every key of a group. # Images > Choose the container image a function runs in: Python packages, shell commands, or your own base image. Canonical page: https://klio.tech/docs/images ## Build an image ```python from klio_compute import function, Image image = Image( python_version="python3.12", python_packages=["transformers", "torch"], commands=["apt-get update && apt-get install -y ffmpeg"], ) @function(image=image, gpu="L4") def run(): ... ``` Images are built once and reused. Klio Compute runs Python 3.12: `Image(python_version="python3.12")`, and plain `Image()`, is the supported image, and anything built on it can import `klio_compute` with nothing to install. Asking for another version stops with a message before anything is sent. ## Your own base image ```python image = Image(base_image="docker.io/library/python:3.11-slim") ``` An image with its own `base_image` may use any Python. `Image` from `klio_compute` adds one build step that installs this release of `klio-compute` without its dependencies, so `from klio_compute import function` works inside it too. Set `KLIO_SDK_INSTALL_SPEC` to install it from somewhere else, such as a private mirror. - Use `Image` from `klio_compute`, not from another package. - Don't list `klio-compute` in `python_packages`: its dependencies would conflict with the versions the platform fixes in images. - `klio run` and `klio deploy` also take `--dockerfile` to build from a Dockerfile. # Deploy an OpenAI-compatible model > Deploy an open model with one command and call it with any OpenAI client and your Klio key. Canonical page: https://klio.tech/docs/models ## Deploy a model ```bash klio model list klio model deploy qwen3-14b ``` `klio model list` shows the models you can deploy. `klio model deploy qwen3-14b` deploys it on an L40S behind an OpenAI-compatible API on your own address, shows progress while it starts, and prints the base URL and how to call it. ## Call it Use the printed base URL with any OpenAI client and a key from Keys in the console. Requests without a key are refused. ```python import os from openai import OpenAI client = OpenAI(base_url="https:///v1", api_key=os.environ["KLIO_TOKEN"]) reply = client.chat.completions.create( model="qwen3-14b", messages=[{"role": "user", "content": "Hello"}], ) print(reply.choices[0].message.content) ``` ```bash curl https:///v1/chat/completions \ -H "Authorization: Bearer $KLIO_TOKEN" \ -H "Content-Type: application/json" \ -d '{"model": "qwen3-14b", "messages": [{"role": "user", "content": "Hello"}]}' ``` ## Starting, stopping and cost - From no machine, the first start has taken about 12 minutes, and that can vary. `--no-wait` returns once it is deployed; `--timeout` sets how many minutes to wait. - It stops by itself after 5 minutes without requests, and `--keep-warm` sets how many seconds it stays up after the last one. You pay while it runs. - The next request starts it again and waits while it loads. - `klio deployment stop ` removes it; the deploy prints the command with its id. # Coming from Modal > Run a Modal project on Klio Compute by changing its import line; klio migrate --check reports what works first. Canonical page: https://klio.tech/docs/migrate-from-modal ## Change one line ```python import klio_compute.compat as modal # was: import modal ``` The rest of your code stays as it is. The same file can move back by restoring the import. Modal is a trademark of its owner; Klio Compute is not affiliated with it. ## Check first ```bash klio login klio migrate --check . klio migrate --write . klio run app.py klio run app.py::fn klio deploy app.py ``` - `klio migrate --check` reports what works, what warns and what is refused, and changes nothing. `--json` prints the report as JSON. - `klio migrate --write` changes the import lines, and nothing else. - `klio run app.py` runs the local entrypoint, or the only function. `klio run app.py::fn` runs a named function; arguments after it are passed to the function by name, from its type hints. - `klio deploy app.py` deploys every function and class, each as `-`. ## What works the same Functions and classes reuse warm containers, `.map()` keeps input order, and exceptions come back as the type your function raised. Web functions are public unless they say `requires_proxy_auth=True`; callers then send a Klio key as `Authorization: Bearer`. Anything that can't be honored stops with a clear message; nothing is silently dropped. ## What differs - `@modal.clustered`, `@modal.batched`, `@modal.web_server`, `@modal.wsgi_app`, `.remote_gen()`, `CloudBucketMount`, `region=` and `cloud=` aren't available, and say so when used. - `Image.run_function` runs your function during the build, without a GPU, volumes or secrets. - A `Cron` in a time zone starts a short container often enough to check its local clock, and that time is billed. - A `Dict` entry lasts a week from when it was written, and a `Queue` a day from its last write. - What a function prints appears in your terminal when each call finishes, not line by line. - `gpu=` takes the types in `klio gpus`, with up to 8 of one type, such as `gpu="H100:8"`. For more, write to support@klio.tech. - `klio secret create NAME K=V K2=V2 --from-dotenv .env --force` makes a group, and `klio volume put/get/ls/rm/cp` work on volumes. - `klio app list`, `klio app logs`, `klio app stop`, `klio curl`, `klio dashboard` and `klio shell` are there too. # Pricing and billing > Prepaid credit, billed per second; GPU prices follow the market and a running container keeps the price it started at. Canonical page: https://klio.tech/docs/pricing ## How billing works - You pay from prepaid credit. Top up in the console, by hand or automatically; when the balance reaches zero, work stops instead of running up an invoice. - Running time is metered per second, for the GPU, CPU and memory a container uses. Nothing is billed while an app is scaled to zero. - GPU prices follow the market and are updated about every 5 minutes. A container pays the price in effect when it starts for as long as it runs; a price change only affects containers that start after it. - Several GPUs in one container have their own price per GPU-hour, paid for each GPU. `klio gpus` shows every price right now. ## GPU prices right now Prices checked Nov 8, 09:00 UTC. GPU prices follow the market and are checked every few minutes. A running container keeps the price it started at. Table: Price per GPU-hour right now, cheapest first | GPU | Memory | Price per GPU-hour | | --- | --- | --- | | A6000 | 48 GB | $0.75/h | | RTX4090 | 24 GB | $0.90/h | | A4000 | 16 GB | $1.20/h | | RTX4000Ada | 20 GB | $1.20/h | | L40 | 48 GB | $1.35/h | | L4 | 24 GB | $1.45/h | | RTX6000Ada | 48 GB | $1.50/h | | A10 | 24 GB | $1.95/h | | A100 | 40 GB | $2.10/h | | A5000 | 24 GB | $2.15/h | | A100-80 | 80 GB | $2.49/h | | A40 | 48 GB | $2.79/h | | V100 | 16 GB | $3.55/h | | V100-32 | 32 GB | $3.55/h | | H200 | 141 GB | $6.95/h | | A16 | 16 GB | Not available right now | | A30 | 24 GB | Not available right now | | B200 | 180 GB | Not available right now | | B300 | 288 GB | Not available right now | | H100 | 80 GB | Not available right now | | L40S | 48 GB | Not available right now | | RTX6000 | 24 GB | Not available right now | | RTXPro6000 | 96 GB | Not available right now | CPU is $0.04 per core-hour and memory $0.005 per GB-hour, both billed per second on top of the GPU. ## Free credit and cards - A new workspace gets $5 of free credit, once per verified mailbox. It expires 30 days after it is granted, and is spent before paid credit. - Until a card is on file, $1 of free credit can be used; after that, new work doesn't start until you add a card. The rest of your free credit stays yours. - Adding a card charges $0.50 to check it, and that amount is added to your balance as paid credit. ## Purchases - A single purchase is between $10 and $1,000. - Each workspace has a monthly limit on credit purchases, $500 unless you change it. Months run on UTC. - The full terms are in the [Terms of Service](https://klio.tech/terms). # Limits > What Klio Compute supports today: Python versions, GPUs per container, defaults and what isn't available. Canonical page: https://klio.tech/docs/limits ## Python - The SDK installs on Python 3.10 to 3.13. - Containers run Python 3.12. An image built on your own base image may use any Python; see [images](https://klio.tech/docs/images.md). ## GPUs - 1, 2, 4 or 8 GPUs of one type per container, on one machine. More than one needs a paid workspace. - A GPU type can be unavailable for a while. `klio gpus` shows what can start now, and a list in `gpu=` falls back to another type. ## Defaults you can change Table: SDK defaults | Setting | Default | | --- | --- | | `@function(timeout=)` | 3600 seconds | | `@endpoint(timeout=)` | 180 seconds | | `@endpoint(keep_warm_seconds=)` | 180 seconds | | `klio model deploy --keep-warm` | 300 seconds | ## Not available - Machines, pools and workers are managed for you, so there are no commands for them. - From the Modal API: `@modal.clustered`, `@modal.batched`, `@modal.web_server`, `@modal.wsgi_app`, `.remote_gen()`, `CloudBucketMount`, `region=` and `cloud=`. See [coming from Modal](https://klio.tech/docs/migrate-from-modal.md). ## Billing limits Purchases are between $10 and $1,000 each, up to a monthly limit of $500 unless you change it. See [pricing](https://klio.tech/docs/pricing.md). # Command line > The klio commands for running, deploying and managing work, and the options agents and scripts need. Canonical page: https://klio.tech/docs/cli ## Overview Installing `klio-compute` also installs the `klio` command. `klio --help` lists every command, and `--help` after a command explains it. Table: The commands these docs use | Command | What it does | | --- | --- | | `klio login` | Sign in, or create an account. `--browserless` prints a link and a code instead of opening a browser. | | `klio run app.py` | Run a file's local entrypoint, or its only function. | | `klio deploy app.py` | Deploy every function and class in a file's App, or one handler as `app.py:handler`. | | `klio gpus` | Every GPU type, its price per hour right now, and whether it can start. Add `--json` for JSON. | | `klio model deploy` | Deploy an open model behind an OpenAI-compatible API. `klio model list` shows the models. | | `klio migrate --check` | Check a Modal project before changing anything. `--write` changes its import lines. | | `klio app list` | List running apps. `info`, `logs` and `stop` work on one app. | | `klio deployment list` | List deployments. `klio deployment stop` stops one by its id. | | `klio task list` | List tasks. `klio task stop` stops one. | | `klio volume list` | List volumes. `create`, `put`, `get`, `ls`, `cp` and `delete` work on them. | | `klio secret list` | List secrets. `create`, `show` and `delete` work on them. | | `klio curl` | Call a private endpoint with your key. | | `klio logs` | Read or follow workload logs. | | `klio doctor` | Check your connection and credentials. | | `klio dashboard` | Open the console. | ## For agents and scripts - `--json` gives machine-readable output, and errors come with a `code`. It implies `--no-input`. - `--no-input` turns off prompts. - `klio login --browserless` signs in from a shell without a browser; `KLIO_TOKEN` skips sign-in entirely. - Several workspaces: `klio login --name NAME` saves a sign-in under its own context, and `--context NAME` picks it.