GPU Cloud
GPU Cloud provisions managed GPU instances for model serving, notebooks, ComfyUI, Web Terminal, Ollama, or your own CUDA image. Billing is metered by the second against your credit balance. Connect to running workloads through authenticated EmpirioLabs API paths.
You can manage instances from the GPU Cloud page in the dashboard or through the API documented here.
How it works
- Pick a GPU from the catalog. Each row shows VRAM, hourly pricing, current availability, and the machine sizes offered for that GPU. Availability counts are conservative, so a GPU is at least as available as the number shown.
- Choose a workload: a curated model, a Hugging Face model id, a template, or a custom CUDA Docker image.
- Deploy the instance. Your dashboard settings show the current GPU limit for your account.
- Wait for readiness. New instances move through
provisioning, thenloading, thenrunning. - Connect through the API with your EmpirioLabs API key, or open a shell with SSH.
- Stop or destroy when you are done. If you selected a volume, files in
/workspaceare saved before the GPU is released. A stopped instance keeps its deploy configuration for the next session. Destroyed instances are permanently removed; their volumes remain available.
Persistent storage comes in two kinds. A workspace volume saves files under /workspace when you stop and restores them on the next attach. An object bucket is S3-compatible storage mounted at /mnt/<name> and read and written in place, with no save or restore step, which suits a dataset larger than the GPU’s local disk. Other runtime files are temporary: without one of these, stopping or destroying the instance loses its files. Save and flush application checkpoints before stopping. A volume saves a directory; it does not restore running processes, installed system packages, or GPU memory.
Pricing and limits
- GPU runtime is billed by the second.
- Each machine size is priced independently and listed in
gpu_boxes, so a multi-GPU box is not always the single-GPU price multiplied by the GPU count. Readgpu_boxes[].price_hourly, which is already the total for the whole box. - A GPU sold only as a multi-GPU node shows that node’s total in the row’s
price_hourly. - Billing starts when an instance is allocated to you, which includes the time it takes to start up.
- GPU runtime billing stops when the allocation has been released, after any volume save finishes.
- Deploying and starting an instance require enough credit balance for the initial running window.
- Running instances are stopped automatically when the balance threshold is no longer sufficient. A failed volume save keeps the instance available for recovery and requires attention.
- GPU Cloud defaults are 10 concurrent GPUs per account. Limits are account-scoped; your dashboard settings show your effective limit.
Runtime disk is included in the GPU hourly price. disk_gb requests a size from 100 GB to 300 GB, and defaults to 150 GB. A value outside that range is set to the nearest limit. The same range is published as runtime_disk_policy on GET /v1/gpu/catalog. The runtime storage a box actually includes is reported in gpu_boxes[].runtime_specs, so check it when your workload needs a specific amount.
Runtime disk is erased when the instance is destroyed. Attach a workspace volume to preserve files under /workspace, or an object bucket to read and write /mnt/<name> directly without copying the data onto the instance.
Persistent-volume storage costs $0.18 per GB-month, billed per second for as long as the volume exists, including while no GPU is running. Storage is billed separately from GPU runtime.
An account starts with 10 persistent volumes and 10,000 GB of total volume capacity, and a single volume holds up to 4,000 GB.
Browse the catalog
The catalog returns customer-safe GPU metadata, pricing, and current availability.
Poll the catalog and cluster endpoints to refresh prices and availability. If either
returns 503, keep your last successful result and retry later.
Machine sizes
gpu_counts lists the machine sizes actually offered for a GPU. A multi-GPU box is one machine holding that many GPUs, not several separate instances, so the GPUs share a host and can be used together by a single workload. Deploy a specific size by setting num_gpus to one of the listed values; any other value is refused with 422 invalid_gpu_count.
A model deployment on a multi-GPU box is served with tensor parallelism across every GPU in the box, so a model too large for one card fits across the combined VRAM shown in gpu_boxes[].vram_gb.
Clusters
A cluster combines multiple GPU nodes for distributed training. Reserve the group in one order and choose how many nodes to use.
Check ordering_enabled in the cluster-list response before ordering.
updated_at reports the Unix timestamp of the latest inventory refresh.
nodes is how many nodes are free on that fabric, and you can order from 2 up to that number. price_hourly is the total for all of them, not per node and not per GPU: divide it by nodes for the per-node rate, and multiply by the count you want. total_vram_gb is the combined VRAM across every GPU in the whole offer, which is the figure that decides whether a model fits at all.
interconnect_measured says whether interconnect_gbps is a speed the fabric was measured at or a nameplate figure. Treat a nameplate number as an upper bound when you size a job.
Some clusters report interconnect as High-speed network with a null interconnect_gbps. The interconnect is high-speed and no rate is quoted. Treat a null rate as unspecified, not slow.
Cluster availability changes in large steps. An empty list means none are free right now; poll again later.
Deploying a cluster
A cluster is ordered as one unit and billed as one unit. Send nodes to take
part of a fabric; leave it out to take all of it. Every node is an
ordinary GPU instance afterwards: it appears in your instance list, it is
reachable over SSH with its own instance id, and you stop or destroy it like any
other.
Each node receives the peer list, its own rank and the rendezvous address at
/etc/empirio/cluster.env, using the conventional MASTER_ADDR, MASTER_PORT,
WORLD_SIZE and NODE_RANK names, so an unmodified torchrun script works.
Source that file and launch as you normally would:
Attach an object bucket at /mnt/<name> or a network volume at /workspace
to share files across the cluster. Workspace volumes support a single instance.
Deploy an instance
Deploying returns the instance and its deployment key and state. Poll GET /v1/gpu/instances/{instance_id} until status is running. If the instance reports error, check its error message before starting another deployment.
With a volume, startup waits for its saved directory to restore. If a volume cannot restore or save, inspect its status and error before retrying. A failed save keeps the last saved data intact and prevents the GPU from being released.
Retry or recover a deployment
Deployment key. Choose a unique order_key in the request body or send
Idempotency-Key as a header. Keys contain 8 to 128 letters, numbers, dots,
colons, underscores or hyphens and start with a letter or number. If you send
both, their values must match exactly. Without a key, each POST starts a new
deployment.
Recovery. Keep the key if the request times out or the response cannot be read. Check the deployment from the same account:
With order_key, this endpoint returns instance, order_key and
order_state instead of the instance list. A key from another account returns
404. A key continues to identify its original deployment after the instance
stops or is destroyed.
Retries. Reuse the same key and request settings. Do not generate a new key after a timeout. Changes to the GPU count, image, environment, volume, name or other settings require a new deployment after checking the original one.
Deploy a model
Pass a curated template_slug or paste any Hugging Face repo id. Model deployments are served from an OpenAI-compatible /v1 endpoint on the instance.
For gated repos, pass the token in env:
Deploy a template
Templates are ready-to-run environments. Available templates include PyTorch + JupyterLab, ComfyUI, Web Terminal, and Ollama.
Deploy a custom Docker image
Run your own CUDA image. CPU-only images may fail because the runtime expects a GPU-compatible container.
Persistent volumes
Create a volume, select it when deploying, and write the files you want to keep under /workspace. This is the default kind, workspace; see Object buckets for storage that is mounted and read in place instead. The same volume can be used with any GPU type that has enough runtime storage. A workspace volume attaches to one instance at a time. For live sharing, see Network volumes. Stop the attached instance before selecting it for a different deployment.
Workspace and network volumes are billed for their configured size at the returned price_gb_month, prorated per second using a 730-hour month. effective_price_gb_month includes your current account discount. Storage billing continues while a volume exists, including when it is empty or no GPU is attached, and ends when the volume is deleted. GPU runtime disk remains included in the GPU price.
New volume names use 2 to 32 lowercase letters, numbers and hyphens. Start and end with a letter or number. Uppercase letters are converted to lowercase. Existing names are preserved.
New network volumes allow up to 8 simultaneous instance attachments. Each volume reports its effective limit as max_attachments. New network volumes include 10,000 files and directories per GB of volume capacity. Each volume reports its total entry limit as inode_limit and current usage as inodes_used.
Create and attach
Use the returned volume.id in your deploy request:
Choose a volume from 1 to 4,000 GB and a GPU with room for the volume plus the workload.
Archive framing counts toward bytes_used. Regular files, directories and relative symlinks within /workspace are supported. Device files and links outside that directory are not supported.
Choosing a volume kind
Choose the storage behavior your workload needs.
The volumes API returns the current storage rate, limits and availability.
Object buckets and network volumes are available with compatible GPU configurations. The deployment selector updates for the GPU size or cluster you choose. An unavailable volume stays visible with a message to help you select a compatible configuration.
Object buckets
Pass "kind":"object" to create an S3-compatible bucket instead of a workspace
volume. It is mounted at /mnt/<name> on every instance you attach it to, and
read and written in place, so nothing is copied onto the GPU at start or off it
at stop. That makes it the right choice for a training set larger than the
instance’s local disk, and for data you want to reach from several instances in
turn.
All storage types use the same naming rules. Invalid names return invalid_volume_name.
Object buckets bill at least the size selected. Usage above that size is rounded
up to whole GB and billed at the returned price_gb_month. billable_gb reports
the current quantity used for storage billing. They count against the same
per-account volume and capacity limits.
GET /v1/gpu/volumes returns kind on every volume, and a capabilities
object whose object_storage field reports whether object buckets can be
created right now. A request for "kind":"object" while that is false is
refused with object_storage_unavailable rather than creating a volume that
could not be mounted.
Use capabilities.object_gpu_boxes[gpu_slug] to choose a compatible num_gpus
value, or capabilities.object_cluster_ids to choose a compatible cluster.
capabilities.object_gpu_slugs lists GPU types with compatible sizes. An object
bucket can be larger than the instance’s local disk.
Network volumes
Create a shared filesystem with "kind":"network":
Pass the returned volume.id as volume_id in each instance or cluster deployment.
Compatible instances see the same files under /workspace. File locks coordinate
concurrent access; applications writing the same file should use them.
Stopping one instance leaves the volume available to its other attachments.
GET /v1/gpu/volumes returns capabilities.network_storage and
capabilities.network_gpu_slugs, and capabilities.network_cluster_ids for current availability. Each volume reports
attachment_count, max_attachments, inode_limit, and inodes_used.
The inode limit covers files and directories. A creating volume cannot attach
and does not accrue storage charges until it becomes available.
The list response reports current defaults in policy.network_max_attachments
and policy.network_inodes_per_gb. Each volume retains its own effective limits.
Use capabilities.network_gpu_boxes[gpu_slug] to choose a compatible num_gpus
value. Compatibility can differ between machine sizes of the same GPU type.
The dashboard uses the same create and deploy controls. A network volume remains selectable while it has available attachments. Stop all attached instances before deleting it. Existing workspace volumes keep their single-instance behavior.
The response also includes policy, with naming rules, the per-volume capacity ceiling, and the default attachment allowance for new network volumes.
List, inspect and delete
For workspace volumes, last_synced_at records the most recent successful save. After stopping, wait until the volume is available before attaching it elsewhere. An error leaves the last saved directory intact. If a save failed, correct the problem, such as a full volume, then retry stopping the instance. GPU runtime charges continue until the GPU is released. If the machine is lost unexpectedly, files written since the last successful save cannot be recovered from the volume.
Deleting an unattached volume permanently removes its saved files and ends its storage billing. A 202 response with deleted: false means deletion is in progress. Poll GET /v1/gpu/volumes/{volume_id} until it returns 404. Deleting an attached volume returns 409. Destroying a GPU does not delete its volume.
Training and fine-tuning
Choose a training preset and attach a persistent volume before deploying. All three presets require a volume; a deploy request without volume_id is refused with volume_required. Deploy in template mode using the preset’s slug. Once the instance is running, open JupyterLab from its dashboard connect link, using the same authenticated connect path as other notebook templates. Training starts only when you run the starter command in a JupyterLab terminal.
Prepare data and start training
All three starters read /workspace/dataset.jsonl. Axolotl expects Alpaca-format JSON objects with instruction, input, and output fields. Unsloth and TRL + PEFT expect one JSON object per line with a text field. Keep your dataset, configs, scripts, and notebooks under /workspace so they are saved with the volume.
Edit the starter for your model and training settings, and choose GPU memory capacity that fits them. For Axolotl, switch to LoRA with adapter: lora and load_in_4bit: false; for full fine-tuning, remove adapter and set load_in_4bit: false. For Unsloth, set load_in_4bit=False in train.py to use LoRA without 4-bit loading. For TRL + PEFT, remove peft_config from the trainer to use full fine-tuning.
Run the command for your chosen preset:
Save and resume
Checkpoints are written under /workspace/checkpoints/<slug>: /workspace/checkpoints/axolotl, /workspace/checkpoints/unsloth, or /workspace/checkpoints/trl-peft. The Unsloth and TRL + PEFT starters also save the final model and tokenizer in a final subdirectory.
Save and flush your training checkpoint before stopping or destroying the instance. Workspace volumes save /workspace before release. Network volumes retain shared files and flush this instance before release. Start the stopped instance again, or attach the same volume to a new deployment of the same preset, to restore your dataset, starter files, and checkpoints. Existing starter files are preserved. Restoring files does not restart training automatically.
For Axolotl, add the saved checkpoint path to /workspace/training/axolotl/config.yml, then rerun the Axolotl training command:
For Unsloth or TRL + PEFT, set RESUME_FROM_CHECKPOINT when running the corresponding script:
Replace checkpoint-N with an existing checkpoint directory from your run. Files outside /workspace remain temporary, and volume storage billing continues until you delete the volume.
Manage lifecycle
Use refresh to re-sync status, stop to save the selected volume and release the allocation, and start to redeploy the saved instance spec and restore its volume.
Destroying an instance saves its selected volume before releasing the GPU and ending runtime billing. Poll until the instance is destroyed. The GPU cannot be restored; its volume can be selected for another deployment.
Statuses
Connect to a running instance
Use the connect endpoint with your EmpirioLabs API key. It supports GET, POST, PUT, PATCH, and DELETE. A JSON request with "stream": true, such as an OpenAI-compatible chat completion, streams its response. Other responses are returned whole and must complete within 120 seconds.
For a model deployment, call the OpenAI-compatible endpoint on the instance:
For JupyterLab, ComfyUI, Web Terminal, or Ollama, open the instance connect URL from the dashboard or send requests to the relevant connect path.
Chat with your model in the dashboard
When you deploy a model (or any instance that serves an OpenAI-compatible API), the dashboard gives you a built-in chat page so you can try the model right away without writing any code. Open the instance from the GPU Cloud page and choose Chat with this model. The chat page streams responses, supports a system prompt and the usual sampling controls (temperature, top-p, max tokens), and lets you attach images or audio for multimodal models. It runs against the same authenticated connect path as the API, so there is no extra setup and no separate billing: the instance is already metered by the second.
Run a command
Run a command inside a running instance’s workload container, as root, and get its output back. This needs only your API key, so a script or an agent can operate the GPU without an SSH client.
Each call starts a fresh shell, so chain steps with && or run a script. Stdout
and stderr are returned up to 1 MiB each. A command stops after
timeout_seconds, up to 300. For longer work, start it in the background, for
example nohup python train.py > train.log 2>&1 &, and read the log with a later
call. An instance whose workload does not accept commands returns 409. The MCP
server offers the same as the run_gpu_command tool.
SSH and shell access
SSH works on GPU instances while they are running, with any standard OpenSSH
client. A session opens a root shell inside your workload, with your files,
processes and GPUs. Each node of a cluster is an ordinary instance, so you
connect to any node with its own instance ID.
Set up once. On the GPU Cloud page, open SSH keys:
- Create one for me. We create a key and give you one setup command for
macOS, Linux or Windows. Paste it into a terminal once. It saves the key as
~/.ssh/empiriolabs_ed25519and adds the entries below to~/.ssh/config. - Or add a key you already have. Paste the contents of
~/.ssh/id_ed25519.pub. To create one yourself, runssh-keygen -t ed25519and press Enter at each prompt.
Keys can also be added with POST /v1/gpu/ssh-keys:
Connect. Copy the SSH command from the instance card, or read ssh.command
from GET /v1/gpu/instances/{instance_id}:
After the one-time setup, ssh gpu-<instance_id> works too. The first
connection to an instance can take a minute. The first time, your client asks
you to confirm two host keys: ssh.empiriolabs.ai and the instance. Ed25519 and
RSA keys (2048 to 8192 bits) are supported. The image needs Debian, Ubuntu, or
Alpine with a working package manager, unless it already includes an SSH server.
An instance that does not offer SSH returns ssh: null and shows no SSH command.
Copy files. scp and sftp use the same connection, and so does rsync
when the image includes it. Copy into any path in the container, such as
/root; /workspace is present when a volume is attached or the image creates
it:
For large datasets, download them directly on the instance or keep them in a volume.
The setup, and editors. The one-time setup adds these entries to
~/.ssh/config. Editors that read that file, such as VS Code and Cursor,
connect to gpu-<instance_id> with them. With a key at a default name, such as
~/.ssh/id_ed25519, keep only the first three lines:
Create a key through the API. For automation, send generate: true instead
of a public key. The response carries the private key once, as private_key. It
is not stored and cannot be shown again. Save it as
~/.ssh/empiriolabs_ed25519; with the entries above, ssh gpu-<instance_id>
connects. Without them, give the key to both hosts:
Manage keys. GET /v1/gpu/ssh-keys lists your keys and
DELETE /v1/gpu/ssh-keys/{key_id} removes one. A removed key can no longer open
new connections; sessions already open stay open until they end. To prepare SSH
on one instance before your first connection, send your public key to
POST /v1/gpu/instances/{instance_id}/ssh, which also adds the key to your account.
Limits. Agent, X11, and port forwarding are disabled. SSH adds no separate
charge. Stopping or replacing an instance closes its SSH sessions; closing a
session does not stop the GPU. After you restart or replace an instance, remove
its old host key with ssh-keygen -R gpu-<instance_id> before reconnecting. Files
outside an attached volume are temporary. For a shell in the browser, deploy the
Web Terminal template.
Usage and billing records
The GPU Cloud dashboard shows running spend and lifetime GPU spend. API lifecycle responses include the instance price, GPU count, billing status, and billed amount so you can reconcile usage from your own systems. Billing starts when the instance is allocated, which is before it reports running, so use billed_amount as the total for an instance.