> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.empiriolabs.ai/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.empiriolabs.ai/_mcp/server.

# GPU Cloud

GPU Cloud provisions managed GPU instances for model serving, notebooks, ComfyUI, Web Terminal, Ollama, or your own CUDA image. Billing is metered by the second against your credit balance. Connect to running workloads through authenticated EmpirioLabs API paths.

You can manage instances from the [**GPU Cloud** page in the dashboard](https://platform.empiriolabs.ai/dashboard/gpu-cloud) or through the API documented here.

## How it works

1. **Pick a GPU** from the catalog. Each row shows VRAM, hourly pricing, current availability, and the machine sizes offered for that GPU. Availability counts are conservative, so a GPU is at least as available as the number shown.
2. **Choose a workload**: a curated model, a Hugging Face model id, a template, or a custom CUDA Docker image.
3. **Deploy the instance**. Your dashboard settings show the current GPU limit for your account.
4. **Wait for readiness**. New instances move through `provisioning`, then `loading`, then `running`.
5. **Connect** through the API with your EmpirioLabs API key, or open a shell with [SSH](#ssh-and-shell-access).
6. **Stop or destroy** when you are done. If you selected a volume, files in `/workspace` are saved before the GPU is released. A stopped instance keeps its deploy configuration for the next session. Destroyed instances are permanently removed; their volumes remain available.

> **Warning**
>
> Persistent storage comes in two kinds. A **workspace volume** saves files under `/workspace` when you stop and restores them on the next attach. An **object bucket** is S3-compatible storage mounted at `/mnt/<name>` and read and written in place, with no save or restore step, which suits a dataset larger than the GPU's local disk. Other runtime files are temporary: without one of these, stopping or destroying the instance loses its files. Save and flush application checkpoints before stopping. A volume saves a directory; it does not restore running processes, installed system packages, or GPU memory.

## Pricing and limits

* GPU runtime is billed by the second.
* Each machine size is priced independently and listed in `gpu_boxes`, so a multi-GPU box is not always the single-GPU price multiplied by the GPU count. Read `gpu_boxes[].price_hourly`, which is already the total for the whole box.
* A GPU sold only as a multi-GPU node shows that node's total in the row's `price_hourly`.
* Billing starts when an instance is allocated to you, which includes the time it takes to start up.
* GPU runtime billing stops when the allocation has been released, after any volume save finishes.
* Deploying and starting an instance require enough credit balance for the initial running window.
* Running instances are stopped automatically when the balance threshold is no longer sufficient. A failed volume save keeps the instance available for recovery and requires attention.
* GPU Cloud defaults are 10 concurrent GPUs per account. Limits are account-scoped; your dashboard settings show your effective limit.

Runtime disk is included in the GPU hourly price. `disk_gb` requests a size from 100 GB to 300 GB, and defaults to 150 GB. A value outside that range is set to the nearest limit. The same range is published as `runtime_disk_policy` on `GET /v1/gpu/catalog`. The runtime storage a box actually includes is reported in `gpu_boxes[].runtime_specs`, so check it when your workload needs a specific amount.

Runtime disk is erased when the instance is destroyed. Attach a workspace volume to preserve files under `/workspace`, or an object bucket to read and write `/mnt/<name>` directly without copying the data onto the instance.

Persistent-volume storage costs \$0.18 per GB-month, billed per second for as long as the volume exists, including while no GPU is running. Storage is billed separately from GPU runtime.

An account starts with 10 persistent volumes and 10,000 GB of total volume capacity, and a single volume holds up to 4,000 GB.

## Browse the catalog

The catalog returns customer-safe GPU metadata, pricing, and current availability.

Poll the catalog and cluster endpoints to refresh prices and availability. If either
returns `503`, keep your last successful result and retry later.

### Request

GET [https://api.empiriolabs.ai/v1/gpu/catalog](https://api.empiriolabs.ai/v1/gpu/catalog)

```curl
curl https://api.empiriolabs.ai/v1/gpu/catalog \
     -H "Content-Type: application/json"
```

```python
import requests

url = "https://api.empiriolabs.ai/v1/gpu/catalog"

payload = {}
headers = {"Content-Type": "application/json"}

response = requests.get(url, json=payload, headers=headers)

print(response.json())
```

```javascript
const url = 'https://api.empiriolabs.ai/v1/gpu/catalog';
const options = {method: 'GET', headers: {'Content-Type': 'application/json'}, body: '{}'};

try {
  const response = await fetch(url, options);
  const data = await response.json();
  console.log(data);
} catch (error) {
  console.error(error);
}
```

```go
package main

import (
	"fmt"
	"strings"
	"net/http"
	"io"
)

func main() {

	url := "https://api.empiriolabs.ai/v1/gpu/catalog"

	payload := strings.NewReader("{}")

	req, _ := http.NewRequest("GET", url, payload)

	req.Header.Add("Content-Type", "application/json")

	res, _ := http.DefaultClient.Do(req)

	defer res.Body.Close()
	body, _ := io.ReadAll(res.Body)

	fmt.Println(res)
	fmt.Println(string(body))

}
```

```ruby
require 'uri'
require 'net/http'

url = URI("https://api.empiriolabs.ai/v1/gpu/catalog")

http = Net::HTTP.new(url.host, url.port)
http.use_ssl = true

request = Net::HTTP::Get.new(url)
request["Content-Type"] = 'application/json'
request.body = "{}"

response = http.request(request)
puts response.read_body
```

```java
import com.mashape.unirest.http.HttpResponse;
import com.mashape.unirest.http.Unirest;

HttpResponse<String> response = Unirest.get("https://api.empiriolabs.ai/v1/gpu/catalog")
  .header("Content-Type", "application/json")
  .body("{}")
  .asString();
```

```php
<?php
require_once('vendor/autoload.php');

$client = new \GuzzleHttp\Client();

$response = $client->request('GET', 'https://api.empiriolabs.ai/v1/gpu/catalog', [
  'body' => '{}',
  'headers' => [
    'Content-Type' => 'application/json',
  ],
]);

echo $response->getBody();
```

```csharp
using RestSharp;

var client = new RestClient("https://api.empiriolabs.ai/v1/gpu/catalog");
var request = new RestRequest(Method.GET);
request.AddHeader("Content-Type", "application/json");
request.AddParameter("application/json", "{}", ParameterType.RequestBody);
IRestResponse response = client.Execute(request);
```

```swift
import Foundation

let headers = ["Content-Type": "application/json"]
let parameters = [] as [String : Any]

let postData = JSONSerialization.data(withJSONObject: parameters, options: [])

let request = NSMutableURLRequest(url: NSURL(string: "https://api.empiriolabs.ai/v1/gpu/catalog")! as URL,
                                        cachePolicy: .useProtocolCachePolicy,
                                    timeoutInterval: 10.0)
request.httpMethod = "GET"
request.allHTTPHeaderFields = headers
request.httpBody = postData as Data

let session = URLSession.shared
let dataTask = session.dataTask(with: request as URLRequest, completionHandler: { (data, response, error) -> Void in
  if (error != nil) {
    print(error as Any)
  } else {
    let httpResponse = response as? HTTPURLResponse
    print(httpResponse)
  }
})

dataTask.resume()
```

```bash
curl https://api.empiriolabs.ai/v1/gpu/catalog
```

```json
{
  "object": "list",
  "data": [
    {
      "slug": "rtx-4090",
      "name": "RTX 4090",
      "vram_gb": 24,
      "price_hourly": 0.6,
      "available": true,
      "available_count": 21,
      "max_gpus": 8,
      "gpu_counts": [1, 2, 4],
      "gpu_boxes": [
        { "n": 1, "price_hourly": 0.6, "vram_gb": 24, "available_count": 21 },
        { "n": 2, "price_hourly": 1.3, "vram_gb": 48, "available_count": 6 },
        { "n": 4, "price_hourly": 2.8, "vram_gb": 96, "available_count": 2 }
      ],
      "regions": ["US", "EU"]
    }
  ]
}
```

### Machine sizes

`gpu_counts` lists the machine sizes actually offered for a GPU. A multi-GPU box is one machine holding that many GPUs, not several separate instances, so the GPUs share a host and can be used together by a single workload. Deploy a specific size by setting `num_gpus` to one of the listed values; any other value is refused with `422 invalid_gpu_count`.

```bash
curl https://api.empiriolabs.ai/v1/gpu/instances \
  -H "Authorization: Bearer $EMPIRIOLABS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "gpu_slug": "rtx-6000-ada",
    "num_gpus": 2,
    "mode": "model",
    "hf_id": "Qwen/Qwen2.5-14B-Instruct"
  }'
```

A model deployment on a multi-GPU box is served with tensor parallelism across every GPU in the box, so a model too large for one card fits across the combined VRAM shown in `gpu_boxes[].vram_gb`.

### Request

GET [https://api.empiriolabs.ai/v1/gpu/catalog/\{slug}](https://api.empiriolabs.ai/v1/gpu/catalog/\{slug})

```curl
curl https://api.empiriolabs.ai/v1/gpu/catalog/rtx-4090
```

```python
import requests

url = "https://api.empiriolabs.ai/v1/gpu/catalog/rtx-4090"

response = requests.get(url)

print(response.json())
```

```javascript
const url = 'https://api.empiriolabs.ai/v1/gpu/catalog/rtx-4090';
const options = {method: 'GET'};

try {
  const response = await fetch(url, options);
  const data = await response.json();
  console.log(data);
} catch (error) {
  console.error(error);
}
```

```go
package main

import (
	"fmt"
	"net/http"
	"io"
)

func main() {

	url := "https://api.empiriolabs.ai/v1/gpu/catalog/rtx-4090"

	req, _ := http.NewRequest("GET", url, nil)

	res, _ := http.DefaultClient.Do(req)

	defer res.Body.Close()
	body, _ := io.ReadAll(res.Body)

	fmt.Println(res)
	fmt.Println(string(body))

}
```

```ruby
require 'uri'
require 'net/http'

url = URI("https://api.empiriolabs.ai/v1/gpu/catalog/rtx-4090")

http = Net::HTTP.new(url.host, url.port)
http.use_ssl = true

request = Net::HTTP::Get.new(url)

response = http.request(request)
puts response.read_body
```

```java
import com.mashape.unirest.http.HttpResponse;
import com.mashape.unirest.http.Unirest;

HttpResponse<String> response = Unirest.get("https://api.empiriolabs.ai/v1/gpu/catalog/rtx-4090")
  .asString();
```

```php
<?php
require_once('vendor/autoload.php');

$client = new \GuzzleHttp\Client();

$response = $client->request('GET', 'https://api.empiriolabs.ai/v1/gpu/catalog/rtx-4090');

echo $response->getBody();
```

```csharp
using RestSharp;

var client = new RestClient("https://api.empiriolabs.ai/v1/gpu/catalog/rtx-4090");
var request = new RestRequest(Method.GET);
IRestResponse response = client.Execute(request);
```

```swift
import Foundation

let request = NSMutableURLRequest(url: NSURL(string: "https://api.empiriolabs.ai/v1/gpu/catalog/rtx-4090")! as URL,
                                        cachePolicy: .useProtocolCachePolicy,
                                    timeoutInterval: 10.0)
request.httpMethod = "GET"

let session = URLSession.shared
let dataTask = session.dataTask(with: request as URLRequest, completionHandler: { (data, response, error) -> Void in
  if (error != nil) {
    print(error as Any)
  } else {
    let httpResponse = response as? HTTPURLResponse
    print(httpResponse)
  }
})

dataTask.resume()
```

## Clusters

A cluster combines multiple GPU nodes for distributed training. Reserve the
group in one order and choose how many nodes to use.

Check `ordering_enabled` in the cluster-list response before ordering.
`updated_at` reports the Unix timestamp of the latest inventory refresh.

### Request

GET [https://api.empiriolabs.ai/v1/gpu/clusters](https://api.empiriolabs.ai/v1/gpu/clusters)

```curl
curl https://api.empiriolabs.ai/v1/gpu/clusters
```

```python
import requests

url = "https://api.empiriolabs.ai/v1/gpu/clusters"

response = requests.get(url)

print(response.json())
```

```javascript
const url = 'https://api.empiriolabs.ai/v1/gpu/clusters';
const options = {method: 'GET'};

try {
  const response = await fetch(url, options);
  const data = await response.json();
  console.log(data);
} catch (error) {
  console.error(error);
}
```

```go
package main

import (
	"fmt"
	"net/http"
	"io"
)

func main() {

	url := "https://api.empiriolabs.ai/v1/gpu/clusters"

	req, _ := http.NewRequest("GET", url, nil)

	res, _ := http.DefaultClient.Do(req)

	defer res.Body.Close()
	body, _ := io.ReadAll(res.Body)

	fmt.Println(res)
	fmt.Println(string(body))

}
```

```ruby
require 'uri'
require 'net/http'

url = URI("https://api.empiriolabs.ai/v1/gpu/clusters")

http = Net::HTTP.new(url.host, url.port)
http.use_ssl = true

request = Net::HTTP::Get.new(url)

response = http.request(request)
puts response.read_body
```

```java
import com.mashape.unirest.http.HttpResponse;
import com.mashape.unirest.http.Unirest;

HttpResponse<String> response = Unirest.get("https://api.empiriolabs.ai/v1/gpu/clusters")
  .asString();
```

```php
<?php
require_once('vendor/autoload.php');

$client = new \GuzzleHttp\Client();

$response = $client->request('GET', 'https://api.empiriolabs.ai/v1/gpu/clusters');

echo $response->getBody();
```

```csharp
using RestSharp;

var client = new RestClient("https://api.empiriolabs.ai/v1/gpu/clusters");
var request = new RestRequest(Method.GET);
IRestResponse response = client.Execute(request);
```

```swift
import Foundation

let request = NSMutableURLRequest(url: NSURL(string: "https://api.empiriolabs.ai/v1/gpu/clusters")! as URL,
                                        cachePolicy: .useProtocolCachePolicy,
                                    timeoutInterval: 10.0)
request.httpMethod = "GET"

let session = URLSession.shared
let dataTask = session.dataTask(with: request as URLRequest, completionHandler: { (data, response, error) -> Void in
  if (error != nil) {
    print(error as Any)
  } else {
    let httpResponse = response as? HTTPURLResponse
    print(httpResponse)
  }
})

dataTask.resume()
```

```bash
curl https://api.empiriolabs.ai/v1/gpu/clusters
```

```json
{
  "object": "list",
  "data": [
    {
      "id": "cl_dc1bc5c1e88b08f55616",
      "gpu": "h100-sxm",
      "gpu_display": "H100 SXM",
      "nodes": 4,
      "gpus_per_node": 8,
      "total_gpus": 32,
      "vram_gb": 80,
      "total_vram_gb": 2560,
      "interconnect": "InfiniBand",
      "interconnect_gbps": 400,
      "interconnect_measured": true,
      "cpu_cores_per_node": 192,
      "memory_gb_per_node": 2048,
      "storage_gb_per_node": 2000,
      "region": "US",
      "reliability": 0.998,
      "price_hourly": 88.0
    }
  ]
}
```

`nodes` is how many nodes are free on that fabric, and you can order from 2 up to that number. `price_hourly` is the total for all of them, not per node and not per GPU: divide it by `nodes` for the per-node rate, and multiply by the count you want. `total_vram_gb` is the combined VRAM across every GPU in the whole offer, which is the figure that decides whether a model fits at all.

`interconnect_measured` says whether `interconnect_gbps` is a speed the fabric was measured at or a nameplate figure. Treat a nameplate number as an upper bound when you size a job.

Some clusters report `interconnect` as `High-speed network` with a null `interconnect_gbps`. The interconnect is high-speed and no rate is quoted. Treat a null rate as unspecified, not slow.

Cluster availability changes in large steps. An empty list means none are free right now; poll again later.

### Deploying a cluster

A cluster is ordered as one unit and billed as one unit. Send `nodes` to take
part of a fabric; leave it out to take all of it. Every node is an
ordinary GPU instance afterwards: it appears in your instance list, it is
reachable over SSH with its own instance id, and you stop or destroy it like any
other.

Each node receives the peer list, its own rank and the rendezvous address at
`/etc/empirio/cluster.env`, using the conventional `MASTER_ADDR`, `MASTER_PORT`,
`WORLD_SIZE` and `NODE_RANK` names, so an unmodified `torchrun` script works.
Source that file and launch as you normally would:

```bash
set -a; . /etc/empirio/cluster.env; set +a
torchrun --nnodes=$WORLD_SIZE --node_rank=$NODE_RANK          --master_addr=$MASTER_ADDR --master_port=$MASTER_PORT train.py
```

Attach an object bucket at `/mnt/<name>` or a network volume at `/workspace`
to share files across the cluster. Workspace volumes support a single instance.

## Deploy an instance

Deploying returns the instance and its deployment key and state. Poll `GET /v1/gpu/instances/{instance_id}` until `status` is `running`. If the instance reports `error`, check its error message before starting another deployment.

With a volume, startup waits for its saved directory to restore. If a volume cannot restore or save, inspect its `status` and `error` before retrying. A failed save keeps the last saved data intact and prevents the GPU from being released.

### Request

POST [https://api.empiriolabs.ai/v1/gpu/instances](https://api.empiriolabs.ai/v1/gpu/instances)

```curl
curl -X POST https://api.empiriolabs.ai/v1/gpu/instances \
     -H "Authorization: Bearer <token>" \
     -H "Content-Type: application/json" \
     -d '{
  "gpu_slug": "rtx-4090"
}'
```

```python
import requests

url = "https://api.empiriolabs.ai/v1/gpu/instances"

payload = { "gpu_slug": "rtx-4090" }
headers = {
    "Authorization": "Bearer <token>",
    "Content-Type": "application/json"
}

response = requests.post(url, json=payload, headers=headers)

print(response.json())
```

```javascript
const url = 'https://api.empiriolabs.ai/v1/gpu/instances';
const options = {
  method: 'POST',
  headers: {Authorization: 'Bearer <token>', 'Content-Type': 'application/json'},
  body: '{"gpu_slug":"rtx-4090"}'
};

try {
  const response = await fetch(url, options);
  const data = await response.json();
  console.log(data);
} catch (error) {
  console.error(error);
}
```

```go
package main

import (
	"fmt"
	"strings"
	"net/http"
	"io"
)

func main() {

	url := "https://api.empiriolabs.ai/v1/gpu/instances"

	payload := strings.NewReader("{\n  \"gpu_slug\": \"rtx-4090\"\n}")

	req, _ := http.NewRequest("POST", url, payload)

	req.Header.Add("Authorization", "Bearer <token>")
	req.Header.Add("Content-Type", "application/json")

	res, _ := http.DefaultClient.Do(req)

	defer res.Body.Close()
	body, _ := io.ReadAll(res.Body)

	fmt.Println(res)
	fmt.Println(string(body))

}
```

```ruby
require 'uri'
require 'net/http'

url = URI("https://api.empiriolabs.ai/v1/gpu/instances")

http = Net::HTTP.new(url.host, url.port)
http.use_ssl = true

request = Net::HTTP::Post.new(url)
request["Authorization"] = 'Bearer <token>'
request["Content-Type"] = 'application/json'
request.body = "{\n  \"gpu_slug\": \"rtx-4090\"\n}"

response = http.request(request)
puts response.read_body
```

```java
import com.mashape.unirest.http.HttpResponse;
import com.mashape.unirest.http.Unirest;

HttpResponse<String> response = Unirest.post("https://api.empiriolabs.ai/v1/gpu/instances")
  .header("Authorization", "Bearer <token>")
  .header("Content-Type", "application/json")
  .body("{\n  \"gpu_slug\": \"rtx-4090\"\n}")
  .asString();
```

```php
<?php
require_once('vendor/autoload.php');

$client = new \GuzzleHttp\Client();

$response = $client->request('POST', 'https://api.empiriolabs.ai/v1/gpu/instances', [
  'body' => '{
  "gpu_slug": "rtx-4090"
}',
  'headers' => [
    'Authorization' => 'Bearer <token>',
    'Content-Type' => 'application/json',
  ],
]);

echo $response->getBody();
```

```csharp
using RestSharp;

var client = new RestClient("https://api.empiriolabs.ai/v1/gpu/instances");
var request = new RestRequest(Method.POST);
request.AddHeader("Authorization", "Bearer <token>");
request.AddHeader("Content-Type", "application/json");
request.AddParameter("application/json", "{\n  \"gpu_slug\": \"rtx-4090\"\n}", ParameterType.RequestBody);
IRestResponse response = client.Execute(request);
```

```swift
import Foundation

let headers = [
  "Authorization": "Bearer <token>",
  "Content-Type": "application/json"
]
let parameters = ["gpu_slug": "rtx-4090"] as [String : Any]

let postData = JSONSerialization.data(withJSONObject: parameters, options: [])

let request = NSMutableURLRequest(url: NSURL(string: "https://api.empiriolabs.ai/v1/gpu/instances")! as URL,
                                        cachePolicy: .useProtocolCachePolicy,
                                    timeoutInterval: 10.0)
request.httpMethod = "POST"
request.allHTTPHeaderFields = headers
request.httpBody = postData as Data

let session = URLSession.shared
let dataTask = session.dataTask(with: request as URLRequest, completionHandler: { (data, response, error) -> Void in
  if (error != nil) {
    print(error as Any)
  } else {
    let httpResponse = response as? HTTPURLResponse
    print(httpResponse)
  }
})

dataTask.resume()
```

### Retry or recover a deployment

**Deployment key.** Choose a unique `order_key` in the request body or send
`Idempotency-Key` as a header. Keys contain 8 to 128 letters, numbers, dots,
colons, underscores or hyphens and start with a letter or number. If you send
both, their values must match exactly. Without a key, each POST starts a new
deployment.

**Recovery.** Keep the key if the request times out or the response cannot be
read. Check the deployment from the same account:

```bash
curl "https://api.empiriolabs.ai/v1/gpu/instances?order_key=gpu-order-20261004-001" \
  -H "Authorization: Bearer $EMPIRIOLABS_API_KEY"
```

With `order_key`, this endpoint returns `instance`, `order_key` and
`order_state` instead of the instance list. A key from another account returns
404\. A key continues to identify its original deployment after the instance
stops or is destroyed.

| Result                              | Next action                                                                          |
| ----------------------------------- | ------------------------------------------------------------------------------------ |
| 200, `order_state: issued`          | Use the returned instance. Read `instance.status` for runtime readiness.             |
| 202, `order_state: reserved`        | Check again, or explicitly retry POST with the same key and unchanged settings.      |
| 409, `order_state: failed`          | Review the refusal. Choose a new key only when starting a new deployment.            |
| 409, `order_payload_conflict`       | Check the original deployment. Its key was already used with different settings.     |
| 404, 503, or an unreadable response | Keep the key and check again. Contact support if the deployment remains unconfirmed. |

**Retries.** Reuse the same key and request settings. Do not generate a new key
after a timeout. Changes to the GPU count, image, environment, volume, name or
other settings require a new deployment after checking the original one.

### Deploy a model

Pass a curated `template_slug` or paste any Hugging Face repo id. Model deployments are served from an OpenAI-compatible `/v1` endpoint on the instance.

```bash
curl https://api.empiriolabs.ai/v1/gpu/instances \
  -H "Authorization: Bearer $EMPIRIOLABS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "order_key": "gpu-order-20261004-001",
    "gpu_slug": "rtx-4090",
    "mode": "model",
    "hf_id": "Qwen/Qwen2.5-7B-Instruct"
  }'
```

For gated repos, pass the token in `env`:

```json
{
  "gpu_slug": "a100-80gb",
  "mode": "model",
  "hf_id": "meta-llama/Llama-3.1-8B-Instruct",
  "env": { "HF_TOKEN": "hf_..." }
}
```

### Deploy a template

Templates are ready-to-run environments. Available templates include PyTorch + JupyterLab, ComfyUI, Web Terminal, and Ollama.

```bash
curl https://api.empiriolabs.ai/v1/gpu/instances \
  -H "Authorization: Bearer $EMPIRIOLABS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "gpu_slug": "rtx-4090",
    "mode": "template",
    "template_slug": "pytorch-jupyter",
    "disk_gb": 150
  }'
```

### Deploy a custom Docker image

Run your own CUDA image. CPU-only images may fail because the runtime expects a GPU-compatible container.

```bash
curl https://api.empiriolabs.ai/v1/gpu/instances \
  -H "Authorization: Bearer $EMPIRIOLABS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "gpu_slug": "rtx-4090",
    "mode": "custom",
    "image": "pytorch/pytorch:2.4.0-cuda12.1-cudnn9-runtime",
    "ports": [8000],
    "disk_gb": 150,
    "env": { "MY_VAR": "value" }
  }'
```

## Persistent volumes

Create a volume, select it when deploying, and write the files you want to keep under `/workspace`. This is the default kind, `workspace`; see [Object buckets](#object-buckets) for storage that is mounted and read in place instead. The same volume can be used with any GPU type that has enough runtime storage. A workspace volume attaches to one instance at a time. For live sharing, see [Network volumes](#network-volumes). Stop the attached instance before selecting it for a different deployment.

Workspace and network volumes are billed for their configured size at the returned `price_gb_month`, prorated per second using a 730-hour month. `effective_price_gb_month` includes your current account discount. Storage billing continues while a volume exists, including when it is empty or no GPU is attached, and ends when the volume is deleted. GPU runtime disk remains included in the GPU price.

New volume names use 2 to 32 lowercase letters, numbers and hyphens. Start and end with a letter or number. Uppercase letters are converted to lowercase. Existing names are preserved.

New network volumes allow up to 8 simultaneous instance attachments. Each volume reports its effective limit as `max_attachments`. New network volumes include 10,000 files and directories per GB of volume capacity. Each volume reports its total entry limit as `inode_limit` and current usage as `inodes_used`.

### Create and attach

### Request

POST [https://api.empiriolabs.ai/v1/gpu/volumes](https://api.empiriolabs.ai/v1/gpu/volumes)

```curl
curl -X POST https://api.empiriolabs.ai/v1/gpu/volumes \
     -H "Authorization: Bearer <token>" \
     -H "Content-Type: application/json" \
     -d '{
  "name": "training-data",
  "size_gb": 100
}'
```

```python
import requests

url = "https://api.empiriolabs.ai/v1/gpu/volumes"

payload = {
    "name": "training-data",
    "size_gb": 100
}
headers = {
    "Authorization": "Bearer <token>",
    "Content-Type": "application/json"
}

response = requests.post(url, json=payload, headers=headers)

print(response.json())
```

```javascript
const url = 'https://api.empiriolabs.ai/v1/gpu/volumes';
const options = {
  method: 'POST',
  headers: {Authorization: 'Bearer <token>', 'Content-Type': 'application/json'},
  body: '{"name":"training-data","size_gb":100}'
};

try {
  const response = await fetch(url, options);
  const data = await response.json();
  console.log(data);
} catch (error) {
  console.error(error);
}
```

```go
package main

import (
	"fmt"
	"strings"
	"net/http"
	"io"
)

func main() {

	url := "https://api.empiriolabs.ai/v1/gpu/volumes"

	payload := strings.NewReader("{\n  \"name\": \"training-data\",\n  \"size_gb\": 100\n}")

	req, _ := http.NewRequest("POST", url, payload)

	req.Header.Add("Authorization", "Bearer <token>")
	req.Header.Add("Content-Type", "application/json")

	res, _ := http.DefaultClient.Do(req)

	defer res.Body.Close()
	body, _ := io.ReadAll(res.Body)

	fmt.Println(res)
	fmt.Println(string(body))

}
```

```ruby
require 'uri'
require 'net/http'

url = URI("https://api.empiriolabs.ai/v1/gpu/volumes")

http = Net::HTTP.new(url.host, url.port)
http.use_ssl = true

request = Net::HTTP::Post.new(url)
request["Authorization"] = 'Bearer <token>'
request["Content-Type"] = 'application/json'
request.body = "{\n  \"name\": \"training-data\",\n  \"size_gb\": 100\n}"

response = http.request(request)
puts response.read_body
```

```java
import com.mashape.unirest.http.HttpResponse;
import com.mashape.unirest.http.Unirest;

HttpResponse<String> response = Unirest.post("https://api.empiriolabs.ai/v1/gpu/volumes")
  .header("Authorization", "Bearer <token>")
  .header("Content-Type", "application/json")
  .body("{\n  \"name\": \"training-data\",\n  \"size_gb\": 100\n}")
  .asString();
```

```php
<?php
require_once('vendor/autoload.php');

$client = new \GuzzleHttp\Client();

$response = $client->request('POST', 'https://api.empiriolabs.ai/v1/gpu/volumes', [
  'body' => '{
  "name": "training-data",
  "size_gb": 100
}',
  'headers' => [
    'Authorization' => 'Bearer <token>',
    'Content-Type' => 'application/json',
  ],
]);

echo $response->getBody();
```

```csharp
using RestSharp;

var client = new RestClient("https://api.empiriolabs.ai/v1/gpu/volumes");
var request = new RestRequest(Method.POST);
request.AddHeader("Authorization", "Bearer <token>");
request.AddHeader("Content-Type", "application/json");
request.AddParameter("application/json", "{\n  \"name\": \"training-data\",\n  \"size_gb\": 100\n}", ParameterType.RequestBody);
IRestResponse response = client.Execute(request);
```

```swift
import Foundation

let headers = [
  "Authorization": "Bearer <token>",
  "Content-Type": "application/json"
]
let parameters = [
  "name": "training-data",
  "size_gb": 100
] as [String : Any]

let postData = JSONSerialization.data(withJSONObject: parameters, options: [])

let request = NSMutableURLRequest(url: NSURL(string: "https://api.empiriolabs.ai/v1/gpu/volumes")! as URL,
                                        cachePolicy: .useProtocolCachePolicy,
                                    timeoutInterval: 10.0)
request.httpMethod = "POST"
request.allHTTPHeaderFields = headers
request.httpBody = postData as Data

let session = URLSession.shared
let dataTask = session.dataTask(with: request as URLRequest, completionHandler: { (data, response, error) -> Void in
  if (error != nil) {
    print(error as Any)
  } else {
    let httpResponse = response as? HTTPURLResponse
    print(httpResponse)
  }
})

dataTask.resume()
```

```bash
curl https://api.empiriolabs.ai/v1/gpu/volumes \
  -H "Authorization: Bearer $EMPIRIOLABS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"name":"training-data","size_gb":50}'
```

Use the returned `volume.id` in your deploy request:

```bash
curl https://api.empiriolabs.ai/v1/gpu/instances \
  -H "Authorization: Bearer $EMPIRIOLABS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "gpu_slug":"rtx-4090",
    "mode":"template",
    "template_slug":"pytorch-jupyter",
    "volume_id":"00000000-0000-4000-8000-000000000001"
  }'
```

Choose a volume from 1 to 4,000 GB and a GPU with room for the volume plus the workload.

Archive framing counts toward `bytes_used`. Regular files, directories and relative symlinks within `/workspace` are supported. Device files and links outside that directory are not supported.

### Choosing a volume kind

Choose the storage behavior your workload needs.

|              | Workspace volume                  | Object bucket                                                                 | Network volume                                  |
| ------------ | --------------------------------- | ----------------------------------------------------------------------------- | ----------------------------------------------- |
| Path         | `/workspace`                      | `/mnt/<name>`                                                                 | `/workspace`                                    |
| Sharing      | One GPU instance                  | One instance or every node of one cluster                                     | Multiple compatible instances and cluster nodes |
| Files        | Saved on stop, restored on attach | Read and written in place                                                     | Live shared files with file locking             |
| Billing size | Provisioned capacity              | Selected minimum or stored usage rounded up to whole GB, whichever is greater | Provisioned capacity                            |

The volumes API returns the current storage rate, limits and availability.

Object buckets and network volumes are available with compatible GPU configurations.
The deployment selector updates for the GPU size or cluster you choose. An unavailable
volume stays visible with a message to help you select a compatible configuration.

### Object buckets

Pass `"kind":"object"` to create an S3-compatible bucket instead of a workspace
volume. It is mounted at `/mnt/<name>` on every instance you attach it to, and
read and written in place, so nothing is copied onto the GPU at start or off it
at stop. That makes it the right choice for a training set larger than the
instance's local disk, and for data you want to reach from several instances in
turn.

```bash
curl https://api.empiriolabs.ai/v1/gpu/volumes \
  -H "Authorization: Bearer $EMPIRIOLABS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"name":"datasets","size_gb":500,"kind":"object"}'
```

All storage types use the same naming rules. Invalid names return `invalid_volume_name`.
Object buckets bill at least the size selected. Usage above that size is rounded
up to whole GB and billed at the returned `price_gb_month`. `billable_gb` reports
the current quantity used for storage billing. They count against the same
per-account volume and capacity limits.

`GET /v1/gpu/volumes` returns `kind` on every volume, and a `capabilities`
object whose `object_storage` field reports whether object buckets can be
created right now. A request for `"kind":"object"` while that is `false` is
refused with `object_storage_unavailable` rather than creating a volume that
could not be mounted.

Use `capabilities.object_gpu_boxes[gpu_slug]` to choose a compatible `num_gpus`
value, or `capabilities.object_cluster_ids` to choose a compatible cluster.
`capabilities.object_gpu_slugs` lists GPU types with compatible sizes. An object
bucket can be larger than the instance's local disk.

### Network volumes

Create a shared filesystem with `"kind":"network"`:

```bash
curl https://api.empiriolabs.ai/v1/gpu/volumes \
  -H "Authorization: Bearer $EMPIRIOLABS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"name":"shared-training","size_gb":100,"kind":"network"}'
```

Pass the returned `volume.id` as `volume_id` in each instance or cluster deployment.
Compatible instances see the same files under `/workspace`. File locks coordinate
concurrent access; applications writing the same file should use them.
Stopping one instance leaves the volume available to its other attachments.

`GET /v1/gpu/volumes` returns `capabilities.network_storage` and
`capabilities.network_gpu_slugs`, and `capabilities.network_cluster_ids` for current availability. Each volume reports
`attachment_count`, `max_attachments`, `inode_limit`, and `inodes_used`.
The inode limit covers files and directories. A `creating` volume cannot attach
and does not accrue storage charges until it becomes available.

The list response reports current defaults in `policy.network_max_attachments`
and `policy.network_inodes_per_gb`. Each volume retains its own effective limits.

Use `capabilities.network_gpu_boxes[gpu_slug]` to choose a compatible `num_gpus`
value. Compatibility can differ between machine sizes of the same GPU type.

The dashboard uses the same create and deploy controls. A network volume remains
selectable while it has available attachments. Stop all attached instances before
deleting it. Existing workspace volumes keep their single-instance behavior.

The response also includes `policy`, with naming rules, the per-volume capacity ceiling, and the default attachment allowance for new network volumes.

### List, inspect and delete

### Request

GET [https://api.empiriolabs.ai/v1/gpu/volumes](https://api.empiriolabs.ai/v1/gpu/volumes)

```curl
curl https://api.empiriolabs.ai/v1/gpu/volumes \
     -H "Authorization: Bearer <token>" \
     -H "Content-Type: application/json"
```

```python
import requests

url = "https://api.empiriolabs.ai/v1/gpu/volumes"

payload = {}
headers = {
    "Authorization": "Bearer <token>",
    "Content-Type": "application/json"
}

response = requests.get(url, json=payload, headers=headers)

print(response.json())
```

```javascript
const url = 'https://api.empiriolabs.ai/v1/gpu/volumes';
const options = {
  method: 'GET',
  headers: {Authorization: 'Bearer <token>', 'Content-Type': 'application/json'},
  body: '{}'
};

try {
  const response = await fetch(url, options);
  const data = await response.json();
  console.log(data);
} catch (error) {
  console.error(error);
}
```

```go
package main

import (
	"fmt"
	"strings"
	"net/http"
	"io"
)

func main() {

	url := "https://api.empiriolabs.ai/v1/gpu/volumes"

	payload := strings.NewReader("{}")

	req, _ := http.NewRequest("GET", url, payload)

	req.Header.Add("Authorization", "Bearer <token>")
	req.Header.Add("Content-Type", "application/json")

	res, _ := http.DefaultClient.Do(req)

	defer res.Body.Close()
	body, _ := io.ReadAll(res.Body)

	fmt.Println(res)
	fmt.Println(string(body))

}
```

```ruby
require 'uri'
require 'net/http'

url = URI("https://api.empiriolabs.ai/v1/gpu/volumes")

http = Net::HTTP.new(url.host, url.port)
http.use_ssl = true

request = Net::HTTP::Get.new(url)
request["Authorization"] = 'Bearer <token>'
request["Content-Type"] = 'application/json'
request.body = "{}"

response = http.request(request)
puts response.read_body
```

```java
import com.mashape.unirest.http.HttpResponse;
import com.mashape.unirest.http.Unirest;

HttpResponse<String> response = Unirest.get("https://api.empiriolabs.ai/v1/gpu/volumes")
  .header("Authorization", "Bearer <token>")
  .header("Content-Type", "application/json")
  .body("{}")
  .asString();
```

```php
<?php
require_once('vendor/autoload.php');

$client = new \GuzzleHttp\Client();

$response = $client->request('GET', 'https://api.empiriolabs.ai/v1/gpu/volumes', [
  'body' => '{}',
  'headers' => [
    'Authorization' => 'Bearer <token>',
    'Content-Type' => 'application/json',
  ],
]);

echo $response->getBody();
```

```csharp
using RestSharp;

var client = new RestClient("https://api.empiriolabs.ai/v1/gpu/volumes");
var request = new RestRequest(Method.GET);
request.AddHeader("Authorization", "Bearer <token>");
request.AddHeader("Content-Type", "application/json");
request.AddParameter("application/json", "{}", ParameterType.RequestBody);
IRestResponse response = client.Execute(request);
```

```swift
import Foundation

let headers = [
  "Authorization": "Bearer <token>",
  "Content-Type": "application/json"
]
let parameters = [] as [String : Any]

let postData = JSONSerialization.data(withJSONObject: parameters, options: [])

let request = NSMutableURLRequest(url: NSURL(string: "https://api.empiriolabs.ai/v1/gpu/volumes")! as URL,
                                        cachePolicy: .useProtocolCachePolicy,
                                    timeoutInterval: 10.0)
request.httpMethod = "GET"
request.allHTTPHeaderFields = headers
request.httpBody = postData as Data

let session = URLSession.shared
let dataTask = session.dataTask(with: request as URLRequest, completionHandler: { (data, response, error) -> Void in
  if (error != nil) {
    print(error as Any)
  } else {
    let httpResponse = response as? HTTPURLResponse
    print(httpResponse)
  }
})

dataTask.resume()
```

### Request

GET [https://api.empiriolabs.ai/v1/gpu/volumes/\{volume_id}](https://api.empiriolabs.ai/v1/gpu/volumes/\{volume_id})

```curl
curl https://api.empiriolabs.ai/v1/gpu/volumes/volume_id \
     -H "Authorization: Bearer <token>" \
     -H "Content-Type: application/json"
```

```python
import requests

url = "https://api.empiriolabs.ai/v1/gpu/volumes/volume_id"

payload = {}
headers = {
    "Authorization": "Bearer <token>",
    "Content-Type": "application/json"
}

response = requests.get(url, json=payload, headers=headers)

print(response.json())
```

```javascript
const url = 'https://api.empiriolabs.ai/v1/gpu/volumes/volume_id';
const options = {
  method: 'GET',
  headers: {Authorization: 'Bearer <token>', 'Content-Type': 'application/json'},
  body: '{}'
};

try {
  const response = await fetch(url, options);
  const data = await response.json();
  console.log(data);
} catch (error) {
  console.error(error);
}
```

```go
package main

import (
	"fmt"
	"strings"
	"net/http"
	"io"
)

func main() {

	url := "https://api.empiriolabs.ai/v1/gpu/volumes/volume_id"

	payload := strings.NewReader("{}")

	req, _ := http.NewRequest("GET", url, payload)

	req.Header.Add("Authorization", "Bearer <token>")
	req.Header.Add("Content-Type", "application/json")

	res, _ := http.DefaultClient.Do(req)

	defer res.Body.Close()
	body, _ := io.ReadAll(res.Body)

	fmt.Println(res)
	fmt.Println(string(body))

}
```

```ruby
require 'uri'
require 'net/http'

url = URI("https://api.empiriolabs.ai/v1/gpu/volumes/volume_id")

http = Net::HTTP.new(url.host, url.port)
http.use_ssl = true

request = Net::HTTP::Get.new(url)
request["Authorization"] = 'Bearer <token>'
request["Content-Type"] = 'application/json'
request.body = "{}"

response = http.request(request)
puts response.read_body
```

```java
import com.mashape.unirest.http.HttpResponse;
import com.mashape.unirest.http.Unirest;

HttpResponse<String> response = Unirest.get("https://api.empiriolabs.ai/v1/gpu/volumes/volume_id")
  .header("Authorization", "Bearer <token>")
  .header("Content-Type", "application/json")
  .body("{}")
  .asString();
```

```php
<?php
require_once('vendor/autoload.php');

$client = new \GuzzleHttp\Client();

$response = $client->request('GET', 'https://api.empiriolabs.ai/v1/gpu/volumes/volume_id', [
  'body' => '{}',
  'headers' => [
    'Authorization' => 'Bearer <token>',
    'Content-Type' => 'application/json',
  ],
]);

echo $response->getBody();
```

```csharp
using RestSharp;

var client = new RestClient("https://api.empiriolabs.ai/v1/gpu/volumes/volume_id");
var request = new RestRequest(Method.GET);
request.AddHeader("Authorization", "Bearer <token>");
request.AddHeader("Content-Type", "application/json");
request.AddParameter("application/json", "{}", ParameterType.RequestBody);
IRestResponse response = client.Execute(request);
```

```swift
import Foundation

let headers = [
  "Authorization": "Bearer <token>",
  "Content-Type": "application/json"
]
let parameters = [] as [String : Any]

let postData = JSONSerialization.data(withJSONObject: parameters, options: [])

let request = NSMutableURLRequest(url: NSURL(string: "https://api.empiriolabs.ai/v1/gpu/volumes/volume_id")! as URL,
                                        cachePolicy: .useProtocolCachePolicy,
                                    timeoutInterval: 10.0)
request.httpMethod = "GET"
request.allHTTPHeaderFields = headers
request.httpBody = postData as Data

let session = URLSession.shared
let dataTask = session.dataTask(with: request as URLRequest, completionHandler: { (data, response, error) -> Void in
  if (error != nil) {
    print(error as Any)
  } else {
    let httpResponse = response as? HTTPURLResponse
    print(httpResponse)
  }
})

dataTask.resume()
```

For workspace volumes, `last_synced_at` records the most recent successful save. After stopping, wait until the volume is `available` before attaching it elsewhere. An `error` leaves the last saved directory intact. If a save failed, correct the problem, such as a full volume, then retry stopping the instance. GPU runtime charges continue until the GPU is released. If the machine is lost unexpectedly, files written since the last successful save cannot be recovered from the volume.

### Request

DELETE [https://api.empiriolabs.ai/v1/gpu/volumes/\{volume_id}](https://api.empiriolabs.ai/v1/gpu/volumes/\{volume_id})

```curl
curl -X DELETE https://api.empiriolabs.ai/v1/gpu/volumes/volume_id \
     -H "Authorization: Bearer <token>" \
     -H "Content-Type: application/json" \
     -d '{}'
```

```python
import requests

url = "https://api.empiriolabs.ai/v1/gpu/volumes/volume_id"

payload = {}
headers = {
    "Authorization": "Bearer <token>",
    "Content-Type": "application/json"
}

response = requests.delete(url, json=payload, headers=headers)

print(response.json())
```

```javascript
const url = 'https://api.empiriolabs.ai/v1/gpu/volumes/volume_id';
const options = {
  method: 'DELETE',
  headers: {Authorization: 'Bearer <token>', 'Content-Type': 'application/json'},
  body: '{}'
};

try {
  const response = await fetch(url, options);
  const data = await response.json();
  console.log(data);
} catch (error) {
  console.error(error);
}
```

```go
package main

import (
	"fmt"
	"strings"
	"net/http"
	"io"
)

func main() {

	url := "https://api.empiriolabs.ai/v1/gpu/volumes/volume_id"

	payload := strings.NewReader("{}")

	req, _ := http.NewRequest("DELETE", url, payload)

	req.Header.Add("Authorization", "Bearer <token>")
	req.Header.Add("Content-Type", "application/json")

	res, _ := http.DefaultClient.Do(req)

	defer res.Body.Close()
	body, _ := io.ReadAll(res.Body)

	fmt.Println(res)
	fmt.Println(string(body))

}
```

```ruby
require 'uri'
require 'net/http'

url = URI("https://api.empiriolabs.ai/v1/gpu/volumes/volume_id")

http = Net::HTTP.new(url.host, url.port)
http.use_ssl = true

request = Net::HTTP::Delete.new(url)
request["Authorization"] = 'Bearer <token>'
request["Content-Type"] = 'application/json'
request.body = "{}"

response = http.request(request)
puts response.read_body
```

```java
import com.mashape.unirest.http.HttpResponse;
import com.mashape.unirest.http.Unirest;

HttpResponse<String> response = Unirest.delete("https://api.empiriolabs.ai/v1/gpu/volumes/volume_id")
  .header("Authorization", "Bearer <token>")
  .header("Content-Type", "application/json")
  .body("{}")
  .asString();
```

```php
<?php
require_once('vendor/autoload.php');

$client = new \GuzzleHttp\Client();

$response = $client->request('DELETE', 'https://api.empiriolabs.ai/v1/gpu/volumes/volume_id', [
  'body' => '{}',
  'headers' => [
    'Authorization' => 'Bearer <token>',
    'Content-Type' => 'application/json',
  ],
]);

echo $response->getBody();
```

```csharp
using RestSharp;

var client = new RestClient("https://api.empiriolabs.ai/v1/gpu/volumes/volume_id");
var request = new RestRequest(Method.DELETE);
request.AddHeader("Authorization", "Bearer <token>");
request.AddHeader("Content-Type", "application/json");
request.AddParameter("application/json", "{}", ParameterType.RequestBody);
IRestResponse response = client.Execute(request);
```

```swift
import Foundation

let headers = [
  "Authorization": "Bearer <token>",
  "Content-Type": "application/json"
]
let parameters = [] as [String : Any]

let postData = JSONSerialization.data(withJSONObject: parameters, options: [])

let request = NSMutableURLRequest(url: NSURL(string: "https://api.empiriolabs.ai/v1/gpu/volumes/volume_id")! as URL,
                                        cachePolicy: .useProtocolCachePolicy,
                                    timeoutInterval: 10.0)
request.httpMethod = "DELETE"
request.allHTTPHeaderFields = headers
request.httpBody = postData as Data

let session = URLSession.shared
let dataTask = session.dataTask(with: request as URLRequest, completionHandler: { (data, response, error) -> Void in
  if (error != nil) {
    print(error as Any)
  } else {
    let httpResponse = response as? HTTPURLResponse
    print(httpResponse)
  }
})

dataTask.resume()
```

```bash
curl -X DELETE https://api.empiriolabs.ai/v1/gpu/volumes/$VOLUME_ID \
  -H "Authorization: Bearer $EMPIRIOLABS_API_KEY"
```

Deleting an unattached volume permanently removes its saved files and ends its storage billing. A `202` response with `deleted: false` means deletion is in progress. Poll `GET /v1/gpu/volumes/{volume_id}` until it returns `404`. Deleting an attached volume returns `409`. Destroying a GPU does not delete its volume.

## Training and fine-tuning

Choose a training preset and attach a persistent volume before deploying. All three presets require a volume; a deploy request without `volume_id` is refused with `volume_required`. Deploy in `template` mode using the preset's slug. Once the instance is running, open JupyterLab from its dashboard connect link, using the same authenticated connect path as other notebook templates. Training starts only when you run the starter command in a JupyterLab terminal.

| Preset                               | What it provides                                                                         | Starter file                             |
| ------------------------------------ | ---------------------------------------------------------------------------------------- | ---------------------------------------- |
| Axolotl (`axolotl`)                  | LoRA, QLoRA, and full fine-tuning from a YAML config. The starter uses QLoRA.            | `/workspace/training/axolotl/config.yml` |
| Unsloth (`unsloth`)                  | Fast single-GPU LoRA and QLoRA. The starter uses 4-bit model loading with LoRA adapters. | `/workspace/training/unsloth/train.py`   |
| Hugging Face TRL + PEFT (`trl-peft`) | Supervised fine-tuning with a PEFT LoRA starter.                                         | `/workspace/training/trl-peft/train.py`  |

### Prepare data and start training

All three starters read `/workspace/dataset.jsonl`. Axolotl expects Alpaca-format JSON objects with `instruction`, `input`, and `output` fields. Unsloth and TRL + PEFT expect one JSON object per line with a `text` field. Keep your dataset, configs, scripts, and notebooks under `/workspace` so they are saved with the volume.

Edit the starter for your model and training settings, and choose GPU memory capacity that fits them. For Axolotl, switch to LoRA with `adapter: lora` and `load_in_4bit: false`; for full fine-tuning, remove `adapter` and set `load_in_4bit: false`. For Unsloth, set `load_in_4bit=False` in `train.py` to use LoRA without 4-bit loading. For TRL + PEFT, remove `peft_config` from the trainer to use full fine-tuning.

Run the command for your chosen preset:

```bash
# Axolotl
axolotl train /workspace/training/axolotl/config.yml

# Unsloth
python /workspace/training/unsloth/train.py

# Hugging Face TRL + PEFT
python /workspace/training/trl-peft/train.py
```

### Save and resume

Checkpoints are written under `/workspace/checkpoints/<slug>`: `/workspace/checkpoints/axolotl`, `/workspace/checkpoints/unsloth`, or `/workspace/checkpoints/trl-peft`. The Unsloth and TRL + PEFT starters also save the final model and tokenizer in a `final` subdirectory.

Save and flush your training checkpoint before stopping or destroying the instance. Workspace volumes save `/workspace` before release. Network volumes retain shared files and flush this instance before release. Start the stopped instance again, or attach the same volume to a new deployment of the same preset, to restore your dataset, starter files, and checkpoints. Existing starter files are preserved. Restoring files does not restart training automatically.

For Axolotl, add the saved checkpoint path to `/workspace/training/axolotl/config.yml`, then rerun the Axolotl training command:

```yaml
resume_from_checkpoint: /workspace/checkpoints/axolotl/checkpoint-N
```

For Unsloth or TRL + PEFT, set `RESUME_FROM_CHECKPOINT` when running the corresponding script:

```bash
# Unsloth
RESUME_FROM_CHECKPOINT=/workspace/checkpoints/unsloth/checkpoint-N python /workspace/training/unsloth/train.py

# Hugging Face TRL + PEFT
RESUME_FROM_CHECKPOINT=/workspace/checkpoints/trl-peft/checkpoint-N python /workspace/training/trl-peft/train.py
```

Replace `checkpoint-N` with an existing checkpoint directory from your run. Files outside `/workspace` remain temporary, and volume storage billing continues until you delete the volume.

## Manage lifecycle

### Request

GET [https://api.empiriolabs.ai/v1/gpu/instances](https://api.empiriolabs.ai/v1/gpu/instances)

```curl
curl https://api.empiriolabs.ai/v1/gpu/instances \
     -H "Authorization: Bearer <token>" \
     -H "Content-Type: application/json"
```

```python
import requests

url = "https://api.empiriolabs.ai/v1/gpu/instances"

payload = {}
headers = {
    "Authorization": "Bearer <token>",
    "Content-Type": "application/json"
}

response = requests.get(url, json=payload, headers=headers)

print(response.json())
```

```javascript
const url = 'https://api.empiriolabs.ai/v1/gpu/instances';
const options = {
  method: 'GET',
  headers: {Authorization: 'Bearer <token>', 'Content-Type': 'application/json'},
  body: '{}'
};

try {
  const response = await fetch(url, options);
  const data = await response.json();
  console.log(data);
} catch (error) {
  console.error(error);
}
```

```go
package main

import (
	"fmt"
	"strings"
	"net/http"
	"io"
)

func main() {

	url := "https://api.empiriolabs.ai/v1/gpu/instances"

	payload := strings.NewReader("{}")

	req, _ := http.NewRequest("GET", url, payload)

	req.Header.Add("Authorization", "Bearer <token>")
	req.Header.Add("Content-Type", "application/json")

	res, _ := http.DefaultClient.Do(req)

	defer res.Body.Close()
	body, _ := io.ReadAll(res.Body)

	fmt.Println(res)
	fmt.Println(string(body))

}
```

```ruby
require 'uri'
require 'net/http'

url = URI("https://api.empiriolabs.ai/v1/gpu/instances")

http = Net::HTTP.new(url.host, url.port)
http.use_ssl = true

request = Net::HTTP::Get.new(url)
request["Authorization"] = 'Bearer <token>'
request["Content-Type"] = 'application/json'
request.body = "{}"

response = http.request(request)
puts response.read_body
```

```java
import com.mashape.unirest.http.HttpResponse;
import com.mashape.unirest.http.Unirest;

HttpResponse<String> response = Unirest.get("https://api.empiriolabs.ai/v1/gpu/instances")
  .header("Authorization", "Bearer <token>")
  .header("Content-Type", "application/json")
  .body("{}")
  .asString();
```

```php
<?php
require_once('vendor/autoload.php');

$client = new \GuzzleHttp\Client();

$response = $client->request('GET', 'https://api.empiriolabs.ai/v1/gpu/instances', [
  'body' => '{}',
  'headers' => [
    'Authorization' => 'Bearer <token>',
    'Content-Type' => 'application/json',
  ],
]);

echo $response->getBody();
```

```csharp
using RestSharp;

var client = new RestClient("https://api.empiriolabs.ai/v1/gpu/instances");
var request = new RestRequest(Method.GET);
request.AddHeader("Authorization", "Bearer <token>");
request.AddHeader("Content-Type", "application/json");
request.AddParameter("application/json", "{}", ParameterType.RequestBody);
IRestResponse response = client.Execute(request);
```

```swift
import Foundation

let headers = [
  "Authorization": "Bearer <token>",
  "Content-Type": "application/json"
]
let parameters = [] as [String : Any]

let postData = JSONSerialization.data(withJSONObject: parameters, options: [])

let request = NSMutableURLRequest(url: NSURL(string: "https://api.empiriolabs.ai/v1/gpu/instances")! as URL,
                                        cachePolicy: .useProtocolCachePolicy,
                                    timeoutInterval: 10.0)
request.httpMethod = "GET"
request.allHTTPHeaderFields = headers
request.httpBody = postData as Data

let session = URLSession.shared
let dataTask = session.dataTask(with: request as URLRequest, completionHandler: { (data, response, error) -> Void in
  if (error != nil) {
    print(error as Any)
  } else {
    let httpResponse = response as? HTTPURLResponse
    print(httpResponse)
  }
})

dataTask.resume()
```

### Request

GET [https://api.empiriolabs.ai/v1/gpu/instances/\{instance_id}](https://api.empiriolabs.ai/v1/gpu/instances/\{instance_id})

```curl
curl https://api.empiriolabs.ai/v1/gpu/instances/instance_id \
     -H "Authorization: Bearer <token>" \
     -H "Content-Type: application/json"
```

```python
import requests

url = "https://api.empiriolabs.ai/v1/gpu/instances/instance_id"

payload = {}
headers = {
    "Authorization": "Bearer <token>",
    "Content-Type": "application/json"
}

response = requests.get(url, json=payload, headers=headers)

print(response.json())
```

```javascript
const url = 'https://api.empiriolabs.ai/v1/gpu/instances/instance_id';
const options = {
  method: 'GET',
  headers: {Authorization: 'Bearer <token>', 'Content-Type': 'application/json'},
  body: '{}'
};

try {
  const response = await fetch(url, options);
  const data = await response.json();
  console.log(data);
} catch (error) {
  console.error(error);
}
```

```go
package main

import (
	"fmt"
	"strings"
	"net/http"
	"io"
)

func main() {

	url := "https://api.empiriolabs.ai/v1/gpu/instances/instance_id"

	payload := strings.NewReader("{}")

	req, _ := http.NewRequest("GET", url, payload)

	req.Header.Add("Authorization", "Bearer <token>")
	req.Header.Add("Content-Type", "application/json")

	res, _ := http.DefaultClient.Do(req)

	defer res.Body.Close()
	body, _ := io.ReadAll(res.Body)

	fmt.Println(res)
	fmt.Println(string(body))

}
```

```ruby
require 'uri'
require 'net/http'

url = URI("https://api.empiriolabs.ai/v1/gpu/instances/instance_id")

http = Net::HTTP.new(url.host, url.port)
http.use_ssl = true

request = Net::HTTP::Get.new(url)
request["Authorization"] = 'Bearer <token>'
request["Content-Type"] = 'application/json'
request.body = "{}"

response = http.request(request)
puts response.read_body
```

```java
import com.mashape.unirest.http.HttpResponse;
import com.mashape.unirest.http.Unirest;

HttpResponse<String> response = Unirest.get("https://api.empiriolabs.ai/v1/gpu/instances/instance_id")
  .header("Authorization", "Bearer <token>")
  .header("Content-Type", "application/json")
  .body("{}")
  .asString();
```

```php
<?php
require_once('vendor/autoload.php');

$client = new \GuzzleHttp\Client();

$response = $client->request('GET', 'https://api.empiriolabs.ai/v1/gpu/instances/instance_id', [
  'body' => '{}',
  'headers' => [
    'Authorization' => 'Bearer <token>',
    'Content-Type' => 'application/json',
  ],
]);

echo $response->getBody();
```

```csharp
using RestSharp;

var client = new RestClient("https://api.empiriolabs.ai/v1/gpu/instances/instance_id");
var request = new RestRequest(Method.GET);
request.AddHeader("Authorization", "Bearer <token>");
request.AddHeader("Content-Type", "application/json");
request.AddParameter("application/json", "{}", ParameterType.RequestBody);
IRestResponse response = client.Execute(request);
```

```swift
import Foundation

let headers = [
  "Authorization": "Bearer <token>",
  "Content-Type": "application/json"
]
let parameters = [] as [String : Any]

let postData = JSONSerialization.data(withJSONObject: parameters, options: [])

let request = NSMutableURLRequest(url: NSURL(string: "https://api.empiriolabs.ai/v1/gpu/instances/instance_id")! as URL,
                                        cachePolicy: .useProtocolCachePolicy,
                                    timeoutInterval: 10.0)
request.httpMethod = "GET"
request.allHTTPHeaderFields = headers
request.httpBody = postData as Data

let session = URLSession.shared
let dataTask = session.dataTask(with: request as URLRequest, completionHandler: { (data, response, error) -> Void in
  if (error != nil) {
    print(error as Any)
  } else {
    let httpResponse = response as? HTTPURLResponse
    print(httpResponse)
  }
})

dataTask.resume()
```

### Request

POST [https://api.empiriolabs.ai/v1/gpu/instances/\{instance_id}/\{action}](https://api.empiriolabs.ai/v1/gpu/instances/\{instance_id}/\{action})

```curl
curl -X POST https://api.empiriolabs.ai/v1/gpu/instances/instance_id/stop \
     -H "Authorization: Bearer <token>" \
     -H "Content-Type: application/json" \
     -d '{}'
```

```python
import requests

url = "https://api.empiriolabs.ai/v1/gpu/instances/instance_id/stop"

payload = {}
headers = {
    "Authorization": "Bearer <token>",
    "Content-Type": "application/json"
}

response = requests.post(url, json=payload, headers=headers)

print(response.json())
```

```javascript
const url = 'https://api.empiriolabs.ai/v1/gpu/instances/instance_id/stop';
const options = {
  method: 'POST',
  headers: {Authorization: 'Bearer <token>', 'Content-Type': 'application/json'},
  body: '{}'
};

try {
  const response = await fetch(url, options);
  const data = await response.json();
  console.log(data);
} catch (error) {
  console.error(error);
}
```

```go
package main

import (
	"fmt"
	"strings"
	"net/http"
	"io"
)

func main() {

	url := "https://api.empiriolabs.ai/v1/gpu/instances/instance_id/stop"

	payload := strings.NewReader("{}")

	req, _ := http.NewRequest("POST", url, payload)

	req.Header.Add("Authorization", "Bearer <token>")
	req.Header.Add("Content-Type", "application/json")

	res, _ := http.DefaultClient.Do(req)

	defer res.Body.Close()
	body, _ := io.ReadAll(res.Body)

	fmt.Println(res)
	fmt.Println(string(body))

}
```

```ruby
require 'uri'
require 'net/http'

url = URI("https://api.empiriolabs.ai/v1/gpu/instances/instance_id/stop")

http = Net::HTTP.new(url.host, url.port)
http.use_ssl = true

request = Net::HTTP::Post.new(url)
request["Authorization"] = 'Bearer <token>'
request["Content-Type"] = 'application/json'
request.body = "{}"

response = http.request(request)
puts response.read_body
```

```java
import com.mashape.unirest.http.HttpResponse;
import com.mashape.unirest.http.Unirest;

HttpResponse<String> response = Unirest.post("https://api.empiriolabs.ai/v1/gpu/instances/instance_id/stop")
  .header("Authorization", "Bearer <token>")
  .header("Content-Type", "application/json")
  .body("{}")
  .asString();
```

```php
<?php
require_once('vendor/autoload.php');

$client = new \GuzzleHttp\Client();

$response = $client->request('POST', 'https://api.empiriolabs.ai/v1/gpu/instances/instance_id/stop', [
  'body' => '{}',
  'headers' => [
    'Authorization' => 'Bearer <token>',
    'Content-Type' => 'application/json',
  ],
]);

echo $response->getBody();
```

```csharp
using RestSharp;

var client = new RestClient("https://api.empiriolabs.ai/v1/gpu/instances/instance_id/stop");
var request = new RestRequest(Method.POST);
request.AddHeader("Authorization", "Bearer <token>");
request.AddHeader("Content-Type", "application/json");
request.AddParameter("application/json", "{}", ParameterType.RequestBody);
IRestResponse response = client.Execute(request);
```

```swift
import Foundation

let headers = [
  "Authorization": "Bearer <token>",
  "Content-Type": "application/json"
]
let parameters = [] as [String : Any]

let postData = JSONSerialization.data(withJSONObject: parameters, options: [])

let request = NSMutableURLRequest(url: NSURL(string: "https://api.empiriolabs.ai/v1/gpu/instances/instance_id/stop")! as URL,
                                        cachePolicy: .useProtocolCachePolicy,
                                    timeoutInterval: 10.0)
request.httpMethod = "POST"
request.allHTTPHeaderFields = headers
request.httpBody = postData as Data

let session = URLSession.shared
let dataTask = session.dataTask(with: request as URLRequest, completionHandler: { (data, response, error) -> Void in
  if (error != nil) {
    print(error as Any)
  } else {
    let httpResponse = response as? HTTPURLResponse
    print(httpResponse)
  }
})

dataTask.resume()
```

Use `refresh` to re-sync status, `stop` to save the selected volume and release the allocation, and `start` to redeploy the saved instance spec and restore its volume.

```bash
curl -X POST https://api.empiriolabs.ai/v1/gpu/instances/$ID/stop \
  -H "Authorization: Bearer $EMPIRIOLABS_API_KEY"
```

### Request

DELETE [https://api.empiriolabs.ai/v1/gpu/instances/\{instance_id}](https://api.empiriolabs.ai/v1/gpu/instances/\{instance_id})

```curl
curl -X DELETE https://api.empiriolabs.ai/v1/gpu/instances/instance_id \
     -H "Authorization: Bearer <token>" \
     -H "Content-Type: application/json" \
     -d '{}'
```

```python
import requests

url = "https://api.empiriolabs.ai/v1/gpu/instances/instance_id"

payload = {}
headers = {
    "Authorization": "Bearer <token>",
    "Content-Type": "application/json"
}

response = requests.delete(url, json=payload, headers=headers)

print(response.json())
```

```javascript
const url = 'https://api.empiriolabs.ai/v1/gpu/instances/instance_id';
const options = {
  method: 'DELETE',
  headers: {Authorization: 'Bearer <token>', 'Content-Type': 'application/json'},
  body: '{}'
};

try {
  const response = await fetch(url, options);
  const data = await response.json();
  console.log(data);
} catch (error) {
  console.error(error);
}
```

```go
package main

import (
	"fmt"
	"strings"
	"net/http"
	"io"
)

func main() {

	url := "https://api.empiriolabs.ai/v1/gpu/instances/instance_id"

	payload := strings.NewReader("{}")

	req, _ := http.NewRequest("DELETE", url, payload)

	req.Header.Add("Authorization", "Bearer <token>")
	req.Header.Add("Content-Type", "application/json")

	res, _ := http.DefaultClient.Do(req)

	defer res.Body.Close()
	body, _ := io.ReadAll(res.Body)

	fmt.Println(res)
	fmt.Println(string(body))

}
```

```ruby
require 'uri'
require 'net/http'

url = URI("https://api.empiriolabs.ai/v1/gpu/instances/instance_id")

http = Net::HTTP.new(url.host, url.port)
http.use_ssl = true

request = Net::HTTP::Delete.new(url)
request["Authorization"] = 'Bearer <token>'
request["Content-Type"] = 'application/json'
request.body = "{}"

response = http.request(request)
puts response.read_body
```

```java
import com.mashape.unirest.http.HttpResponse;
import com.mashape.unirest.http.Unirest;

HttpResponse<String> response = Unirest.delete("https://api.empiriolabs.ai/v1/gpu/instances/instance_id")
  .header("Authorization", "Bearer <token>")
  .header("Content-Type", "application/json")
  .body("{}")
  .asString();
```

```php
<?php
require_once('vendor/autoload.php');

$client = new \GuzzleHttp\Client();

$response = $client->request('DELETE', 'https://api.empiriolabs.ai/v1/gpu/instances/instance_id', [
  'body' => '{}',
  'headers' => [
    'Authorization' => 'Bearer <token>',
    'Content-Type' => 'application/json',
  ],
]);

echo $response->getBody();
```

```csharp
using RestSharp;

var client = new RestClient("https://api.empiriolabs.ai/v1/gpu/instances/instance_id");
var request = new RestRequest(Method.DELETE);
request.AddHeader("Authorization", "Bearer <token>");
request.AddHeader("Content-Type", "application/json");
request.AddParameter("application/json", "{}", ParameterType.RequestBody);
IRestResponse response = client.Execute(request);
```

```swift
import Foundation

let headers = [
  "Authorization": "Bearer <token>",
  "Content-Type": "application/json"
]
let parameters = [] as [String : Any]

let postData = JSONSerialization.data(withJSONObject: parameters, options: [])

let request = NSMutableURLRequest(url: NSURL(string: "https://api.empiriolabs.ai/v1/gpu/instances/instance_id")! as URL,
                                        cachePolicy: .useProtocolCachePolicy,
                                    timeoutInterval: 10.0)
request.httpMethod = "DELETE"
request.allHTTPHeaderFields = headers
request.httpBody = postData as Data

let session = URLSession.shared
let dataTask = session.dataTask(with: request as URLRequest, completionHandler: { (data, response, error) -> Void in
  if (error != nil) {
    print(error as Any)
  } else {
    let httpResponse = response as? HTTPURLResponse
    print(httpResponse)
  }
})

dataTask.resume()
```

Destroying an instance saves its selected volume before releasing the GPU and ending runtime billing. Poll until the instance is `destroyed`. The GPU cannot be restored; its volume can be selected for another deployment.

## Statuses

| Status         | Meaning                                                                                                                       |
| -------------- | ----------------------------------------------------------------------------------------------------------------------------- |
| `provisioning` | Capacity is being allocated.                                                                                                  |
| `loading`      | The workload is starting or warming.                                                                                          |
| `running`      | The workload is reachable through the connect path and billing is active.                                                     |
| `stopping`     | A selected volume is saving, or the GPU is being released. Runtime billing continues until release.                           |
| `stopped`      | GPU runtime billing is paused. Start redeploys the saved spec and restores its selected volume.                               |
| `error`        | Provisioning or runtime setup failed, or allocation did not become ready in time. The instance can be refreshed or destroyed. |
| `destroyed`    | The instance has been permanently removed.                                                                                    |

## Connect to a running instance

Use the connect endpoint with your EmpirioLabs API key. It supports GET, POST, PUT, PATCH, and DELETE. A JSON request with `"stream": true`, such as an OpenAI-compatible chat completion, streams its response. Other responses are returned whole and must complete within 120 seconds.

### Request

GET [https://api.empiriolabs.ai/v1/gpu/connect/\{instance_id}/\{path}](https://api.empiriolabs.ai/v1/gpu/connect/\{instance_id}/\{path})

```curl
curl https://api.empiriolabs.ai/v1/gpu/connect/instance_id/v1%2Fchat%2Fcompletions \
     -H "Authorization: Bearer <token>" \
     -H "Content-Type: application/json"
```

```python
import requests

url = "https://api.empiriolabs.ai/v1/gpu/connect/instance_id/v1%2Fchat%2Fcompletions"

payload = {}
headers = {
    "Authorization": "Bearer <token>",
    "Content-Type": "application/json"
}

response = requests.get(url, json=payload, headers=headers)

print(response.json())
```

```javascript
const url = 'https://api.empiriolabs.ai/v1/gpu/connect/instance_id/v1%2Fchat%2Fcompletions';
const options = {
  method: 'GET',
  headers: {Authorization: 'Bearer <token>', 'Content-Type': 'application/json'},
  body: '{}'
};

try {
  const response = await fetch(url, options);
  const data = await response.json();
  console.log(data);
} catch (error) {
  console.error(error);
}
```

```go
package main

import (
	"fmt"
	"strings"
	"net/http"
	"io"
)

func main() {

	url := "https://api.empiriolabs.ai/v1/gpu/connect/instance_id/v1%2Fchat%2Fcompletions"

	payload := strings.NewReader("{}")

	req, _ := http.NewRequest("GET", url, payload)

	req.Header.Add("Authorization", "Bearer <token>")
	req.Header.Add("Content-Type", "application/json")

	res, _ := http.DefaultClient.Do(req)

	defer res.Body.Close()
	body, _ := io.ReadAll(res.Body)

	fmt.Println(res)
	fmt.Println(string(body))

}
```

```ruby
require 'uri'
require 'net/http'

url = URI("https://api.empiriolabs.ai/v1/gpu/connect/instance_id/v1%2Fchat%2Fcompletions")

http = Net::HTTP.new(url.host, url.port)
http.use_ssl = true

request = Net::HTTP::Get.new(url)
request["Authorization"] = 'Bearer <token>'
request["Content-Type"] = 'application/json'
request.body = "{}"

response = http.request(request)
puts response.read_body
```

```java
import com.mashape.unirest.http.HttpResponse;
import com.mashape.unirest.http.Unirest;

HttpResponse<String> response = Unirest.get("https://api.empiriolabs.ai/v1/gpu/connect/instance_id/v1%2Fchat%2Fcompletions")
  .header("Authorization", "Bearer <token>")
  .header("Content-Type", "application/json")
  .body("{}")
  .asString();
```

```php
<?php
require_once('vendor/autoload.php');

$client = new \GuzzleHttp\Client();

$response = $client->request('GET', 'https://api.empiriolabs.ai/v1/gpu/connect/instance_id/v1%2Fchat%2Fcompletions', [
  'body' => '{}',
  'headers' => [
    'Authorization' => 'Bearer <token>',
    'Content-Type' => 'application/json',
  ],
]);

echo $response->getBody();
```

```csharp
using RestSharp;

var client = new RestClient("https://api.empiriolabs.ai/v1/gpu/connect/instance_id/v1%2Fchat%2Fcompletions");
var request = new RestRequest(Method.GET);
request.AddHeader("Authorization", "Bearer <token>");
request.AddHeader("Content-Type", "application/json");
request.AddParameter("application/json", "{}", ParameterType.RequestBody);
IRestResponse response = client.Execute(request);
```

```swift
import Foundation

let headers = [
  "Authorization": "Bearer <token>",
  "Content-Type": "application/json"
]
let parameters = [] as [String : Any]

let postData = JSONSerialization.data(withJSONObject: parameters, options: [])

let request = NSMutableURLRequest(url: NSURL(string: "https://api.empiriolabs.ai/v1/gpu/connect/instance_id/v1%2Fchat%2Fcompletions")! as URL,
                                        cachePolicy: .useProtocolCachePolicy,
                                    timeoutInterval: 10.0)
request.httpMethod = "GET"
request.allHTTPHeaderFields = headers
request.httpBody = postData as Data

let session = URLSession.shared
let dataTask = session.dataTask(with: request as URLRequest, completionHandler: { (data, response, error) -> Void in
  if (error != nil) {
    print(error as Any)
  } else {
    let httpResponse = response as? HTTPURLResponse
    print(httpResponse)
  }
})

dataTask.resume()
```

For a model deployment, call the OpenAI-compatible endpoint on the instance:

```bash
curl https://api.empiriolabs.ai/v1/gpu/connect/$ID/v1/chat/completions \
  -H "Authorization: Bearer $EMPIRIOLABS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen/Qwen2.5-7B-Instruct",
    "messages": [{ "role": "user", "content": "Hello!" }]
  }'
```

```python
from openai import OpenAI

client = OpenAI(
    base_url="https://api.empiriolabs.ai/v1/gpu/connect/INSTANCE_ID/v1",
    api_key="EMPIRIOLABS_API_KEY",
)

resp = client.chat.completions.create(
    model="Qwen/Qwen2.5-7B-Instruct",
    messages=[{"role": "user", "content": "Hello!"}],
)
print(resp.choices[0].message.content)
```

For JupyterLab, ComfyUI, Web Terminal, or Ollama, open the instance connect URL from the dashboard or send requests to the relevant connect path.

## Chat with your model in the dashboard

When you deploy a model (or any instance that serves an OpenAI-compatible API), the dashboard gives you a built-in chat page so you can try the model right away without writing any code. Open the instance from the **GPU Cloud** page and choose **Chat with this model**. The chat page streams responses, supports a system prompt and the usual sampling controls (temperature, top-p, max tokens), and lets you attach images or audio for multimodal models. It runs against the same authenticated connect path as the API, so there is no extra setup and no separate billing: the instance is already metered by the second.

## Run a command

Run a command inside a running instance's workload container, as root, and get
its output back. This needs only your API key, so a script or an agent can
operate the GPU without an SSH client.

### Request

POST [https://api.empiriolabs.ai/v1/gpu/instances/\{instance_id}/exec](https://api.empiriolabs.ai/v1/gpu/instances/\{instance_id}/exec)

```curl
curl -X POST https://api.empiriolabs.ai/v1/gpu/instances/instance_id/exec \
     -H "Authorization: Bearer <token>" \
     -H "Content-Type: application/json" \
     -d '{
  "command": "nvidia-smi --query-gpu=name,memory.used --format=csv"
}'
```

```python
import requests

url = "https://api.empiriolabs.ai/v1/gpu/instances/instance_id/exec"

payload = { "command": "nvidia-smi --query-gpu=name,memory.used --format=csv" }
headers = {
    "Authorization": "Bearer <token>",
    "Content-Type": "application/json"
}

response = requests.post(url, json=payload, headers=headers)

print(response.json())
```

```javascript
const url = 'https://api.empiriolabs.ai/v1/gpu/instances/instance_id/exec';
const options = {
  method: 'POST',
  headers: {Authorization: 'Bearer <token>', 'Content-Type': 'application/json'},
  body: '{"command":"nvidia-smi --query-gpu=name,memory.used --format=csv"}'
};

try {
  const response = await fetch(url, options);
  const data = await response.json();
  console.log(data);
} catch (error) {
  console.error(error);
}
```

```go
package main

import (
	"fmt"
	"strings"
	"net/http"
	"io"
)

func main() {

	url := "https://api.empiriolabs.ai/v1/gpu/instances/instance_id/exec"

	payload := strings.NewReader("{\n  \"command\": \"nvidia-smi --query-gpu=name,memory.used --format=csv\"\n}")

	req, _ := http.NewRequest("POST", url, payload)

	req.Header.Add("Authorization", "Bearer <token>")
	req.Header.Add("Content-Type", "application/json")

	res, _ := http.DefaultClient.Do(req)

	defer res.Body.Close()
	body, _ := io.ReadAll(res.Body)

	fmt.Println(res)
	fmt.Println(string(body))

}
```

```ruby
require 'uri'
require 'net/http'

url = URI("https://api.empiriolabs.ai/v1/gpu/instances/instance_id/exec")

http = Net::HTTP.new(url.host, url.port)
http.use_ssl = true

request = Net::HTTP::Post.new(url)
request["Authorization"] = 'Bearer <token>'
request["Content-Type"] = 'application/json'
request.body = "{\n  \"command\": \"nvidia-smi --query-gpu=name,memory.used --format=csv\"\n}"

response = http.request(request)
puts response.read_body
```

```java
import com.mashape.unirest.http.HttpResponse;
import com.mashape.unirest.http.Unirest;

HttpResponse<String> response = Unirest.post("https://api.empiriolabs.ai/v1/gpu/instances/instance_id/exec")
  .header("Authorization", "Bearer <token>")
  .header("Content-Type", "application/json")
  .body("{\n  \"command\": \"nvidia-smi --query-gpu=name,memory.used --format=csv\"\n}")
  .asString();
```

```php
<?php
require_once('vendor/autoload.php');

$client = new \GuzzleHttp\Client();

$response = $client->request('POST', 'https://api.empiriolabs.ai/v1/gpu/instances/instance_id/exec', [
  'body' => '{
  "command": "nvidia-smi --query-gpu=name,memory.used --format=csv"
}',
  'headers' => [
    'Authorization' => 'Bearer <token>',
    'Content-Type' => 'application/json',
  ],
]);

echo $response->getBody();
```

```csharp
using RestSharp;

var client = new RestClient("https://api.empiriolabs.ai/v1/gpu/instances/instance_id/exec");
var request = new RestRequest(Method.POST);
request.AddHeader("Authorization", "Bearer <token>");
request.AddHeader("Content-Type", "application/json");
request.AddParameter("application/json", "{\n  \"command\": \"nvidia-smi --query-gpu=name,memory.used --format=csv\"\n}", ParameterType.RequestBody);
IRestResponse response = client.Execute(request);
```

```swift
import Foundation

let headers = [
  "Authorization": "Bearer <token>",
  "Content-Type": "application/json"
]
let parameters = ["command": "nvidia-smi --query-gpu=name,memory.used --format=csv"] as [String : Any]

let postData = JSONSerialization.data(withJSONObject: parameters, options: [])

let request = NSMutableURLRequest(url: NSURL(string: "https://api.empiriolabs.ai/v1/gpu/instances/instance_id/exec")! as URL,
                                        cachePolicy: .useProtocolCachePolicy,
                                    timeoutInterval: 10.0)
request.httpMethod = "POST"
request.allHTTPHeaderFields = headers
request.httpBody = postData as Data

let session = URLSession.shared
let dataTask = session.dataTask(with: request as URLRequest, completionHandler: { (data, response, error) -> Void in
  if (error != nil) {
    print(error as Any)
  } else {
    let httpResponse = response as? HTTPURLResponse
    print(httpResponse)
  }
})

dataTask.resume()
```

```bash
curl https://api.empiriolabs.ai/v1/gpu/instances/<instance_id>/exec \
  -H "Authorization: Bearer $EMPIRIOLABS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"command": "nvidia-smi --query-gpu=name,memory.used --format=csv", "timeout_seconds": 60}'
```

```json
{
  "exit_code": 0,
  "stdout": "name, memory.used [MiB]\nNVIDIA RTX A6000, 1 MiB\n",
  "stderr": "",
  "stdout_truncated": false,
  "stderr_truncated": false,
  "timed_out": false,
  "duration_ms": 812
}
```

Each call starts a fresh shell, so chain steps with `&&` or run a script. Stdout
and stderr are returned up to 1 MiB each. A command stops after
`timeout_seconds`, up to 300. For longer work, start it in the background, for
example `nohup python train.py > train.log 2>&1 &`, and read the log with a later
call. An instance whose workload does not accept commands returns `409`. The MCP
server offers the same as the `run_gpu_command` tool.

## SSH and shell access

SSH works on GPU instances while they are `running`, with any standard OpenSSH
client. A session opens a root shell inside your workload, with your files,
processes and GPUs. Each node of a cluster is an ordinary instance, so you
connect to any node with its own instance ID.

**Set up once.** On the **GPU Cloud** page, open **SSH keys**:

* **Create one for me.** We create a key and give you one setup command for
  macOS, Linux or Windows. Paste it into a terminal once. It saves the key as
  `~/.ssh/empiriolabs_ed25519` and adds the entries below to `~/.ssh/config`.
* **Or add a key you already have.** Paste the contents of
  `~/.ssh/id_ed25519.pub`. To create one yourself, run `ssh-keygen -t ed25519`
  and press Enter at each prompt.

Keys can also be added with `POST /v1/gpu/ssh-keys`:

### Request

POST [https://api.empiriolabs.ai/v1/gpu/ssh-keys](https://api.empiriolabs.ai/v1/gpu/ssh-keys)

```curl
curl -X POST https://api.empiriolabs.ai/v1/gpu/ssh-keys \
     -H "Authorization: Bearer <token>" \
     -H "Content-Type: application/json" \
     -d '{}'
```

```python
import requests

url = "https://api.empiriolabs.ai/v1/gpu/ssh-keys"

payload = {}
headers = {
    "Authorization": "Bearer <token>",
    "Content-Type": "application/json"
}

response = requests.post(url, json=payload, headers=headers)

print(response.json())
```

```javascript
const url = 'https://api.empiriolabs.ai/v1/gpu/ssh-keys';
const options = {
  method: 'POST',
  headers: {Authorization: 'Bearer <token>', 'Content-Type': 'application/json'},
  body: '{}'
};

try {
  const response = await fetch(url, options);
  const data = await response.json();
  console.log(data);
} catch (error) {
  console.error(error);
}
```

```go
package main

import (
	"fmt"
	"strings"
	"net/http"
	"io"
)

func main() {

	url := "https://api.empiriolabs.ai/v1/gpu/ssh-keys"

	payload := strings.NewReader("{}")

	req, _ := http.NewRequest("POST", url, payload)

	req.Header.Add("Authorization", "Bearer <token>")
	req.Header.Add("Content-Type", "application/json")

	res, _ := http.DefaultClient.Do(req)

	defer res.Body.Close()
	body, _ := io.ReadAll(res.Body)

	fmt.Println(res)
	fmt.Println(string(body))

}
```

```ruby
require 'uri'
require 'net/http'

url = URI("https://api.empiriolabs.ai/v1/gpu/ssh-keys")

http = Net::HTTP.new(url.host, url.port)
http.use_ssl = true

request = Net::HTTP::Post.new(url)
request["Authorization"] = 'Bearer <token>'
request["Content-Type"] = 'application/json'
request.body = "{}"

response = http.request(request)
puts response.read_body
```

```java
import com.mashape.unirest.http.HttpResponse;
import com.mashape.unirest.http.Unirest;

HttpResponse<String> response = Unirest.post("https://api.empiriolabs.ai/v1/gpu/ssh-keys")
  .header("Authorization", "Bearer <token>")
  .header("Content-Type", "application/json")
  .body("{}")
  .asString();
```

```php
<?php
require_once('vendor/autoload.php');

$client = new \GuzzleHttp\Client();

$response = $client->request('POST', 'https://api.empiriolabs.ai/v1/gpu/ssh-keys', [
  'body' => '{}',
  'headers' => [
    'Authorization' => 'Bearer <token>',
    'Content-Type' => 'application/json',
  ],
]);

echo $response->getBody();
```

```csharp
using RestSharp;

var client = new RestClient("https://api.empiriolabs.ai/v1/gpu/ssh-keys");
var request = new RestRequest(Method.POST);
request.AddHeader("Authorization", "Bearer <token>");
request.AddHeader("Content-Type", "application/json");
request.AddParameter("application/json", "{}", ParameterType.RequestBody);
IRestResponse response = client.Execute(request);
```

```swift
import Foundation

let headers = [
  "Authorization": "Bearer <token>",
  "Content-Type": "application/json"
]
let parameters = [] as [String : Any]

let postData = JSONSerialization.data(withJSONObject: parameters, options: [])

let request = NSMutableURLRequest(url: NSURL(string: "https://api.empiriolabs.ai/v1/gpu/ssh-keys")! as URL,
                                        cachePolicy: .useProtocolCachePolicy,
                                    timeoutInterval: 10.0)
request.httpMethod = "POST"
request.allHTTPHeaderFields = headers
request.httpBody = postData as Data

let session = URLSession.shared
let dataTask = session.dataTask(with: request as URLRequest, completionHandler: { (data, response, error) -> Void in
  if (error != nil) {
    print(error as Any)
  } else {
    let httpResponse = response as? HTTPURLResponse
    print(httpResponse)
  }
})

dataTask.resume()
```

**Connect.** Copy the SSH command from the instance card, or read `ssh.command`
from `GET /v1/gpu/instances/{instance_id}`:

```sh
ssh -J ssh.empiriolabs.ai:<port> root@gpu-<instance_id>
```

After the one-time setup, `ssh gpu-<instance_id>` works too. The first
connection to an instance can take a minute. The first time, your client asks
you to confirm two host keys: `ssh.empiriolabs.ai` and the instance. Ed25519 and
RSA keys (2048 to 8192 bits) are supported. The image needs Debian, Ubuntu, or
Alpine with a working package manager, unless it already includes an SSH server.
An instance that does not offer SSH returns `ssh: null` and shows no SSH command.

**Copy files.** `scp` and `sftp` use the same connection, and so does `rsync`
when the image includes it. Copy into any path in the container, such as
`/root`; `/workspace` is present when a volume is attached or the image creates
it:

```sh
scp -J ssh.empiriolabs.ai:<port> ./train.py root@gpu-<instance_id>:/root/
rsync -av -e "ssh -J ssh.empiriolabs.ai:<port>" ./data/ root@gpu-<instance_id>:/root/data/
```

For large datasets, download them directly on the instance or keep them in a
[volume](#persistent-volumes).

**The setup, and editors.** The one-time setup adds these entries to
`~/.ssh/config`. Editors that read that file, such as VS Code and Cursor,
connect to `gpu-<instance_id>` with them. With a key at a default name, such as
`~/.ssh/id_ed25519`, keep only the first three lines:

```text
Host gpu-*
  User root
  ProxyJump ssh.empiriolabs.ai:<port>
  IdentityFile ~/.ssh/empiriolabs_ed25519
  IdentitiesOnly yes

Host ssh.empiriolabs.ai
  IdentityFile ~/.ssh/empiriolabs_ed25519
  IdentitiesOnly yes
```

**Create a key through the API.** For automation, send `generate: true` instead
of a public key. The response carries the private key once, as `private_key`. It
is not stored and cannot be shown again. Save it as
`~/.ssh/empiriolabs_ed25519`; with the entries above, `ssh gpu-<instance_id>`
connects. Without them, give the key to both hosts:

```sh
curl -sS https://api.empiriolabs.ai/v1/gpu/ssh-keys \
  -H "Authorization: Bearer $EMPIRIOLABS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"generate": true, "name": "Agent key"}' \
  | jq -r .private_key > ~/.ssh/empiriolabs_ed25519
chmod 600 ~/.ssh/empiriolabs_ed25519
ssh -i ~/.ssh/empiriolabs_ed25519 \
  -o ProxyCommand="ssh -i ~/.ssh/empiriolabs_ed25519 -p <port> -W %h:%p ssh.empiriolabs.ai" \
  root@gpu-<instance_id>
```

**Manage keys.** `GET /v1/gpu/ssh-keys` lists your keys and
`DELETE /v1/gpu/ssh-keys/{key_id}` removes one. A removed key can no longer open
new connections; sessions already open stay open until they end. To prepare SSH
on one instance before your first connection, send your public key to
`POST /v1/gpu/instances/{instance_id}/ssh`, which also adds the key to your account.

**Limits.** Agent, X11, and port forwarding are disabled. SSH adds no separate
charge. Stopping or replacing an instance closes its SSH sessions; closing a
session does not stop the GPU. After you restart or replace an instance, remove
its old host key with `ssh-keygen -R gpu-<instance_id>` before reconnecting. Files
outside an attached volume are temporary. For a shell in the browser, deploy the
Web Terminal template.

## Usage and billing records

The GPU Cloud dashboard shows running spend and lifetime GPU spend. API lifecycle responses include the instance price, GPU count, billing status, and billed amount so you can reconcile usage from your own systems. Billing starts when the instance is allocated, which is before it reports `running`, so use `billed_amount` as the total for an instance.