> ## Documentation Index
> Fetch the complete documentation index at: https://docs.coreweave.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Run a GPU sandbox

> Request GPUs for serverless or CKS placement and compare the isolation of each supported configuration.

This guide shows how to create a sandbox with one or more GPUs, confirm the GPU is visible from inside the sandbox, and set the CPU and memory requests that go with it. GPU sandboxes run in either placement mode: on serverless capacity that CoreWeave operates, or on a CoreWeave Kubernetes Service (CKS) cluster you own. The two modes differ in how the sandbox is isolated, which GPUs you can get, and who sets the limits, so this page covers them in separate sections.

For CPU-only sandboxes, see [Get started](https://docs.coreweave.com/products/sandboxes/get-started.md). To compare serverless and CKS capacity, see [Sandbox placement and spillover](https://docs.coreweave.com/products/sandboxes/placement.md).

> [!NOTE]
>
> GPU sandboxes are in private preview and require your organization to be allowlisted, even if it already uses CPU-only sandboxes. To request access, contact your account team or email [forge-support@coreweave.com](mailto:forge-support@coreweave.com). Until your organization is allowlisted, requests to create GPU sandboxes fail with `CWSANDBOX_GPU_NOT_ALLOWED`.

For concurrent sandbox quotas and resource limits, see [Limits and quotas](https://docs.coreweave.com/products/sandboxes/reference/limits-and-quotas.md).

## Isolation depends on the placement mode

GPU virtualization is supported only on serverless capacity. The supported CKS GPU configuration uses the cluster's default runtime class (`runc`), without virtualization. The comparison and warnings on this page describe that configuration. Kata-based GPU runtime classes on customer CKS clusters aren't supported.

| | Serverless | Your CKS cluster (supported configuration) |
| - | - | - |
| Runtime | GPU virtual machine runtime, selected automatically | The cluster's default runtime class, which runs containers under `runc` |
| GPU virtualization | Yes. GPUs are attached to the sandbox's virtual machine. | No. The sandbox is a standard container that shares the Node's kernel. |
| Secure container runtime | Yes | **No** |
| Suitable for untrusted code | Yes | **No.** Run only code you trust. |

> [!WARNING]
>
> In the supported CKS configuration, GPU sandboxes run under the cluster's default runtime class (`runc`), with no virtualization. They have the same isolation as any other container on the Node and **can't be considered a secure container runtime**. Don't use CKS GPU sandboxes to run untrusted code, such as code generated by a model or submitted by users. To run untrusted code on a GPU, use [serverless capacity](#run-a-gpu-sandbox-on-serverless-capacity).

## Before you begin

You need the following:

* GPU sandboxes enabled for your organization.
* A Forge API key or CoreWeave API access token for an account with serverless sandbox access for the serverless examples. CKS placement requires a CoreWeave API access token with the `sandbox_user` Identity and Access Management (IAM) action. See [Choose and get credentials](https://docs.coreweave.com/products/sandboxes/placement.md#choose-and-get-credentials).

The TypeScript client doesn't expose GPU resources yet. Use the Python client for GPU sandboxes.

## Choose an image for your workload

A runtime image can run prebuilt GPU libraries but might omit build tools needed by your application. Check the exact image and package versions before downloading model weights. A successful import or model load doesn't test just-in-time (JIT) compilation at the first kernel launch.

| Workload step | Check in the image |
| - | - |
| Install a Git-pinned Python dependency | The `git` executable is available. |
| Compile host code for a Triton kernel | A C compiler and development headers are available, such as `gcc` and `libc6-dev` on Ubuntu. |
| Compile CUDA kernels, including stacks that use FlashInfer | The required CUDA toolkit and `nvcc` are available, or compatible precompiled kernels are installed. Requirements depend on package versions and the GPU architecture. |

We recommend including system tools in a pinned custom image. A framework's development image can provide a starting point. Verify its contents and compatibility with your stack. Before building a large snapshot, test dependency installation, one representative kernel or inference request, and server startup on one sandbox.

File System Snapshots capture files in the configured mount. They don't capture packages installed into the container root filesystem with `apt-get`. Include those packages in the image so they're available after a restore. Python environments and precompiled kernels can persist when stored inside the snapshot mount. If a kernel installer invokes `pip`, check that `pip` is installed in that environment. An environment created with `uv` might not include it.

## Choose authentication

Choose the credential for the account that owns your sandboxes. To obtain a Forge API key or CoreWeave API access token, see [Choose and get credentials](https://docs.coreweave.com/products/sandboxes/placement.md#choose-and-get-credentials).

Run the installation commands in an [active Python virtual environment](https://docs.coreweave.com/products/sandboxes/client.md#prepare-your-python-environment).

Save the Python code from your credential tab as a `sandbox_auth.py` file beside the Python script that uses it. This file stores only the authentication choice. Import `AUTH` from that file. When you create a sandbox or session, pass `AUTH` explicitly or set it in `SandboxDefaults`.

> [tab] Forge API key
>
> Use your Forge API key with the `wandb` integration in the software development kit (SDK). Replace `[WANDB-API-KEY]` with your API key:
>
> ```bash theme={"system"}
> uv pip install 'cwsandbox[wandb]>=1.14.2'
> export WANDB_API_KEY="[WANDB-API-KEY]"
> ```
>
> ```python title="sandbox_auth.py" theme={"system"}
> from cwsandbox import AuthStrategy
>
> AUTH = AuthStrategy.WANDB
> ```

> [tab] CoreWeave API access token
>
> Use the CoreWeave API access token for your organization. Replace `[API-ACCESS-TOKEN]` with your token:
>
> ```bash theme={"system"}
> uv pip install 'cwsandbox>=1.14.2'
> export CWSANDBOX_API_KEY="[API-ACCESS-TOKEN]"
> ```
>
> ```python title="sandbox_auth.py" theme={"system"}
> from cwsandbox import AuthStrategy
>
> AUTH = AuthStrategy.COREWEAVE_API_KEY
> ```

The examples use serverless capacity unless a section specifies CoreWeave Kubernetes Service (CKS). For CKS placement, choose a CoreWeave API access token and [enable a runner](https://docs.coreweave.com/products/sandboxes/get-started-cks.md). Set `placement_mode="cks"` when you create a sandbox or in `SandboxDefaults`.

To authenticate a session and the sandboxes it owns, create the session with `auth=AUTH`. When you reconnect, list, or clean up sandboxes, keep the same credential.

## How GPU requests work

The same rules apply in both placement modes:

* **A GPU request reserves GPUs and nothing else.** Set CPU and memory explicitly, sized for the work you'll run alongside the GPU. Omitted CPU and memory values can use the policy's `defaultCpu` and `defaultMemory`, but the resolved allocation must still satisfy the policy's resource requirements.
* **Sandboxes get whole GPUs.** GPUs aren't shared, partitioned, or time-sliced between sandboxes.
* **A sandbox runs on a single Node.** In either placement mode, all requested GPUs, CPU, and memory must be available on one Node at the same time. The request must also satisfy your organization's GPUs-per-sandbox limit and the applicable policy.
* **Your organization has a GPUs-per-sandbox limit.** It defaults to 1 and applies to both serverless and CKS placement. A request above it fails with `CWSANDBOX_GPU_COUNT_EXCEEDED` before placement and doesn't trigger spillover. For error details and how to discuss a higher limit, see [GPUs per sandbox](https://docs.coreweave.com/products/sandboxes/reference/limits-and-quotas.md#gpus-per-sandbox).
* **The GPU type is a filter, not a menu.** The optional `type` key must match exactly, including case, one of the GPU types the runner advertises. Omit it to accept any GPU the runner has.

## Run a GPU sandbox on serverless capacity

Serverless placement needs no runner and no policy of your own. CoreWeave owns the policy and the hardware. GPU sandboxes run in a virtual machine, so they suit untrusted code.

### Available GPUs

Serverless capacity runs one GPU model, the NVIDIA RTX PRO 6000 Blackwell Server Edition. Your organization's [GPUs-per-sandbox limit](https://docs.coreweave.com/products/sandboxes/reference/limits-and-quotas.md#gpus-per-sandbox) and the serverless policy constrain the GPU count. The request must also fit the available capacity on a single Node.

| GPU | GPU memory | GPUs per sandbox |
| - | - | - |
| NVIDIA RTX PRO 6000 Blackwell Server Edition | 96 GB | Up to your organization's limit (default: 1), subject to policy and available capacity. |

Two instance types carry it: [High Memory](https://docs.coreweave.com/platform/instances/gpu/rtxp6000-8x.md) and [Standard Memory](https://docs.coreweave.com/platform/instances/gpu/rtxp6000-8x-v2.md). They differ in host RAM, not in the GPU, and both present the same GPU type to a sandbox, so which one a sandbox lands on isn't something you select.

Leave the `type` key out of the GPU request so the platform assigns a GPU from CoreWeave-managed compute. The field is still accepted here, but it filters rather than selects: a `type` that matches no runner fails with `CWSANDBOX_RUNNER_UNAVAILABLE`. A runner that receives an unsupported type rejects it with `CWSANDBOX_PLACEMENT_CONSTRAINT_UNSATISFIED`.

Use `resources` to choose CPU and memory alongside the GPU count, as shown in the following example.

Disk is requested separately from CPU, memory, and GPU rather than alongside them: `ResourceOptions` has no disk field. The container's root filesystem is Node-local ephemeral storage that the sandbox doesn't reserve a share of, so `df` inside the sandbox reports the Node's filesystem rather than a per-sandbox quota. For a dedicated writable path, declare a scratch volume:

```python theme={"system"}
from cwsandbox import Sandbox, ScratchVolumeOptions
from sandbox_auth import AUTH

with Sandbox.run(
    auth=AUTH,
    placement_mode="serverless",
    resources={"cpu": "2", "memory": "8Gi", "gpu": 1},
    volumes=[ScratchVolumeOptions(name="work", mount_path="/work", size="20Gi")],
    max_lifetime_seconds=3600,
) as sandbox:
    result = sandbox.exec(["df", "-h", "/work"]).result()
    print(result.stdout)
```

A disk-backed volume, the default, draws on the same Node-local storage as the root filesystem. Set `medium="memory"` for a tmpfs instead: a memory-backed volume must declare a size, and the memory-backed volumes on one container can't total more than 80% of its memory request.

Leave `runtime_class` unset too. A GPU request selects the GPU virtual machine runtime class on its own, and a runtime class you pin is used exactly as given, so pinning the CPU class alongside a GPU request produces a sandbox that can't reach the GPU.

### Create the sandbox

Complete [Choose authentication](#choose-authentication), then run the following example. It creates a sandbox with 1 GPU, 2 CPUs, and 8 GiB of memory, then prints the GPU that `nvidia-smi` reports.

> [tab] Python
>
> This example uses the credential selected in `sandbox_auth.py`.
>
> ```python theme={"system"}
> from cwsandbox import Sandbox
> from sandbox_auth import AUTH
>
> with Sandbox.run(
>     auth=AUTH,
>     placement_mode="serverless",
>     resources={"cpu": "2", "memory": "8Gi", "gpu": 1},
>     max_lifetime_seconds=3600,
> ) as sandbox:
>     result = sandbox.exec(
>         ["nvidia-smi", "--query-gpu=name,memory.total", "--format=csv"]
>     ).result()
>     print(result.stdout)
>     print(sandbox.resource_gpu)
> ```
>
> The flat `resources` dict sets requests and limits to the same values. `"gpu": 1` is shorthand for `"gpu": {"count": 1}`. This example requests 1 GPU to fit the default GPUs-per-sandbox limit. Before requesting more, confirm your organization's limit and the capacity required by your workload.
>
> The `resource_gpu` property returns the GPU allocation the platform confirmed, such as `{'count': 1}`.

> [tab] TypeScript
>
> The `@coreweave/cwsandbox` package doesn't accept GPU resources yet. Its `resources` option covers CPU and memory only, and a `gpu` key is ignored, so the sandbox starts without a GPU. Use the Python client to create GPU sandboxes.

Sample output:

```text theme={"system"}
name, memory.total [MiB]
NVIDIA RTX PRO 6000 Blackwell Server Edition, 97887 MiB

{'count': 1}
```

GPU sandboxes take longer to start than CPU-only sandboxes because the platform attaches the GPUs to the sandbox's virtual machine. Allow several minutes for startup. To allow up to 10 minutes for startup, call `sandbox.wait(timeout=600.0)` before the first operation. If you omit `timeout`, the SDK uses `request_timeout_seconds` for the startup wait (300 seconds by default). This includes the implicit wait before `exec()`.

`request_timeout_seconds` also bounds most individual remote procedure calls (RPCs). Status polling uses `poll_rpc_timeout_seconds` (15 seconds by default) for each RPC. The `timeout_seconds` parameter on `exec()` applies after startup. For the timeout phases and how the sandbox's maximum lifetime includes startup, see [Timeout phases](https://docs.coreweave.com/products/sandboxes/client/guides/sandbox-lifecycle.md#timeout-phases).

## Run a GPU sandbox on your CKS cluster

On CKS placement, your administrators decide which GPUs sandboxes can use and how many. The runner offers the GPU types present on the cluster's Nodes, and the policy it carries bounds what a sandbox may request.

> [!WARNING]
>
> GPU virtualization isn't supported on your own CKS cluster. In the supported CKS configuration, GPU sandboxes run under the cluster's default runtime class (`runc`), with no virtualization. The sandbox is an ordinary container that shares the Node's kernel and has direct access to the GPU devices assigned to it. This mode **isn't a secure container runtime**: don't use it to run untrusted code. Use it only for workloads you'd already be comfortable running as a regular Pod on the cluster. For untrusted code, use [serverless capacity](#run-a-gpu-sandbox-on-serverless-capacity).

### Prerequisites

Complete the following before you request a GPU on CKS:

* A CKS cluster with GPU Nodes and a runner in the `Ready` state. See [Use your own compute](https://docs.coreweave.com/products/sandboxes/get-started-cks.md).
* A policy on that runner that permits GPUs. `maxGpuCount` under `resources` caps how many GPUs each container may request: leave it out for no cap, or set it to `0` to reject GPU requests. Leave `defaultGpuRuntimeClass` unset so GPU sandboxes run under the cluster's default runtime class. Kata-based GPU runtime classes, such as `cw-kata-nvidia-gpu`, aren't supported on customer CKS clusters. For every field, see [Configure a sandbox policy](https://docs.coreweave.com/products/sandboxes/profiles/configure.md).
* The `sandbox_user` IAM action and a CoreWeave API access token, set as `CWSANDBOX_API_KEY`.

### Find the GPU types on your runner

Each runner advertises the GPU types its Nodes carry. List them before you choose a `type`:

```python theme={"system"}
from cwsandbox import AuthStrategy, list_runners

for runner in list_runners(auth=AuthStrategy.COREWEAVE_API_KEY):
    print(runner.runner_id, runner.supported_gpu_types, runner.max_gpu_count)
```

The values are the exact strings to pass as `type`. Matching is case-sensitive, so `B200` and `b200` aren't the same type. To filter runners by GPU type instead, pass `gpu_type` to `list_runners()`. For more, see [Discover available runners](https://docs.coreweave.com/products/sandboxes/client/guides/discovery.md).

### Create the sandbox

The following example places a sandbox on CKS with 2 GPUs of a specific type, 8 CPUs, and 32 GiB of memory. To run it, your organization's maximum must be at least 2 GPUs per sandbox. Replace `[GPU-TYPE]` with one of the types your runner advertises, or drop the `type` key to accept any GPU on the runner. Leave `runtime_class` unset so the sandbox uses the cluster's default runtime class.

```python theme={"system"}
from cwsandbox import AuthStrategy, ResourceOptions, Sandbox

with Sandbox.run(
    auth=AuthStrategy.COREWEAVE_API_KEY,
    placement_mode="cks",
    resources=ResourceOptions(
        requests={"cpu": "8", "memory": "32Gi"},
        limits={"cpu": "8", "memory": "32Gi"},
        gpu={"count": 2, "type": "[GPU-TYPE]"},
    ),
    max_lifetime_seconds=3600,
) as sandbox:
    result = sandbox.exec(["nvidia-smi", "-L"]).result()
    print(result.stdout)
    print(sandbox.resource_gpu)
```

To target one runner rather than any CKS runner in your organization, add `runner_ids=["[RUNNER-ID]"]`. The confirmed allocation in `resource_gpu` reports the count only, such as `{'count': 2}`, even when you requested a type.

The GPU count is capped by your organization's maximum, by the policy's `maxGpuCount`, and by the number of free GPUs on a single Node. A request above your organization's maximum fails with `CWSANDBOX_GPU_COUNT_EXCEEDED`, and a request above the policy cap fails with `CWSANDBOX_RESOURCE_CEILING_EXCEEDED`. A `type` the runner doesn't offer, or one with a different case, finds no eligible runner and fails with `CWSANDBOX_RUNNER_UNAVAILABLE`.

## Container images

The following example uses serverless capacity and the credential selected in `sandbox_auth.py`. On CKS, use a CoreWeave token, set `placement_mode="cks"`, and choose an image compatible with your cluster's GPU.

The platform provides the NVIDIA driver and the `nvidia-smi` tool inside a GPU sandbox, so the default image can already see the GPU. To run CUDA applications, use an image that ships the CUDA runtime and libraries your code needs, such as a `pytorch/pytorch` or `nvidia/cuda` image.

Match the image to the GPU. The RTX PRO 6000 Blackwell Server Edition has [compute capability 12.0](https://developer.nvidia.com/cuda/gpus) (`sm_120`). [PyTorch 2.7 introduced Blackwell support](https://pytorch.org/blog/pytorch-2-7/) with CUDA 12.8 builds.

A framework can report the GPU's name even when it can't execute CUDA kernels. Kernel execution requires compatible GPU binary code or forward-compatible Parallel Thread Execution (PTX) code. See [NVIDIA's Blackwell compatibility guide](https://docs.nvidia.com/cuda/blackwell-compatibility-guide/index.html). An older image's version numbers alone don't establish whether it can run your workload.

To check basic CUDA execution on the allocated GPU, inspect the framework's compiled architectures, then execute and synchronize a CUDA tensor operation. The `check=True` argument raises `SandboxExecutionError` if the command exits with a nonzero status:

```python theme={"system"}
from cwsandbox import Sandbox
from sandbox_auth import AUTH

CHECK_GPU = """
import torch

print(torch.cuda.get_device_capability(0))
print(torch.cuda.get_arch_list())

x = torch.ones(32, device="cuda")
print((x + x).sum().item())
torch.cuda.synchronize()
"""

with Sandbox.run(
    auth=AUTH,
    placement_mode="serverless",
    container_image="pytorch/pytorch:2.8.0-cuda12.8-cudnn9-runtime",
    resources={"cpu": "2", "memory": "8Gi", "gpu": 1},
) as sandbox:
    result = sandbox.exec(["python", "-c", CHECK_GPU], check=True).result()
    print(result.stdout)
```

Sample output:

```text theme={"system"}
(12, 0)
['sm_70', 'sm_75', 'sm_80', 'sm_86', 'sm_90', 'sm_100', 'sm_120']
64.0
```

A large framework image takes longer to pull than the default image, so allow a few minutes for the sandbox to become ready.

## Common errors

| Error | Cause | What to do |
| - | - | - |
| `CWSANDBOX_GPU_NOT_ALLOWED` | Your organization is not allowlisted for GPU sandboxes. | Contact your account team or [forge-support@coreweave.com](mailto:forge-support@coreweave.com) to request access. |
| `CWSANDBOX_GPU_COUNT_EXCEEDED` | The request asks for more GPUs than your organization's maximum per sandbox, which is 1 unless CoreWeave raises it. | Lower the count to fit your organization's limit. To discuss a higher limit for either placement mode, see [GPUs per sandbox](https://docs.coreweave.com/products/sandboxes/reference/limits-and-quotas.md#gpus-per-sandbox). |
| `SandboxResourceExhaustedError` (`runner capacity exhausted`) | No Node has enough free GPUs, CPU, or memory for the request right now. | Retry after a short wait. On CKS, you can also request fewer GPUs. |
| `CWSANDBOX_RUNNER_UNAVAILABLE` (`no eligible runner is available`) | No runner matches the request. One cause is a GPU `type` that no runner offers, including a type with the wrong case. The same code also appears when no eligible runner is connected right now. | On serverless, remove `type`. On CKS, use a type from `supported_gpu_types`, or drop `type`. If `type` is already correct, retry. |
| `CWSANDBOX_RESOURCE_CEILING_EXCEEDED` (`gpu count exceeds the policy maximum`) | The request asks for more GPUs than the policy's `maxGpuCount`. | Lower the count, or ask an administrator to raise the cap. |
| `nvidia-smi: not found` | The sandbox has no GPU, or its image doesn't expose the NVIDIA tools. | Check `sandbox.resource_gpu` first. If it reports a positive count, the GPUs are allocated and the image is what to change. |

## Next steps

* [Sandbox configuration](https://docs.coreweave.com/products/sandboxes/client/guides/sandbox-configuration.md) covers every `ResourceOptions` field, QoS classes, and timeouts.
* [Configure a sandbox policy](https://docs.coreweave.com/products/sandboxes/profiles/configure.md) explains the GPU, runtime class, and resource constraints administrators set on a CKS runner.
* [Get started](https://docs.coreweave.com/products/sandboxes/get-started.md) covers CPU-only serverless sandboxes and credentials.
* [Use your own compute](https://docs.coreweave.com/products/sandboxes/get-started-cks.md) covers runner setup and your first sandbox on a cluster.
