> ## Documentation Index
> Fetch the complete documentation index at: https://docs.coreweave.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Datasets

> Build reproducible train and test data from project traces or an existing W&B project.

A dataset is a durable snapshot of usable model interactions. Model Distillation stores entries in Postgres for inspection and evaluation, then exports immutable artifacts when training starts.

> [card] [Create your first dataset](https://docs.coreweave.com/model-distillation/studio/datasets-quickstart.md)
>
> Follow the step-by-step guide from source selection through row inspection.

## Choose a source

The UI builds datasets from the project's own traffic. The API also accepts an existing W\&B project as a source, or examples that you write or upload yourself. The following sections describe all three.

### Project traffic

When the proxy recorded the traffic, use the current project's traces. In the UI, the **Source data** section of the **Create Dataset** dialog lets you choose the following settings:

* **Project version:** Pins the prompt and routing regime used to select traces.
* **Served by:** Keeps only traces that one model served. To keep training data teacher-only, select the teacher. **All models** mixes the outputs of every model that served the version.
* **After (UTC)** and **Before (UTC):** A fixed time window. The UI prefills the last 14 days as fixed timestamps.
* **Maximum traces:** The random sample size after filtering. The default is 20,000 and the maximum is 100,000.

Through the API, `filters` also accepts scalar trace metadata. Project-source reads use project-attributed Agent Spans and reconstruct the exact provider-shaped Chat Completions request and response.

### Existing W\&B project

To import compatible self-logged Weave calls that don't have Model Distillation project identity, set `source` to an object with `entity` and `project` in the API request instead of the string `"task"`. This source isn't available in the UI dialog.

### Your own examples

To bring in examples that were never logged to W\&B, create an empty dataset through the API, then write rows to it in batches or upload a JSONL file. Model Distillation validates every row before it changes the dataset. See [Upload a dataset](https://docs.coreweave.com/model-distillation/studio/datasets-upload.md).

## Sampling and split

Whichever source you choose, the following settings control how many rows the dataset contains and how those rows are divided between training and test.

* **Maximum traces** caps the random sample after filtering.
* **Train / test split** reserves test rows for evaluations and model comparisons. The remaining rows are training data. The default is 90% training and 10% test.

Split assignment is deterministic and grouped by scenario, agent run, conversation, or trace. Related examples don't leak across train and test, and Model Distillation deduplicates inputs automatically.

> [!NOTE]
>
> Use a fixed date range. A moving window would make the same dataset configuration produce different training data later.

## Inspect and filter rows

The dataset workbench shows messages, original output, relabeled output, comparison-model output, token estimates, split, and per-row evaluation results. Filters support the following criteria:

* Text in input or original output
* Train or test split
* Relabel status
* Whether an evaluation was applied

Delete invalid entries before training. Deleting a row doesn't alter the source trace.

## Append new traffic

To grow a dataset incrementally, use **Append** with a later fixed range. Model Distillation deduplicates entries by trace and turn. Appending increments the dataset revision and makes older relabel or evaluation results stale when their coverage no longer matches.

## What is excluded

Errored traces, incomplete streams, invalid Chat Completions shapes, and traces whose fidelity mapping is incomplete aren't eligible for training.

## Delete a dataset

You can delete a dataset in the UI or through the API. If fine-tunes or evaluations depend on a dataset, you can delete it together with its dependents, but you can't delete it by itself.

Queued or running fine-tunes, models with active routing, and active evaluations block deletion until they finish or you remove them. An import or relabel run in progress doesn't block deletion. It stops at its next batch, and Model Distillation discards any results that arrive after deletion begins.

> [!WARNING]
>
> If you delete a dataset together with its dependents, the deletion also destroys the dependent fine-tunes' training artifacts and any results that use those models.

To delete a dataset in the UI, follow these steps:

1. Open the dataset's **Settings** and select **Delete dataset**.
2. If the dataset has dependents, select **Also delete the dependent fine-tunes and affected evaluations** and review the listed impact.
3. To confirm, type the dataset name.
4. If the dataset has dependents, select **Delete Dataset and Dependents**. Otherwise, select **Delete Dataset**.

After you confirm, deletion runs in the background. If it fails before destructive work begins, the dataset becomes available again. If it fails afterward, the dataset stays locked until a retry succeeds. To retry, confirm the deletion again. Retries stay within the scope that the original request recorded.

> [accordion] API: Delete a dataset (DELETE /projects/\{alias}/datasets/\{datasetId})
>
> Deletion takes one request, and the server determines the dependents and blockers itself. To also delete the dependent fine-tunes and affected evaluations that exist when the server accepts the request, set `cascade=true`. Without it, a dataset with dependents returns `409 Conflict` with type `dataset_in_use`. The following request deletes a dataset together with its dependents:
>
> ```bash theme={"system"}
> curl --request DELETE \
>   --url "https://forge.coreweave.com/api/distillation/v1/projects/ticket-classifier/datasets/$DATASET_ID?cascade=true" \
>   --header "Authorization: Bearer $WANDB_API_KEY" \
>   --header "Forge-Entity: your-team"
> ```
>
> The request returns `202 Accepted` with the deletion operation and its status. If you repeat the request, it returns the same operation while that operation is queued, running, or completed. A repeated request also retries failed cleanup within the scope that the original request recorded. When the dataset is already gone and no operation exists, the request returns `204 No Content`, and `GET` on the dataset returns `404 Not Found`.
>
> To preview the dependents and blockers before deleting, call `GET /projects/{alias}/datasets/{datasetId}/deletion-plan`. The preview is informational and also reports the status of a running deletion. For reference details, see [Delete a dataset](https://docs.coreweave.com/model-distillation/reference/management/datasets/delete-a-dataset.md) and [Preview dataset deletion](https://docs.coreweave.com/model-distillation/reference/management/datasets/preview-dataset-deletion.md).

> [accordion] API: List datasets for a project (GET /projects/\{alias}/datasets)
>
> Agents can discover ready dataset IDs before starting relabeling, training, or evaluation:
>
> ```bash theme={"system"}
> curl --url "https://forge.coreweave.com/api/distillation/v1/projects/ticket-classifier/datasets" \
>   --header "Authorization: Bearer $WANDB_API_KEY" \
>   --header "Forge-Entity: your-team"
> ```
>
> To create or append data, follow the copyable request in [Datasets Quick Start](https://docs.coreweave.com/model-distillation/studio/datasets-quickstart.md). For reference details about this operation, see [List datasets](https://docs.coreweave.com/model-distillation/reference/management/datasets/list-datasets.md).
