> ## Documentation Index
> Fetch the complete documentation index at: https://docs.coreweave.com/llms.txt
> Use this file to discover all available pages before exploring further.

> Start sweep agents and an optional local scheduler to run your hyperparameter search.

# Start a sweep agent

This page explains how to start a sweep agent on one or more machines so that Weights & Biases can run your hyperparameter search. Sweep agents use the sweep configuration that you define when you [initialize a sweep](https://docs.coreweave.com/products/wandb/sweeps/initialize-sweeps.md) to explore different hyperparameter combinations. Weights & Biases creates a new run for each hyperparameter combination that the sweep agent runs.

See [Manage sweeps](https://docs.coreweave.com/products/wandb/sweeps/pause-resume-and-cancel-sweeps.md) to learn how to pause, resume, stop, or cancel a running sweep agent.

Before you continue:

* Configure your training script to create and track hyperparameter combinations with Weights & Biases. See [Add Weights & Biases to your code](https://docs.coreweave.com/products/wandb/sweeps/add-w-and-b-to-your-code.md) for examples.
* Define a [configuration file](https://docs.coreweave.com/products/wandb/sweeps/define-sweep-configuration.md) for your sweep.
* [Initialize the sweep](https://docs.coreweave.com/products/wandb/sweeps/initialize-sweeps.md) and note its sweep ID.

## Start a scheduler

If you specify a custom scheduler in your sweep configuration, you need to start the scheduler before launching any agents.

> [!TIP]
>
> Sweeps that use the default hosted controller don't require starting a local scheduler. Only start `wandb sweep-scheduler` if your sweep configuration includes a custom `scheduler`.

In the scheduler environment, [install the dependencies for your engine](https://docs.coreweave.com/products/wandb/sweeps/configure-engine.md#install-dependencies). If you configured `scheduler.source`, make the source file and its dependencies available there, and run the scheduler from a directory where the relative source path resolves.

Replace `ENTITY`, `PROJECT`, and `SWEEP_ID` with the entity, project, and ID of your initialized sweep:

```bash theme={"system"}
wandb sweep-scheduler ENTITY/PROJECT/SWEEP_ID
```

Use one scheduler per sweep. Multiple schedulers for the same sweep don't share optimizer state and can generate duplicate runs. Keep the scheduler running as a separate process while agents execute your training code.

| Option | Description |
| - | - |
| `--batch-size` | Number of unfinished runs to keep available for agents. For example, if nine runs are running and none are queued, the scheduler generates one more. |
| `--poll-interval` | Seconds between polls of the sweep's runs. |

For all command options and default values, see [`wandb sweep-scheduler`](https://docs.coreweave.com/products/wandb/ref/cli/wandb-sweep-scheduler).

The scheduler respects [sweep lifecycle commands](https://docs.coreweave.com/products/wandb/sweeps/pause-resume-and-cancel-sweeps.md), including pausing, resuming, and stopping the sweep. If you kill or delete the sweep, the scheduler exits.

### Restart a scheduler

To restart a scheduler, run `wandb sweep-scheduler` with the same sweep path. The scheduler reads the existing run data to initialize the engine, then continues optimization. This restores observations from Weights & Biases rather than a saved optimizer checkpoint.

If you use a custom optimizer factory, see [Handle scheduler restarts](https://docs.coreweave.com/products/wandb/sweeps/configure-engine.md#handle-scheduler-restarts) for requirements on optimizer state and source files.

## Start an agent

Each agent environment requires the Weights & Biases Python SDK (`wandb`), your training program, and its dependencies. If you started a scheduler, start agents in separate terminals or environments. Start additional agents with the same sweep path to evaluate configurations concurrently. See [Parallelize agents](https://docs.coreweave.com/products/wandb/sweeps/parallelize-agents.md).

These examples show how to start a sweep agent with the W\&B CLI or from Python. You need the sweep ID that Weights & Biases returns when you initialize the sweep. You can use the short sweep ID on its own (for example, `dtzl1o7u`), or include the entity and project as a path:

```bash theme={"system"}
entity/project/sweep_ID
```

Where:

* `entity`: Your Weights & Biases team name.
* `project`: The name of the project where you want Weights & Biases to store the output of the run. If you don't specify the project, Weights & Biases puts the run in a project called "Uncategorized".
* `sweep_ID`: The pseudo-random, unique ID that Weights & Biases generates.

> [tab] CLI
>
> Use the `wandb agent` command to start a sweep agent. Provide the sweep ID that Weights & Biases returns when you initialize the sweep.
>
> Replace `[SWEEP-ID]` with your sweep ID in the following command:
>
> ```bash theme={"system"}
> wandb agent [SWEEP-ID]
> ```
>
> For graceful shutdown when you interrupt the agent (for example, with `Ctrl+C`), use `wandb agent --forward-signals [SWEEP-ID]` so the current run receives the signal and can shut down cleanly. See [Signal handling and sweep runs](https://docs.coreweave.com/products/wandb/sweeps/signal-handling-sweep-runs.md) for details.

> [tab] Python script or notebook
>
> Use [`wandb.agent()`](https://docs.coreweave.com/products/wandb/ref/python/functions/agent) to start a sweep agent. Provide the sweep ID that Weights & Biases returns when you initialize the sweep, along with the name of the function that serves as the entrypoint to your training script.
>
> Replace `[SWEEP-ID]` with your sweep ID and `[FUNCTION-NAME]` with the name of your training function in the following code:
>
> ```python theme={"system"}
> wandb.agent(sweep_id="[SWEEP-ID]", function="[FUNCTION-NAME]")
> ```
>
> Weights & Biases only supports signal forwarding from the agent to the training run when you use the CLI (`wandb agent --forward-signals`). Weights & Biases doesn't support signal forwarding for `wandb.agent()` in Python because the training function runs in a thread, not as a child process.
>
> See the [Python script or notebook tab](https://docs.coreweave.com/products/wandb/sweeps/add-w-and-b-to-your-code.md) in Add Weights & Biases to your code for an example of how to set up your training script if you use this method.
>
> > [!WARNING]
> >
> > **Multiprocessing**
> >
> > You must wrap your `wandb.agent()` and `wandb.sweep()` calls with `if __name__ == '__main__':` if you use the Python standard library's `multiprocessing` or PyTorch's `pytorch.multiprocessing` package. For example:
> >
> > ```python theme={"system"}
> > if __name__ == '__main__':
> >     wandb.agent(sweep_id="[SWEEP-ID]", function="[FUNCTION]", count="[COUNT]")
> > ```
> >
> > Wrapping your code with this convention ensures that Python only runs it when you run the script directly, and not when a worker process imports it as a module.
> >
> > For more information about multiprocessing, see [Python standard library `multiprocessing`](https://docs.python.org/3/library/multiprocessing.html#the-spawn-and-forkserver-start-methods) or [PyTorch `multiprocessing`](https://docs.pytorch.org/docs/stable/notes/multiprocessing.html#asynchronous-multiprocess-training-e-g-hogwild). For more information about the `if __name__ == '__main__':` convention, see [Defining main functions in Python](https://realpython.com/if-name-main-python/).

## Limit the number of runs a sweep agent tries

By default, random and Bayesian searches run indefinitely, so cap how many runs an agent attempts. Specify the number of runs a sweep agent should try to bound its work. The following code snippets demonstrate how to set a maximum number of [runs](https://docs.coreweave.com/products/wandb/ref/python/experiments/run.md) with the CLI and within a Jupyter notebook or Python script.

> [!WARNING]
>
> Random and Bayesian searches run indefinitely. If you don't set a run count, you must stop the process from the command line, from within your Python script, or from the [Sweeps UI](https://docs.coreweave.com/products/wandb/sweeps/visualize-sweep-results.md).

> [tab] CLI
>
> First, initialize your sweep with the [`wandb sweep`](https://docs.coreweave.com/products/wandb/ref/cli/wandb-sweep) command. For more information, see [Initialize sweeps](https://docs.coreweave.com/products/wandb/sweeps/initialize-sweeps.md).
>
> ```bash theme={"system"}
> wandb sweep config.yaml
> ```
>
> Next, pass an integer value to the count flag to set the maximum number of runs to try.
>
> ```bash theme={"system"}
> NUM=10
> SWEEPID="dtzl1o7u"
> wandb agent --count $NUM $SWEEPID
> ```

> [tab] Python script or notebook
>
> First, initialize your sweep. For more information, see [Initialize sweeps](https://docs.coreweave.com/products/wandb/sweeps/initialize-sweeps.md).
>
> ```python theme={"system"}
> sweep_id = wandb.sweep(sweep_config)
> ```
>
> Next, start the sweep job. Provide the sweep ID that sweep initiation generates. Pass an integer value to the count parameter to set the maximum number of runs to try.
>
> ```python theme={"system"}
> sweep_id, count = "dtzl1o7u", 10
> wandb.agent(sweep_id, count=count)
> ```
>
> > [!WARNING]
> >
> > If you start a new run after the sweep agent finishes, within the same script or notebook, call `wandb.teardown()` before you start the new run.
