> ## Documentation Index
> Fetch the complete documentation index at: https://docs.coreweave.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Inference proxy overview

> Use one OpenAI-compatible endpoint for project routing, provider credentials, and trace capture.

The inference proxy is the only component on your application's online path. It exposes:

```text theme={"system"}
POST https://proxy.training.coreweave.com/v1/chat/completions
Authorization: Bearer <wandb-api-key>
OpenAI-Project: [ENTITY]/[PROJECT]
```

The bearer token is your Forge API key. `OpenAI-Project` selects the Model Distillation project by its entity and Weights & Biases project name. Your existing OpenAI client continues to work: the OpenAI SDKs send the header from their `project` option or the `OPENAI_PROJECT_ID` environment variable. The `model` value determines whether the request uses the project's weighted routing or overrides the model.

## What the proxy adds

> [card] Project routing
>
> Resolve a project version, choose a weighted target, apply its parameters, and keep conversations sticky.

> [card] Credential isolation
>
> Replace the API key with the selected provider credential without forwarding client headers.

> [card] Training-grade traces
>
> Record the complete dispatched request and response with project and provider identity.

> [card] OpenAI-compatible transport
>
> Preserve non-streaming and streaming Chat Completions response behavior.

## Supported routes

With the `OpenAI-Project` header, `model` takes one of these values:

| `model` value | Result |
| - | - |
| `router`, blank, or omitted | Weighted routing for the selected project version |
| `provider/model-id` | [Model override](https://docs.coreweave.com/model-distillation/proxy/model-routing.md#model-overrides) through a registered provider, within the project |

`metadata["forge.project_version"]` selects the project version, such as `"2"`. Omitting it selects version 1.

Routed requests and model overrides are both project traffic. The proxy traces, samples, and caches them in the selected version.

Non-streaming project requests can also opt into [exact-response caching](https://docs.coreweave.com/model-distillation/proxy/response-caching.md) with the `Forge-Cache-Mode` header, which replays an identical earlier response instead of calling the provider.

> [accordion] API: Send a project-routed request (POST /chat/completions)
>
> ```bash theme={"system"}
> curl --request POST \
>   --url "https://proxy.training.coreweave.com/v1/chat/completions" \
>   --header "Authorization: Bearer $WANDB_API_KEY" \
>   --header "Content-Type: application/json" \
>   --header "OpenAI-Project: your-team/ticket-classifier" \
>   --data '{
>     "model": "router",
>     "messages": [{"role": "user", "content": "Classify this request"}],
>     "metadata": {"gen_ai.conversation.id": "conversation-4827"}
>   }'
> ```
>
> See [Create a routed chat completion](https://docs.coreweave.com/model-distillation/reference/proxy/chat-completions/create-a-routed-chat-completion.md) for the complete request and response schema.

## Fail closed behavior

The proxy rejects unknown API keys, entities, projects, versions, and providers before calling an arbitrary endpoint. Provider URLs and credentials come only from server-managed materialization; callers cannot supply them in request headers or tokens.
