> ## Documentation Index
> Fetch the complete documentation index at: https://docs.coreweave.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Send existing OpenTelemetry spans to Agent Lens

> Send OTLP trace data from any OpenTelemetry pipeline to the Agent Lens endpoint. No SDK integration required.

CoreWeave Agent Lens is built on [OpenTelemetry (OTel)](https://opentelemetry.io/docs/concepts/) and provides a dedicated OTLP endpoint that ingests spans into Agent Lens. This means that you can quickly redirect your OTel-instrumented application's spans to Agent Lens by swapping your OTel exporter's current endpoint configuration with Agent Lens's endpoint configuration. You do not need to install the CoreWeave Forge SDK to do this.

This page covers how to authenticate with and send spans to the endpoint.

> [!NOTE]
>
> Agent Lens runs on the same backend as Weights & Biases, so some configuration names keep the `wandb` prefix: the `WANDB_API_KEY` environment variable, the `wandb-api-key` header, and the `wandb.entity` and `wandb.project` resource attributes. Use these names as shown.

## Authentication

You can send spans to the following endpoint:

* **Path**: `/agents/otel/v1/traces`
* **Method**: `POST`
* **Content-Type**: `application/x-protobuf`
* **Base URL**: `https://trace.wandb.ai` for Forge.

The endpoint accepts OTLP protobuf payloads only. Use a protobuf HTTP exporter such as `opentelemetry-exporter-otlp-proto-http` (Python) or `@opentelemetry/exporter-trace-otlp-proto` (TypeScript). The endpoint also accepts `gzip` and `deflate` content encoding.

To authenticate with the endpoint, set your [API key](https://forge.coreweave.com/settings#apikeys) in your `OTLPSpanExporter` configuration using one of the following values:

* `wandb-api-key`: Your Forge API key as the value.
* `Authorization`: HTTP Basic authentication with `api` as the user and your Forge API key as the password. This form is useful when your exporter or collector only supports standard authorization headers.

## Project routing

Agent Lens routes spans to a project using either OTel resource attributes or a request header:

* **Resource attributes** (recommended): Set `wandb.entity` to your Forge team or user name and `wandb.project` to the project name on your `TracerProvider` resource.
* **`project_id` header**: Set the value to `[YOUR-TEAM]/[YOUR-PROJECT]`.

If both are present, the resource attributes take precedence. Spans that arrive with no entity and project are dropped.

## Shape spans for the Conversations tab

Agent Lens accepts any OTel span and stores all of its attributes, but spans must follow the [OpenTelemetry semantic conventions for Agent Lens](https://docs.coreweave.com/products/agent-lens/reference/semantic-conventions.md) to correctly render in the Conversations tab. Set `gen_ai.operation.name` on each span to tell Agent Lens what the span represents:

| `gen_ai.operation.name` | Represents | Rendered as |
| - | - | - |
| `invoke_agent` | One complete user-agent exchange. | A turn. |
| `chat`, `generate_content`, or `text_completion` | One call to a model API. | An LLM call within a turn. |
| `execute_tool` | One tool execution. | A tool call. |

To show token counts and estimate cost, also set the [token usage attribute](https://docs.coreweave.com/products/agent-lens/reference/semantic-conventions.md#token-usage-and-cost) on each LLM span.

To check what Agent Lens received for a span, open the span’s **Raw** tab in the [conversation view](https://docs.coreweave.com/products/agent-lens/conversations/view-activity.md#view-conversation-turns-spans-and-timelines). For the agent data model that these spans map to, see [Trace your agents](https://docs.coreweave.com/products/agent-lens/tracing/instrument.md#the-agent-data-model).

## Configure with environment variables only

If your application or collector reads the standard OTel exporter environment variables, you can set the following variables and route spans to Agent Lens without changing any code:

```bash lines theme={"system"}
export OTEL_EXPORTER_OTLP_TRACES_ENDPOINT="https://trace.wandb.ai/agents/otel/v1/traces"
export OTEL_EXPORTER_OTLP_TRACES_HEADERS="wandb-api-key=$WANDB_API_KEY"
export OTEL_RESOURCE_ATTRIBUTES="wandb.entity=[YOUR-TEAM],wandb.project=[YOUR-PROJECT]"
export OTEL_EXPORTER_OTLP_TRACES_PROTOCOL=http/protobuf
```

* `OTEL_EXPORTER_OTLP_TRACES_ENDPOINT`: The Agent Lens endpoint your exporter sends spans to.
* `OTEL_EXPORTER_OTLP_TRACES_HEADERS`: Authenticates each request with your Forge API key, sent in the `wandb-api-key` header.
* `OTEL_RESOURCE_ATTRIBUTES`: Sends spans to your project. Set `wandb.entity` to your Forge team or user name and `wandb.project` to the project name.
* `OTEL_EXPORTER_OTLP_TRACES_PROTOCOL`: Sets the exporter to send OTLP protobuf over HTTP. It must be `http/protobuf`, because the endpoint accepts only protobuf payloads.

## Example: emit agent spans without the Agent Lens SDK

The following example instruments one agent turn using only OTel packages. A user asks about the weather, the model calls a `get_weather` tool, and then the model answers with the result. The example emits an `invoke_agent` span for the turn, a `chat` span for each LLM call, and an `execute_tool` span for the tool call, then exports them to the Agent Lens agents endpoint.

The turn and each LLM call carry the user's prompt and the model's replies in `gen_ai.input.messages` and `gen_ai.output.messages`. Each LLM call also carries the provider, the requested and returned model IDs, and its token counts. The tool call carries its input and output in `gen_ai.tool.call.arguments` and `gen_ai.tool.call.result`. As we recommend, every span carries the conversation ID, the agent name, and the user's ID and email. The example hardcodes all of these values for demonstration purposes.

First, install the required dependencies:

```bash Python theme={"system"}
pip install opentelemetry-sdk opentelemetry-exporter-otlp-proto-http
```

```bash TypeScript theme={"system"}
npm install @opentelemetry/api @opentelemetry/sdk-trace-node @opentelemetry/sdk-trace-base @opentelemetry/resources @opentelemetry/exporter-trace-otlp-proto
```

Set the `WANDB_API_KEY` environment variable to your [API key](https://forge.coreweave.com/settings#apikeys):

```bash theme={"system"}
export WANDB_API_KEY="[YOUR-API-KEY]"
```

Then replace `[YOUR-TEAM]` and `[YOUR-PROJECT]` in the following code with your Forge team and project names, and run it:

```python Python lines highlight="9,10" theme={"system"}
import json
import os
from opentelemetry import trace
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk import trace as trace_sdk
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace.export import BatchSpanProcessor

ENTITY = "[YOUR-TEAM]"
PROJECT = "[YOUR-PROJECT]"

AGENT_LENS_OTLP_ENDPOINT = "https://trace.wandb.ai/agents/otel/v1/traces"

# Read the API key that you exported in the previous step.
WANDB_API_KEY = os.environ["WANDB_API_KEY"]

exporter = OTLPSpanExporter(
    endpoint=AGENT_LENS_OTLP_ENDPOINT,
    headers={"wandb-api-key": WANDB_API_KEY},
)

tracer_provider = trace_sdk.TracerProvider(resource=Resource({
    "wandb.entity": ENTITY,
    "wandb.project": PROJECT,
}))
tracer_provider.add_span_processor(BatchSpanProcessor(exporter))
trace.set_tracer_provider(tracer_provider)

tracer = trace.get_tracer("my-agent")

TOOL_CALL_ID = "call-001"
ARGUMENTS = {"city": "Paris"}
WEATHER = {"temperature_c": 18, "conditions": "partly cloudy"}

# Set these on every span in the turn, not just the turn's own span.
SHARED_ATTRIBUTES = {
    "gen_ai.conversation.id": "conversation-001",
    "gen_ai.agent.name": "my-agent",
    "user.id": "user-001",
    "user.email": "user@example.com",
}

# Each message has a role and a list of typed parts, per the GenAI semantic conventions.
user_message = {
    "role": "user",
    "parts": [{"type": "text", "content": "What's the weather in Paris?"}],
}
tool_call_message = {
    "role": "assistant",
    "parts": [{"type": "tool_call", "id": TOOL_CALL_ID, "name": "get_weather", "arguments": ARGUMENTS}],
}
tool_result_message = {
    "role": "tool",
    "parts": [{"type": "tool_call_response", "id": TOOL_CALL_ID, "response": WEATHER}],
}
answer_message = {
    "role": "assistant",
    "parts": [{"type": "text", "content": "It's 18°C and partly cloudy in Paris."}],
}

# One turn: the user asks a question, the model calls a tool, then answers.
with tracer.start_as_current_span("invoke_agent my-agent") as turn:
    turn.set_attribute("gen_ai.operation.name", "invoke_agent")
    turn.set_attributes(SHARED_ATTRIBUTES)
    turn.set_attribute("gen_ai.input.messages", json.dumps([user_message]))

    # First LLM call: the model asks to call a tool.
    with tracer.start_as_current_span("chat gpt-4o") as llm:
        llm.set_attribute("gen_ai.operation.name", "chat")
        llm.set_attributes(SHARED_ATTRIBUTES)
        llm.set_attribute("gen_ai.provider.name", "openai")
        llm.set_attribute("gen_ai.request.model", "gpt-4o")
        llm.set_attribute("gen_ai.response.model", "gpt-4o-2024-08-06")
        llm.set_attribute("gen_ai.input.messages", json.dumps([user_message]))
        llm.set_attribute("gen_ai.output.messages", json.dumps([tool_call_message]))
        llm.set_attribute("gen_ai.usage.input_tokens", 100)
        llm.set_attribute("gen_ai.usage.output_tokens", 20)

    # The tool runs under the turn, after the LLM call that requested it.
    with tracer.start_as_current_span("execute_tool get_weather") as tool:
        tool.set_attribute("gen_ai.operation.name", "execute_tool")
        tool.set_attributes(SHARED_ATTRIBUTES)
        tool.set_attribute("gen_ai.tool.name", "get_weather")
        tool.set_attribute("gen_ai.tool.call.id", TOOL_CALL_ID)
        tool.set_attribute("gen_ai.tool.call.arguments", json.dumps(ARGUMENTS))
        tool.set_attribute("gen_ai.tool.call.result", json.dumps(WEATHER))

    # Second LLM call: the model answers from the tool result.
    with tracer.start_as_current_span("chat gpt-4o") as llm:
        llm.set_attribute("gen_ai.operation.name", "chat")
        llm.set_attributes(SHARED_ATTRIBUTES)
        llm.set_attribute("gen_ai.provider.name", "openai")
        llm.set_attribute("gen_ai.request.model", "gpt-4o")
        llm.set_attribute("gen_ai.response.model", "gpt-4o-2024-08-06")
        llm.set_attribute("gen_ai.input.messages", json.dumps(
            [user_message, tool_call_message, tool_result_message]
        ))
        llm.set_attribute("gen_ai.output.messages", json.dumps([answer_message]))
        llm.set_attribute("gen_ai.usage.input_tokens", 140)
        llm.set_attribute("gen_ai.usage.output_tokens", 15)

    turn.set_attribute("gen_ai.output.messages", json.dumps([answer_message]))

tracer_provider.shutdown()  # Flush all pending spans before exit.
```

```typescript TypeScript lines highlight="7,8" theme={"system"}
import { trace } from "@opentelemetry/api";
import { NodeTracerProvider } from "@opentelemetry/sdk-trace-node";
import { BatchSpanProcessor } from "@opentelemetry/sdk-trace-base";
import { OTLPTraceExporter } from "@opentelemetry/exporter-trace-otlp-proto";
import { resourceFromAttributes } from "@opentelemetry/resources";

const ENTITY = "[YOUR-TEAM]";
const PROJECT = "[YOUR-PROJECT]";

const AGENT_LENS_OTLP_ENDPOINT = "https://trace.wandb.ai/agents/otel/v1/traces";

// Read the API key that you exported in the previous step.
const WANDB_API_KEY = process.env.WANDB_API_KEY!;

const exporter = new OTLPTraceExporter({
  url: AGENT_LENS_OTLP_ENDPOINT,
  headers: { "wandb-api-key": WANDB_API_KEY },
});

const provider = new NodeTracerProvider({
  resource: resourceFromAttributes({
    "wandb.entity": ENTITY,
    "wandb.project": PROJECT,
  }),
  spanProcessors: [new BatchSpanProcessor(exporter)],
});

provider.register();

const tracer = trace.getTracer("my-agent");

const TOOL_CALL_ID = "call-001";
const ARGUMENTS = { city: "Paris" };
const WEATHER = { temperature_c: 18, conditions: "partly cloudy" };

// Set these on every span in the turn, not just the turn's own span.
const SHARED_ATTRIBUTES = {
  "gen_ai.conversation.id": "conversation-001",
  "gen_ai.agent.name": "my-agent",
  "user.id": "user-001",
  "user.email": "user@example.com",
};

// Each message has a role and a list of typed parts, per the GenAI semantic conventions.
const userMessage = {
  role: "user",
  parts: [{ type: "text", content: "What's the weather in Paris?" }],
};
const toolCallMessage = {
  role: "assistant",
  parts: [{ type: "tool_call", id: TOOL_CALL_ID, name: "get_weather", arguments: ARGUMENTS }],
};
const toolResultMessage = {
  role: "tool",
  parts: [{ type: "tool_call_response", id: TOOL_CALL_ID, response: WEATHER }],
};
const answerMessage = {
  role: "assistant",
  parts: [{ type: "text", content: "It's 18°C and partly cloudy in Paris." }],
};

// One turn: the user asks a question, the model calls a tool, then answers.
tracer.startActiveSpan("invoke_agent my-agent", (turn) => {
  turn.setAttribute("gen_ai.operation.name", "invoke_agent");
  turn.setAttributes(SHARED_ATTRIBUTES);
  turn.setAttribute("gen_ai.input.messages", JSON.stringify([userMessage]));

  // First LLM call: the model asks to call a tool.
  tracer.startActiveSpan("chat gpt-4o", (llm) => {
    llm.setAttribute("gen_ai.operation.name", "chat");
    llm.setAttributes(SHARED_ATTRIBUTES);
    llm.setAttribute("gen_ai.provider.name", "openai");
    llm.setAttribute("gen_ai.request.model", "gpt-4o");
    llm.setAttribute("gen_ai.response.model", "gpt-4o-2024-08-06");
    llm.setAttribute("gen_ai.input.messages", JSON.stringify([userMessage]));
    llm.setAttribute("gen_ai.output.messages", JSON.stringify([toolCallMessage]));
    llm.setAttribute("gen_ai.usage.input_tokens", 100);
    llm.setAttribute("gen_ai.usage.output_tokens", 20);
    llm.end();
  });

  // The tool runs under the turn, after the LLM call that requested it.
  tracer.startActiveSpan("execute_tool get_weather", (tool) => {
    tool.setAttribute("gen_ai.operation.name", "execute_tool");
    tool.setAttributes(SHARED_ATTRIBUTES);
    tool.setAttribute("gen_ai.tool.name", "get_weather");
    tool.setAttribute("gen_ai.tool.call.id", TOOL_CALL_ID);
    tool.setAttribute("gen_ai.tool.call.arguments", JSON.stringify(ARGUMENTS));
    tool.setAttribute("gen_ai.tool.call.result", JSON.stringify(WEATHER));
    tool.end();
  });

  // Second LLM call: the model answers from the tool result.
  tracer.startActiveSpan("chat gpt-4o", (llm) => {
    llm.setAttribute("gen_ai.operation.name", "chat");
    llm.setAttributes(SHARED_ATTRIBUTES);
    llm.setAttribute("gen_ai.provider.name", "openai");
    llm.setAttribute("gen_ai.request.model", "gpt-4o");
    llm.setAttribute("gen_ai.response.model", "gpt-4o-2024-08-06");
    llm.setAttribute(
      "gen_ai.input.messages",
      JSON.stringify([userMessage, toolCallMessage, toolResultMessage]),
    );
    llm.setAttribute("gen_ai.output.messages", JSON.stringify([answerMessage]));
    llm.setAttribute("gen_ai.usage.input_tokens", 140);
    llm.setAttribute("gen_ai.usage.output_tokens", 15);
    llm.end();
  });

  turn.setAttribute("gen_ai.output.messages", JSON.stringify([answerMessage]));
  turn.end();
});

// Flush all pending spans before exit.
provider.shutdown().catch(console.error);
```

To add more turns to the same conversation, emit more `invoke_agent` spans with the same `gen_ai.conversation.id`. Each turn must be the root span of its own trace, so turns can't share a parent span. Give all of a turn's spans timestamps from one clock, and end the turn's span after its children. See [Keep span times consistent](https://docs.coreweave.com/products/agent-lens/tracing/instrument.md#keep-span-times-consistent).

After your application exports spans, open the **Conversations** view of your Agent Lens project.
