> ## Documentation Index
> Fetch the complete documentation index at: https://docs.coreweave.com/llms.txt
> Use this file to discover all available pages before exploring further.

> Log rich media, from 3D point clouds and molecules to HTML and histograms

# Log media and objects

[Try in Colab](https://colab.research.google.com/github/wandb/examples/blob/master/colabs/wandb-log/Log_\(Almost\)_Anything_with_W%26B_Media.ipynb)

Log media with the Weights & Biases Python SDK to inspect model inputs and outputs and compare results across runs, models, and datasets. Supported types include images, audio, video, histograms, 3D point clouds, molecules, and HTML.

For API details, see the [Data Types reference](https://docs.coreweave.com/products/wandb/ref/python/data-types.md). For examples, see the [model predictions report](https://forge.coreweave.com/wandb/lavanyashukla/visualize-predictions/reports/Visualize-Model-Predictions--Vmlldzo1NjM4OA) or watch the [video walkthrough](https://www.youtube.com/watch?v=96MxRvx15Ts).

## Pre-requisites

Some media types require additional dependencies. To install the Weights & Biases SDK with media dependencies, run:

```bash theme={"system"}
pip install wandb[media]
```

## Images

Use `wandb.Image` to log images from NumPy arrays, PIL images, or local files. Log images to inspect inputs, outputs, filter weights, and activations.

![Autoencoder inputs and outputs](https://mintcdn.com/coreweave-dbfa0e8d/3Dv_sw2eg8feUJlx/products/wandb/_media/log_images.png?fit=max\&auto=format\&n=3Dv_sw2eg8feUJlx\&q=85\&s=aa7bdc36cfa6eccb313ec7f98f08223a)

> [!NOTE]
>
> CoreWeave recommends logging fewer than 50 images per step to reduce the impact on training performance and image loading times.

The following tabs demonstrate how to log images using different input types.

> [tab] Arrays
>
> Provide arrays directly when constructing images manually, such as by using [`make_grid` from `torchvision`](https://pytorch.org/vision/stable/utils.html#torchvision.utils.make_grid).
>
> > [!NOTE]
> >
> > Weights & Biases converts arrays to PNG images using [Pillow](https://pillow.readthedocs.io/en/stable/index.html).
>
> ```python theme={"system"}
> import wandb
> import numpy as np
>
> with wandb.init(project="image-log-example") as run:
>     images = wandb.Image(image_array, caption="Top: Output, Bottom: Input")
>     run.log({"examples": images})
> ```
>
> For NumPy arrays, use the shape `(height, width, channels)`. Weights & Biases infers grayscale, RGB, or RGBA from a channel count of 1, 3, or 4, respectively. Images are saved as PNG files by default.
>
> Pixel values are normalized to the range 0–255 if values fall outside that range or all values fall within 0–1. To turn off normalization, pass `normalize=False`. For more control over array conversion, create a PIL image as shown in the **PIL images** tab.

> [tab] PIL Images
>
> Create a [`PIL.Image`](https://pillow.readthedocs.io/en/stable/reference/Image.html) and pass it to `wandb.Image`.
>
> ```python theme={"system"}
> from PIL import Image
>
> image_array = np.random.randint(0, 256, (128, 128, 3), dtype=np.uint8)
>
> with wandb.init(project="pil-image-example") as run:
>     # Create a PIL image from a NumPy array
>     image = Image.fromarray(image_array)
>
>     # Optionally, convert to RGB if needed
>     if image.mode != "RGB":
>         image = image.convert("RGB")
>
>     # Log the image
>     run.log({"example": wandb.Image(image, caption="My Image")})
> ```

> [tab] Image files
>
> To log an existing image file, pass its path to `wandb.Image`.
>
> ```python theme={"system"}
> with wandb.init(project="media-log-example") as run:
>     run.log({"example": wandb.Image("myimage.jpg")})
> ```

To compare images across training steps, expand the image panel in the Weights & Biases App UI and use the step slider. Select an image to open it in full-screen view, where you can zoom and pan with the mouse or [keyboard shortcuts](https://docs.coreweave.com/products/wandb/app/keyboard-shortcuts.md#media-panels).

To compare images or videos across runs, steps, or indices in one view, use [Compare mode](https://docs.coreweave.com/products/wandb/app/features/panels/media.md#compare-mode).

## Image overlays

Log semantic segmentation masks to adjust their opacity and inspect changes across steps in the Weights & Biases App UI.

> [tab] Segmentation Masks
>
> Log images with segmentation masks or bounding boxes to inspect model predictions and ground truth in the Weights & Biases App UI.
>
> ![Interactive mask viewing](https://mintcdn.com/coreweave-dbfa0e8d/3Dv_sw2eg8feUJlx/products/wandb/_media/semantic_segmentation.gif?s=b9e667527af86f499a6613c1e16907d9)
>
> Pass a dictionary to the `masks` argument of `wandb.Image`. Each key names a mask, such as `predictions` or `ground_truth`. Each value is a dictionary with the following fields:
>
> | Field | Description |
> | - | - |
> | `mask_data` | A 2D NumPy array containing an integer class label for each pixel. Provide either `mask_data` or `path`. |
> | `path` | The path to a saved mask image. Provide either `path` or `mask_data`. |
> | `class_labels` | Optional dictionary that maps integer class labels to readable names. |
>
> To log multiple masks for an image, provide a separate dictionary for each mask:
>
> ```python theme={"system"}
> mask_data = np.array([[1, 2, 2, ..., 2, 2, 1], ...])
>
> class_labels = {1: "tree", 2: "car", 3: "road"}
>
> mask_img = wandb.Image(
>     image,
>     masks={
>         "predictions": {"mask_data": mask_data, "class_labels": class_labels},
>         "ground_truth": {
>             # ...
>         },
>         # ...
>     },
> )
> ```
>
> Segmentation masks for each key are defined at each step (each call to `wandb.Run.log()`).
>
> * If steps provide different values for the same mask key, only the most recent value for that key is applied to the image.
> * If steps provide different mask keys, the visibility controls show all keys, but only masks defined at the selected step are applied to the image. Toggling a mask that is not defined at that step does not change the image.
>
> For a complete example, see the [segmentation masks report](https://forge.coreweave.com/wandb/stacey/deep-drive/reports/Image-Masks-for-Semantic-Segmentation--Vmlldzo4MTUwMw) and [example notebook](https://colab.research.google.com/drive/1SOVl3EvW82Q4QKJXX6JtHye4wFix_P4J).

> [tab] Bounding Boxes
>
> Log images with bounding boxes, then use filters and visibility controls to inspect groups of boxes in the Weights & Biases App UI.
>
> ![Bounding box example](https://mintcdn.com/coreweave-dbfa0e8d/3Dv_sw2eg8feUJlx/products/wandb/_media/bb-docs.jpeg?fit=max\&auto=format\&n=3Dv_sw2eg8feUJlx\&q=85\&s=94b07a91c1a09bd5b95e6111ec06fc77)
>
> Pass a dictionary to the `boxes` argument of `wandb.Image`. Each key names a group, such as `predictions` or `ground_truth`. Each value contains these fields:
>
> | Field | Description |
> | - | - |
> | `box_data` | A list of dictionaries, one for each bounding box. |
> | `class_labels` | Optional dictionary that maps class IDs to readable names. If omitted, Weights & Biases generates names such as `class_0` and `class_1`. |
>
> Each dictionary in `box_data` describes a box:
>
> | Field | Description |
> | - | - |
> | `position` | The box position and size. Use either `minX`, `maxX`, `minY`, and `maxY`, or `middle`, `width`, and `height`. For `middle`, provide `[x, y]` coordinates. Boxes can use different formats. |
> | `class_id` | An integer class ID that corresponds to an entry in `class_labels`. |
> | `scores` | A dictionary that maps score names to numeric values. Use these scores to filter boxes in the Weights & Biases App UI. |
> | `domain` | Coordinate units. Set to `"pixel"` for pixel coordinates. By default, coordinates are fractions of the image dimensions, from 0 to 1. |
> | `box_caption` | Optional text to display on the box. |
>
> This example logs one box with relative coordinates and another with pixel coordinates:
>
> ```python theme={"system"}
> import wandb
>
> class_id_to_label = {
>     1: "car",
>     2: "road",
>     3: "building"
> }
>
> img = wandb.Image(
>     image,
>     boxes={
>         "predictions": {
>             "box_data": [
>                 {
>                     # one box expressed in the default relative/fractional domain
>                     "position": {"minX": 0.1, "maxX": 0.2, "minY": 0.3, "maxY": 0.4},
>                     "class_id": 2,
>                     "box_caption": class_id_to_label[2],
>                     "scores": {"acc": 0.1, "loss": 1.2},
>                     # another box expressed in the pixel domain
>                     # (for illustration purposes only, all boxes are likely
>                     # to be in the same domain/format)
>                     "position": {"middle": [150, 20], "width": 68, "height": 112},
>                     "domain": "pixel",
>                     "class_id": 3,
>                     "box_caption": "a building",
>                     "scores": {"acc": 0.5, "loss": 0.7},
>                     # ...
>                     # Log as many boxes as needed
>                 }
>             ],
>             "class_labels": class_id_to_label,
>         },
>         # Log each meaningful group of boxes with a unique key name
>         "ground_truth": {
>             # ...
>         },
>     },
> )
>
> with wandb.init(project="my_project") as run:
>     run.log({"driving_scene": img})
> ```

For an interactive example, see the [bounding boxes report](https://forge.coreweave.com/wandb/stacey/yolo-drive/reports/Bounding-Boxes-for-Object-Detection--Vmlldzo4Nzg4MQ).

## Image overlays in Tables

To display image overlays in a Weights & Biases Table, add a `wandb.Image` object with masks or bounding boxes to each row.

> [tab] Segmentation Masks
>
> ![Interactive Segmentation Masks in Tables](https://mintcdn.com/coreweave-dbfa0e8d/3Dv_sw2eg8feUJlx/products/wandb/_media/Segmentation_Masks.gif?s=cec1c5e9cebb5923fcd0763f8d7b41da)
>
> This example logs one image with segmentation masks for each row in the table:
>
> ```python theme={"system"}
> table = wandb.Table(columns=["ID", "Image"])
>
> for id, img, label in zip(ids, images, labels):
>     mask_img = wandb.Image(
>         img,
>         masks={
>             "prediction": {
>                 "mask_data": label,
>                 "class_labels": class_labels
>                 }
>             # ...
>         },
>     )
>
>     table.add_data(id, mask_img)
>
> with wandb.init(project="my_project") as run:
>     run.log({"Table": table})
> ```

> [tab] Bounding Boxes
>
> ![Interactive Bounding Boxes in Tables](https://mintcdn.com/coreweave-dbfa0e8d/3Dv_sw2eg8feUJlx/products/wandb/_media/Bounding_Boxes.gif?s=b9faeac88fc44883009bf81c79002ee1)
>
> This example logs one image with bounding boxes for each row in the table:
>
> ```python theme={"system"}
> table = wandb.Table(columns=["ID", "Image"])
>
> for id, img, boxes in zip(ids, images, boxes_set):
>     box_img = wandb.Image(
>         img,
>         boxes={
>             "prediction": {
>                 "box_data": [
>                     {
>                         "position": {
>                             "minX": box["minX"],
>                             "minY": box["minY"],
>                             "maxX": box["maxX"],
>                             "maxY": box["maxY"],
>                         },
>                         "class_id": box["class_id"],
>                         "box_caption": box["caption"],
>                         "domain": "pixel",
>                     }
>                     for box in boxes
>                 ],
>                 "class_labels": class_labels,
>             }
>         },
>     )
> ```

## Histograms

Use `wandb.Histogram` to log numeric distributions, such as model gradients. In the Weights & Biases App UI, the x-axis shows training steps, the y-axis shows metric values, and color represents the count in each bin.

> [tab] Basic Histogram Logging
>
> Pass a sequence of numbers, such as a list, array, or tensor, to `wandb.Histogram`. Weights & Biases flattens arrays and tensors and uses `numpy.histogram()` to calculate the histogram.
>
> By default, histograms use 64 bins. To customize the number of bins, set `num_bins` a value up to 512.
>
> ```python theme={"system"}
> with wandb.init() as run:
>     run.log({"gradients": wandb.Histogram(grads)})
> ```
>
> ![GAN discriminator gradients](https://mintcdn.com/coreweave-dbfa0e8d/3Dv_sw2eg8feUJlx/products/wandb/_media/histograms.png?fit=max\&auto=format\&n=3Dv_sw2eg8feUJlx\&q=85\&s=3a7e212a1394f41a738ae0a48e19c613)

> [tab] Flexible Histogram Logging
>
> To control histogram calculation, call `np.histogram()` and pass the returned tuple to `np_histogram`:
>
> ```python theme={"system"}
> import numpy as np
>
> np_hist_grads = np.histogram(grads, density=True, range=(0.0, 1.0))
>
> with wandb.init() as run:
>     run.log({"gradients": wandb.Histogram(np_hist_grads)})
> ```

Optionally set `density=True` to represent bin values as a probability density rather than counts.

## 3D visualizations

Use `wandb.Object3D` to log 3D point clouds and lidar scenes with bounding boxes. Provide a NumPy array with point coordinates and optional colors or category labels.

Point clouds are interactive in the Weights & Biases App UI. Drag to rotate the view, zoom to inspect details, or hold **Control** while using the mouse to move through the scene.

The following report shows scenes from the Lyft dataset with 3D bounding boxes:

**LIDAR point clouds of driving scenes report**

> [!NOTE]
>
> The Weights & Biases App UI displays up to 300,000 points per point cloud.

### NumPy array formats

Use one of the following array formats, where `n` is the number of points:

| Shape | Point format | Description |
| - | - | - |
| `(n, 3)` | `[x, y, z]` | Point coordinates. |
| `(n, 4)` | `[x, y, z, category]` | Coordinates and a category from 1 to 14, such as a segmentation class. |
| `(n, 6)` | `[x, y, z, r, g, b]` | Coordinates and red, green, and blue values from 0 to 255. |

### Python object

Use [`wandb.Object3D.from_point_cloud()`](https://docs.coreweave.com/products/wandb/ref/python/data-types/object3d#method-object3d-from_point_cloud) to create a scene with points, bounding boxes, and vectors.

```python theme={"system"}
import wandb
import numpy as np

points = np.array([
    [0.2, 0.3, 0.4, 255, 0, 0],
    [0.5, 0.6, 0.7, 0, 255, 0],
    [0.8, 0.9, 0.2, 0, 0, 255],
])

boxes = [{
    "corners": [
        [0, 0, 0], [1, 0, 0], [1, 1, 0], [0, 1, 0],
        [0, 0, 1], [1, 0, 1], [1, 1, 1], [0, 1, 1],
    ],
    "color": [0, 0, 255],
    "label": "example region",
    "score": 0.6,
}]

scene = wandb.Object3D.from_point_cloud(
    points=points,
    boxes=boxes,
    vectors=[{"start": [0, 0, 0], "end": [0.1, 0.2, 0.5], "color": [255, 0, 0]}],
    point_cloud_type="lidar/beta",
)

with wandb.init() as run:
    run.log({"my_first_point_cloud": scene})
```

To move through a point cloud in the Weights & Biases App UI, hold **Control** while moving the mouse.

### Point cloud files

Use [`wandb.Object3D.from_file()`](https://docs.coreweave.com/products/wandb/ref/python/data-types/object3d#method-object3d-from_file) to load point cloud data from a JSON file:

```python theme={"system"}
run.log({"my_cloud_from_file": wandb.Object3D.from_file(
     "./my_point_cloud.pts.json"
)})
```

The following JSON illustrates the format for point cloud data:

```json theme={"system"}
{
  "points": [
    [0.2, 0.3, 0.4, 255, 0, 0],
    [0.5, 0.6, 0.7, 0, 255, 0],
    [0.8, 0.9, 0.2, 0, 0, 255]
  ],
  "boxes": [
    {
      "corners": [
        [0, 0, 0], [1, 0, 0], [1, 1, 0], [0, 1, 0],
        [0, 0, 1], [1, 0, 1], [1, 1, 1], [0, 1, 1]
      ],
      "color": [0, 0, 255],
      "label": "example region",
      "score": 0.6
    }
  ],
  "type": "lidar/beta"
}
```

### NumPy arrays

Use `wandb.Object3D.from_numpy()` with any of the [supported array formats](#numpy-array-formats):

```python theme={"system"}

# Coordinates only.
xyz_points = np.array([[0.4, 1, 1.3], [1, 1, 1], [1.2, 1, 1.2]])

# Coordinates and category labels.
category_points = np.array([[0.4, 1, 1.3, 1], [1, 1, 1, 1], [1.2, 1, 1.2, 12]])

# Coordinates and RGB colors.
rgb_points = np.array([
    [0.4, 1, 1.3, 255, 0, 0],
    [1, 1, 1, 0, 255, 0],
    [1.2, 1, 1.2, 0, 0, 255],
])

with wandb.init() as run:
    run.log({
        "xyz_point_cloud": wandb.Object3D.from_numpy(xyz_points),
        "category_point_cloud": wandb.Object3D.from_numpy(category_points),
        "rgb_point_cloud": wandb.Object3D.from_numpy(rgb_points),
    })
```

## Molecules

Use `wandb.Molecule` to log molecular data and inspect 3D structures in the Weights & Biases App UI.

```python theme={"system"}
run.log({"protein": wandb.Molecule("6lu7.pdb")})
```

Supported file types are `pdb`, `pqr`, `mmcif`, `mcif`, `cif`, `sdf`, `sd`, `gro`, `mol2`, and `mmtf`.

You can also log SMILES strings, RDKit `mol` files, and `rdkit.Chem.rdchem.Mol` objects. These examples require [RDKit](https://www.rdkit.org/docs/index.html).

```python theme={"system"}
from rdkit import Chem

resveratrol = rdkit.Chem.MolFromSmiles("Oc1ccc(cc1)C=Cc1cc(O)cc(c1)O")

with wandb.init() as run:
    run.log(
        {
            "resveratrol": wandb.Molecule.from_rdkit(resveratrol),
            "green fluorescent protein": wandb.Molecule.from_rdkit("2b3p.mol"),
            "acetaminophen": wandb.Molecule.from_smiles("CC(=O)Nc1ccc(O)cc1"),
        }
    )
```

For an interactive example, see the [AlphaFold workspace](https://wandb.me/alphafold-workspace).

![Molecule structure](https://mintcdn.com/coreweave-dbfa0e8d/3Dv_sw2eg8feUJlx/products/wandb/_media/docs-molecule.png?fit=max\&auto=format\&n=3Dv_sw2eg8feUJlx\&q=85\&s=737a9bdbb5d1b11e1cdb3e9a60482045)

## 2D view of a molecule

You can log a 2D view of a molecule using the [`wandb.Image`](https://docs.coreweave.com/products/wandb/ref/python/data-types/image) data type and [`rdkit`](https://www.rdkit.org/docs/index.html):

```python theme={"system"}
from rdkit import Chem
from rdkit.Chem import AllChem, Draw

molecule = Chem.MolFromSmiles("CC(=O)O")
AllChem.Compute2DCoords(molecule)
pil_image = Draw.MolToImage(molecule, size=(300, 300))

with wandb.init() as run:
    run.log({"acetic_acid": wandb.Image(pil_image)})
```

## Other media

Weights & Biases also supports logging of a variety of other media types.

### Audio

Use `wandb.Audio` to log audio data. Set `sample_rate` to the recording's sample rate in hertz.

```python theme={"system"}
with wandb.init() as run:
    run.log({"whale songs": wandb.Audio(np_array, caption="OooOoo", sample_rate=32)})
```

You can log up to 100 audio clips per step. For more information, see [`audio-file`](https://docs.coreweave.com/products/wandb/ref/query-panel/audio-file).

### Video

Use `wandb.Video` to log a video file:

```python theme={"system"}
with wandb.init() as run:
    run.log({"video": wandb.Video("myvideo.mp4")})
```

To log a NumPy array of video frames, specify the output format and frame rate:

```python theme={"system"}
with wandb.init() as run:
    run.log({"video": wandb.Video(video_array, fps=4, format="gif")})
```

Use the shape `(frames, channels, height, width)` for a single video or `(batch, frames, channels, height, width)` for a batch. Encoding NumPy arrays requires `FFmpeg` and `MoviePy`.

Supported formats are `"gif"`, `"mp4"`, `"webm"`, and `"ogg"`. When passing an array or an in-memory stream, specify `format`. When encoding raw frames, the default frame rate is 4 frames per second.

When you provide a file path, Weights & Biases checks that the file exists and uses a supported format. When you provide a `BytesIO` object, Weights & Biases creates a temporary file with the specified format as its extension.

To view and play videos, open a media panel in your project workspace, run workspace, or report. Use [Compare mode](https://docs.coreweave.com/products/wandb/app/features/panels/media.md#compare-mode) to compare videos across runs or steps.

### Text

Use `wandb.Table` to display text alongside predictions, labels, or other values. Specify column names when you create the table.

By default, the column headers are `["Input", "Output", "Expected"]`. To ensure optimal UI performance, the default maximum number of rows is set to 10,000. However, you can explicitly override the maximum with `wandb.Table.MAX_ROWS = {DESIRED_MAX}`.

```python theme={"system"}
columns = ["Text", "Predicted Sentiment", "True Sentiment"]
data = [["I love my phone", "1", "1"], ["My phone is terrible", "0", "-1"]]

with wandb.init(project="my_project") as run:
    table = wandb.Table(data=data, columns=columns)
    run.log({"examples": table})
```

Alternatively, add rows with `wandb.Table.add_data()`:

```python theme={"system"}
table = wandb.Table(columns=columns)
table.add_data("I love my phone", "positive", "positive")
table.add_data("My phone stopped working", "negative", "negative")
run.log({"text_examples": table})
```

To create a table from a pandas DataFrame, pass it to `dataframe`:

```python theme={"system"}
table = wandb.Table(dataframe=my_dataframe)
with wandb.init() as run:
    run.log({"examples": table})
```

For more usage information, see [`string`](https://docs.coreweave.com/products/wandb/ref/query-panel).

### HTML

Use `wandb.Html` to display custom HTML in a panel on the run page. Pass an HTML file or a string:

```python theme={"system"}

with wandb.init() as run:
    run.log({"custom_string": wandb.Html('<a href="https://mysite">Link</a>')})
    
    with open("some.html") as f:
        run.log({"custom_file": wandb.Html(f)})
```

By default, Weights & Biases adds styles to the HTML. To disable this behavior, pass `inject=False`. To turn off this behavior pass `inject=False`:

```python theme={"system"}
with wandb.init() as run:
    with open("some.html") as html_file:
        run.log({"custom_file": wandb.Html(html_file, inject=False)})
```

For more usage information, see [`html-file`](https://docs.coreweave.com/products/wandb/ref/query-panel/html-file).
