# Depth Maps

Source: https://www.potatoannotator.com/docs/vision-spatial/depth-maps

**Potato renders depth maps legibly, reports the distance under the cursor in metres, and — given camera intrinsics — turns a depth map into a point cloud you can put 3D boxes on.** Depth is how monocular estimators, stereo rigs and RGB-D sensors deliver 3D, and it is the format most robot datasets ship alongside their video.

Runnable example: [`examples/spatial/depth-eval/`](https://github.com/davidjurgens/potato/tree/main/examples/spatial/depth-eval).

## Three things make depth different from an image

A depth map opened as an image is a black rectangle. If it were not, you would still be looking at colours with no idea what distance any of them means.

**The file does not know its own unit.** Millimetres for RealSense, Azure Kinect and NYU-Depth; 1/256 m for the KITTI completion benchmark; metres for anything written from a float array. `depth_scale` is metres per stored unit and defaults to `0.001`. Getting it wrong is silent, because the window rescales and the picture looks the same, which is exactly why the metres readout exists.

**Zero is not a distance.** It is the near-universal "no return" code. Read as depth, it paints a bright wall across every hole in the sensor's coverage. Potato carries non-measurements as NaN and paints them **magenta**, a colour in none of the colormaps, so a hole cannot be mistaken for near or far depth. The info line reports what fraction of the map is holes. A stereo rig facing a textureless wall really does return 80% holes, and an annotator who cannot see that will read them as geometry.

**The interesting range is almost never the full range.** The near/far window defaults to the 2nd and 98th percentile of the *valid* pixels.

## Configuration

```yaml
instance_display:
  fields:
    - key: depth
      type: depth_map
      label: "Depth"
```

Supported formats: 16-bit PNG, 16-bit TIFF, NPY and NPZ, PFM, and EXR.

## Windowing is measurable, not cosmetic

The percentile window is the substantive part, and its effect can be stated as a number. A 16-bit scan whose content sits between 1200 and 1800 renders with **4 grey levels of separation out of 255** under a naive 8-bit cast, and **254** under the percentile window. The same applies to scientific TIFF generally. See [media ingest](/docs/vision-spatial/media-ingest).

## Unprojection

Given intrinsics, a depth map unprojects into the same 3D viewer the [point cloud](/docs/vision-spatial/point-clouds) surface uses, so the same cuboid tools work on it. This is what makes an RGB-D dataset annotatable in 3D without a separate conversion step.

## Related

- [Point clouds](/docs/vision-spatial/point-clouds)
- [Calibration](/docs/vision-spatial/calibration)
- [Guide: depth map annotation](/docs/guides/depth-map-annotation)
- [Source documentation](https://github.com/davidjurgens/potato/blob/main/docs/annotation-types/spatial/depth_maps.md)