# 3D Point Clouds and Cuboids

Source: https://www.potatoannotator.com/docs/vision-spatial/point-clouds

**Potato annotates lidar and photogrammetry point clouds with oriented 3D cuboids, points, polylines and per-point segments, in the browser.** Octree level of detail keeps a multi-million-point scan interactive on a laptop, and camera calibration projects every box into the images so you can verify in 2D while editing in 3D.

```yaml
annotation_schemes:
  - annotation_type: spatial_annotation
    name: objects
    description: "Put a 3D box around every vehicle and pedestrian."
    source_field: point_cloud
    tools: [cuboid_3d, point_3d]
    labels:
      - {name: car, color: "#FF6B6B", key_value: "1"}
      - {name: pedestrian, color: "#FFD93D", key_value: "2"}
    color_mode: height
    point_size: 2.0
    max_points: 400000
```

Runnable example: [`examples/spatial/kitti-cuboids/`](https://github.com/davidjurgens/potato/tree/main/examples/spatial/kitti-cuboids).

## Formats

Clouds are read **server-side** and converted into one compact binary the browser understands. Parsing each format in JavaScript would mean four parsers, four sets of endianness and record-layout bugs, and re-parsing a two-million-point scan on every page load.

| Format | Notes |
|---|---|
| KITTI velodyne `.bin` | Raw `float32` x, y, z, intensity, no header |
| [PCD](https://pointclouds.org/documentation/tutorials/pcd_file_format.html) | `ascii`, `binary`, and `binary_compressed` (LZF) |
| [PLY](https://en.wikipedia.org/wiki/PLY_(file_format)) | `ascii`, `binary_little_endian`, `binary_big_endian` |
| [LAS](https://en.wikipedia.org/wiki/LAS_file_format) | 1.0 to 1.4, point record formats 0 to 3 and 6 to 8 |
| `.xyz` / `.pts` | `x y z [r g b]` per line |
| LAZ | Not read directly. Convert first: `laszip -i scan.laz -o scan.las` |

Colour is read where a format carries it; intensity is read from KITTI, PCD and LAS. Asking for `color_mode: rgb` on a file with no colour falls back to height **and says so** in the status line, rather than quietly rendering something that looks like the mode you asked for.

## Tools

| Tool | What it draws |
|---|---|
| `cuboid_3d` | An oriented 3D box |
| `point_3d` | A single 3D point |
| `polyline_3d` | A path through the cloud |
| `segment_3d` | A per-point label over a selected set |

## Rotation is a quaternion, not a yaw angle

Potato stores a cuboid's rotation as a full [quaternion](https://en.wikipedia.org/wiki/Quaternion). Most formats store a single yaw, so this looks like over-engineering until a round trip loses something.

It is what makes KITTI import lossless, including the roughly 0.85° camera-to-lidar mounting tilt that a yaw-only field discards without saying so. Exporting back to KITTI does have to drop pitch and roll — and it reports **how much** orientation it discarded rather than silently flattening the box.

The gap between what Potato can represent and what a format can store is crossed in the format-specific code, loudly, rather than in the storage layer, silently.

## Agreement uses exact rotated 3D IoU

Two annotators boxing the same car will not produce the same box. Potato computes agreement with exact rotated 3D IoU, which is correct under arbitrary rotation rather than only for axis-aligned boxes.

That matters as soon as the data is not level: drone, handheld and indoor scans all produce boxes with real pitch and roll, and an axis-aligned approximation reports disagreement that is an artefact of the measure rather than of the annotators. See [geometry agreement](/docs/measurement/geometry-agreement).

## Working at scale

- **Octree level of detail** is on by default (`lod: true`). Points load by region and by distance, so a large scan stays interactive.
- **`max_points`** caps decimation. It is a viewer setting, not a data setting.
- **Orthographic slab panels** (`mpr: true`) give three axis-aligned views for precise, fully keyboard-driven editing. Drag one face while the opposite stays fixed.

One bug worth knowing about, since it affected stored data rather than display: `segment_3d` point indices used to assume a fixed decimation, so lowering `max_points` silently re-pointed existing per-point labels at different points. Indices now refer to the source file, so a viewer setting cannot move an annotation.

## Related

- [Calibration and 2D verification](/docs/vision-spatial/calibration)
- [Depth maps](/docs/vision-spatial/depth-maps) — unproject depth into the same viewer
- [Geometry agreement](/docs/measurement/geometry-agreement)
- [Guide: point cloud annotation](/docs/guides/point-cloud-annotation)
- [Source documentation](https://github.com/davidjurgens/potato/blob/main/docs/annotation-types/spatial/point_cloud.md)