博客

来自 Potato 团队的新闻、教程和更新。

GuidesMay 20, 2026

2026 年如何选择开源标注工具

坦诚地探讨如何挑选开源数据标注工具、哪些问题才真正能缩小选择范围，以及在 Label Studio、Prodigy、Doccano、brat 和 Argilla 之中 Potato 处于什么位置。

Potato Team1 min read

Guides1 min read

用片段标注发现幻觉

通过高亮出确切的词语并标注错在哪里，以 MQM 风格在 Potato 中用片段标注捕捉模型幻觉和事实错误。

May 6, 2026

Guides1 min read

你究竟需要多少名标注者？

为标注项目确定标注者数量和重叠度：客观与主观任务的经验法则、覆盖与重叠之间的权衡，以及如何在 Potato 中进行设置。

Apr 22, 2026

Guides1 min read

如何用人工标注评估 RAG 系统

评估检索增强生成的实用指南：分别衡量检索相关性和回答忠实度，并在 Potato 中用片段标注标出缺乏依据的论断。

Apr 8, 2026

Announcements9 min read

Announcing Coding Agent Annotation: Evaluate Claude Code, Aider, and SWE-Agent Traces

Potato now supports coding agent annotation with diff rendering, terminal output display, and process reward schemas. Import traces from Claude Code, Aider, and SWE-Agent.

Mar 27, 2026

Guides

How to Collect Process Reward Data for Training Better Coding Agents

Step-by-step guide to collecting per-step reward signals for PRM training using Potato. Covers first-error mode, per-step annotation, and export to training pipelines.

Mar 26, 202611 min read

Tutorials

Watch, Pause, and Rewind: Live Coding Agent Observation in Potato

Tutorial for setting up live coding agent observation with Ollama, Anthropic API, or Claude Agent SDK. Includes pause, rollback, branching, and trajectory export.

Mar 25, 202612 min read

Guides14 min read

Comparing AI Agents Side by Side: Binary, Scale, and Multi-Dimension Modes

Set up pairwise agent comparison in Potato with three modes: binary preference, continuous scale, and per-dimension multi-criteria judgment with required justification.

Mar 24, 2026

Guides15 min read

Per-Step Error Localization: Using Trajectory Evaluation to Find Where Agents Fail

Use Potato's trajectory_eval schema for per-step error localization with hierarchical error taxonomies, severity scoring, and running score tracking across agent traces.

Mar 23, 2026

Tutorials12 min read

GitHub PR-Style Code Review for AI Coding Agents

Set up GitHub PR-style code review annotation in Potato with inline diff comments, file-level quality ratings, and approve or reject verdicts for coding agent output.

Mar 22, 2026

Guides13 min read

Potato vs LangSmith and Langfuse for Agent Evaluation: A Practical Comparison

Compare Potato with LangSmith and Langfuse for AI agent evaluation across trace rendering, annotation, coding agent support, live observation, pricing, and self-hosting.

Mar 21, 2026

Stay updated

Get notified about new tutorials, releases, and community highlights.

Tutorials

MT-Bench-Style Rubric Evaluation for AI Agents in Potato

Set up multi-criteria rubric evaluation with custom criteria, configurable rating scales, and dimension weights for systematic AI agent evaluation using Potato's rubric_eval.

Mar 20, 202616 min read

Release

Potato 2.4.0: Web Agent Annotation, Live Evaluation, and HuggingFace Integration

Potato 2.4.0 ships web agent trace review, real-time live agent evaluation, an LLM chat sidebar, HuggingFace Hub export, webhooks, SSO/OAuth, and five active learning strategies.

Mar 17, 20265 min read

Announcements6 min read