ब्लॉग

मानव एनोटेशन के साथ RAG सिस्टम का मूल्यांकन कैसे करें

रिट्रीवल-ऑगमेंटेड जेनरेशन के मूल्यांकन के लिए एक व्यावहारिक मार्गदर्शिका: रिट्रीवल प्रासंगिकता और उत्तर की निष्ठा को अलग-अलग स्कोर करें, और Potato में स्पैन एनोटेशन के साथ असमर्थित दावों को चिह्नित करें।

Apr 8, 2026

Announcements9 min read

Announcing Coding Agent Annotation: Evaluate Claude Code, Aider, and SWE-Agent Traces

Potato now supports coding agent annotation with diff rendering, terminal output display, and process reward schemas. Import traces from Claude Code, Aider, and SWE-Agent.

Mar 27, 2026

How to Collect Process Reward Data for Training Better Coding Agents

Step-by-step guide to collecting per-step reward signals for PRM training using Potato. Covers first-error mode, per-step annotation, and export to training pipelines.

Mar 26, 202611 min read

Watch, Pause, and Rewind: Live Coding Agent Observation in Potato

Tutorial for setting up live coding agent observation with Ollama, Anthropic API, or Claude Agent SDK. Includes pause, rollback, branching, and trajectory export.

Mar 25, 202612 min read

Guides14 min read

Comparing AI Agents Side by Side: Binary, Scale, and Multi-Dimension Modes

Set up pairwise agent comparison in Potato with three modes: binary preference, continuous scale, and per-dimension multi-criteria judgment with required justification.

Mar 24, 2026

Guides15 min read

Per-Step Error Localization: Using Trajectory Evaluation to Find Where Agents Fail

Use Potato's trajectory_eval schema for per-step error localization with hierarchical error taxonomies, severity scoring, and running score tracking across agent traces.

Mar 23, 2026

Tutorials12 min read

GitHub PR-Style Code Review for AI Coding Agents

Set up GitHub PR-style code review annotation in Potato with inline diff comments, file-level quality ratings, and approve or reject verdicts for coding agent output.

Mar 22, 2026

Guides13 min read

Potato vs LangSmith and Langfuse for Agent Evaluation: A Practical Comparison

Compare Potato with LangSmith and Langfuse for AI agent evaluation across trace rendering, annotation, coding agent support, live observation, pricing, and self-hosting.

Mar 21, 2026

Stay updated

Get notified about new tutorials, releases, and community highlights.

MT-Bench-Style Rubric Evaluation for AI Agents in Potato

Set up multi-criteria rubric evaluation with custom criteria, configurable rating scales, and dimension weights for systematic AI agent evaluation using Potato's rubric_eval.

Mar 20, 202616 min read

Release

Potato 2.4.0: Web Agent Annotation, Live Evaluation, and HuggingFace Integration

Potato 2.4.0 ships web agent trace review, real-time live agent evaluation, an LLM chat sidebar, HuggingFace Hub export, webhooks, SSO/OAuth, and five active learning strategies.

Mar 17, 20265 min read

Announcements11 min read

Potato 2.3: Agentic Annotation, Solo Mode, और Human Evaluation का भविष्य

Potato 2.3.0 में agentic annotation के साथ 12 trace format converters, Solo Mode, Best-Worst Scaling, SSO/OAuth, Parquet export, और 15 demo projects प्रस्तुत किए गए हैं।

Mar 9, 2026

Guides10 min read

AI Agents का मूल्यांकन: Agent Traces की Human Annotation के लिए संपूर्ण गाइड

Potato की agentic annotation सुविधाओं का उपयोग करके AI agent outputs का human evaluation कैसे सेट करें — trace import से annotation schema design तक।

Mar 8, 2026

Tutorials12 min read

Solo Mode: एक Annotator कैसे 10,000 Examples Label कर सकता है

Human-LLM collaboration के साथ बड़े datasets को कुशलतापूर्वक label करने के लिए Potato के Solo Mode का उपयोग करने पर चरण-दर-चरण tutorial, annotation लागत को 90% तक कम करते हुए।

Mar 7, 2026

Guides11 min read

Web Browsing Agents को Annotate करना: WebArena Traces से Human Evaluation तक

Potato के web agent trace display का उपयोग करके स्वायत्त web browsing agents का मूल्यांकन कैसे करें — चरण-दर-चरण screenshots, SVG overlays, और per-step annotation schemas के साथ।

Mar 6, 2026

Potato 2.2: Events, Entity Linking, Export, and 55 Survey Instruments

Potato 2.2.0 adds 9 new annotation schemas, a pluggable export system, MACE competence estimation, 55 validated survey instruments, and remote data sources.

Feb 20, 20265 min read

Potato 2.1: Instance Display, Visual AI, and Span Linking

Potato 2.1.0 brings the instance display system, visual AI support for image and video annotation, span linking, multi-field spans, and layout customization.

Feb 14, 20266 min read

Guides11 min read

Image और Video Annotation को तेज़ करने के लिए Visual AI का उपयोग

YOLO, Ollama, OpenAI, और Claude के साथ image और video tasks के लिए AI-powered object detection, pre-annotation, और classification सेट करें।

Feb 14, 2026

Guides8 min read

Migrating from Label Studio to Potato

Migrate from Label Studio to Potato, convert project configs, annotation schemas, and exported data with this step-by-step guide covering common annotation types.

Jan 20, 2026

Running Annotation Studies on Prolific

Integrate Potato with Prolific for crowdsourced annotation, completion URL setup, participant ID tracking, attention checks, payment configuration, and quality filtering.

Jan 20, 2026

Guides6 min read

Legal Document Annotation Best Practices

Annotate legal documents in Potato, contracts, court filings, and regulatory text, with span labeling, entity extraction, and privacy-first self-hosted deployment.

Jan 18, 2026

Medical Image Annotation with Potato

Best practices for annotating medical images in Potato, DICOM display, radiology report labeling, adverse event extraction, and IRB-compliant self-hosted deployment.

Jan 18, 20265 min read

Polygon Annotation for Segmentation Tasks

Configure polygon drawing tools in Potato for image segmentation tasks, with tips for complex shapes, overlapping regions, multi-class polygons, and export to COCO format.

Jan 18, 20264 min read

Active Learning: स्मार्ट तरीके से Annotate करें, कठिन नहीं

Annotations को prioritize करने और कुल labeling effort को 50% तक कम करने के लिए uncertainty sampling का उपयोग कैसे करें।

Tutorials4 min read

Drawing Bounding Boxes for Object Detection

Set up bounding box annotation for computer vision in Potato, configure label colors, minimum box size, multi-class support, validation rules, and COCO/YOLO export.

Guides5 min read

Exporting Annotations to Hugging Face Datasets

Convert Potato annotations to HuggingFace Datasets format, covering JSON export structure, dataset card generation, Hub upload, and integration with transformers training.

Tutorials7 min read

Image Comparison and Preference Tasks

Build side-by-side image comparison tasks in Potato for preference ranking, A/B testing, and visual quality assessment, with randomized order and pairwise scoring.

Introducing Potato 2.0: AI-Powered Annotation

Potato 2.0 ships AI-powered pre-annotation with OpenAI and Claude, multimedia support for audio and video, active learning, bounding box annotation, and a redesigned UI.

Jan 15, 20263 min read

Automatic Keyword Highlighting

Configure AI-powered keyword highlighting in Potato to draw annotator attention to important terms. Covers OpenAI, Claude, and custom keyword list configuration.

Jan 15, 20264 min read

Guides7 min read

Deploying to Amazon Mechanical Turk

Run Potato annotation tasks on Amazon Mechanical Turk, HIT configuration, qualification tests, approval and rejection workflows, bonus payments, and quality monitoring.

Creating a Sentiment Analysis Task

Build a complete sentiment classification task with radio buttons, tooltips, and keyboard shortcuts for efficient labeling.

Guides7 min read

Content Moderation Annotation Setup

Configure Potato for toxicity detection, hate speech classification, and sensitive content labeling with annotator wellbeing in mind.

Jan 12, 2026

Guides5 min read

Custom HTML Templates in Potato

Build custom annotation interfaces in Potato using HTML templates, CSS styling, and JavaScript, side-by-side layouts, custom widgets, and embedded media displays.

Jan 12, 2026

Pronunciation Assessment Annotation

Build a pronunciation quality annotation task in Potato with audio playback, waveform visualization, Likert rating scales, and per-recording free-text feedback fields.

Jan 12, 20266 min read

Speaker Diarization Annotation

Build a speaker identification task in Potato with interactive audio waveforms, timestamp markers, speaker label assignment, and inter-annotator agreement measurement.

Jan 12, 20264 min read

Tutorials3 min read

Video Annotation with Frame-by-Frame Controls

Set up video annotation in Potato with frame-by-frame navigation, timestamp markers, temporal event labeling, and per-segment classification using radio or span schemas.

Jan 12, 2026

Audio Event Detection and Tagging

Set up annotation for detecting specific sounds like speech, music, applause, or environmental noises with timestamp spans.

Jan 10, 2026

Tutorials2 min read

5 मिनट में Potato के साथ शुरुआत करें

Installation से अपने पहले labeled data तक, Potato के साथ अपना पहला annotation project सेट करने के लिए एक त्वरित गाइड।

Jan 10, 2026

Guides6 min read

Measuring Inter-Annotator Agreement

Calculate and interpret Cohen's Kappa, Fleiss' Kappa, and Krippendorff's Alpha for Potato annotation projects, with Python code examples and interpretation guidelines.

Jan 10, 2026

Potato Featured at EMNLP 2022

Our paper on Potato was accepted at EMNLP 2022. Learn about the research behind the tool and how to cite it in your work.

Jan 10, 20263 min read

Quality Control for Crowdsourced Annotation

Best practices for ensuring annotation quality in annotation projects, including practical strategies you can implement with and beyond Potato.

Jan 10, 20267 min read

Tutorials4 min read

Image Classification with Potato

Set up image classification in Potato with thumbnail previews, zoom controls, multi-label checkbox schemas, and keyboard shortcuts for high-throughput visual annotation.

Smart Annotation Hints के लिए LLMs को Integrate करना

OpenAI, Claude, या Gemini का उपयोग करके अपने annotators को intelligent hints और suggestions प्रदान करना सीखें।

Music Genre Classification Annotation

Create a music annotation task in Potato with waveform visualization, 30-second preview playback, hierarchical genre label trees, and pairwise preference comparisons.

Guides6 min read

Deploying Potato on a Server

Deploy Potato to production with Docker, nginx, SSL, and cloud platforms including AWS, GCP, and Azure. Includes systemd service configuration and scaling tips.

Multi-Object Tracking Annotation

An overview of multi-object tracking annotation concepts and how Potato's video annotation capabilities can support basic tracking workflows.

Jan 6, 20264 min read

Classifying Emotions in Speech

Create an audio emotion classification task in Potato with interactive waveform display, playback speed controls, Likert scales, and configurable emotion label sets.

Jan 5, 20266 min read

Tutorials4 min read

Building Your First NER Annotation Task

Step-by-step guide to building a named entity recognition task in Potato, span annotation config, label colors, overlapping spans, keyboard shortcuts, and CoNLL export.

Jan 5, 2026

Understanding Potato Data Formats

A deep dive into JSON and JSONL data formats in Potato, structuring instances for text, image, audio, video, and multimodal annotation with real config examples.

Jan 5, 2026