Potato टीम से समाचार, ट्यूटोरियल और अपडेट।
A full walkthrough from a folder of Whisper output to labeled speaker turns: choosing an annotation unit, handling diarization, writing the config, running the task, and exporting with the time alignment intact.
Majority vote throws away most of what your annotators told you. Item response theory gives every label a posterior probability and a confidence interval, estimates annotator ability and item difficulty, and finds broken codebook entries, with no gold labels and no LLM.
Potato 2.7 adds labels with error bars, live norming rooms, counterfactual boundary probes, peer-prediction scoring, local voice rationales, one-command dataset reports, and mobile annotation, none of which need an LLM. Plus multi-agent and multimodal agent evaluation.
Get notified about new tutorials, releases, and community highlights.
Potato में कंप्यूटर-उपयोग और GUI एजेंटों के लिए ह्यूमन मूल्यांकन का एक वॉकथ्रू: प्रत्येक क्रिया को आँकना, स्क्रीनशॉट पर क्लिक ग्राउंडिंग की जाँच करना, और टूल कॉल की एक-एक करके समीक्षा करना।
Potato का उपयोग करके यह पता लगाना कि एक मल्टी-एजेंट LLM सिस्टम क्यों विफल हुआ: इंटरैक्शन ग्राफ़, विफलता एट्रिब्यूशन, हैंडऑफ़ समीक्षा, प्रति-एजेंट स्कोरकार्ड, टूल-कंटेंशन टाइमलाइन, और उभरते-व्यवहार टैगिंग।
एजेंट मूल्यांकन में मानवीय समीक्षा का समय सबसे दुर्लभ संसाधन है। Potato 2.6 एक सिग्नल-आधारित ट्राएज कतार को जज-मानव संरेखण के साथ जोड़ता है, ताकि सबसे खराब ट्रेस पहले लोगों तक पहुँचें और आपका LLM जज लगातार बेहतर होता रहे।
Annotator identity shapes labels, so demographics are worth collecting, but only with consent, care, and a reason. A guide to what to ask, what to leave alone, and how to do it in Potato.
A practical guide to deciding when an LLM can annotate your data, where model annotators fail, and how to combine automation with human verification in Potato.
एक ओपन-सोर्स डेटा एनोटेशन टूल कैसे चुनें इस पर एक ईमानदार नज़र, कौन से सवाल वास्तव में विकल्प को सीमित करते हैं, और Label Studio, Prodigy, Doccano, brat और Argilla के बीच Potato कहाँ फ़िट होता है।
Potato now supports coding agent annotation with diff rendering, terminal output display, and process reward schemas. Import traces from Claude Code, Aider, and SWE-Agent.
Step-by-step guide to collecting per-step reward signals for PRM training using Potato. Covers first-error mode, per-step annotation, and export to training pipelines.
Compare Potato with LangSmith, Langfuse, Labelbox, and Scale AI for agent evaluation: trace rendering, per-step and multi-agent annotation, multimodal-agent review, coding agents, live observation, pricing, and self-hosting.
Set up multi-criteria rubric evaluation with custom criteria, configurable rating scales, and dimension weights for systematic AI agent evaluation using Potato's rubric_eval.
Potato के web agent trace display का उपयोग करके स्वायत्त web browsing agents का मूल्यांकन कैसे करें — चरण-दर-चरण screenshots, SVG overlays, और per-step annotation schemas के साथ।
Potato 2.2.0 adds 9 new annotation schemas, a pluggable export system, MACE competence estimation, 55 validated survey instruments, and remote data sources.
Annotate legal documents in Potato, contracts, court filings, and regulatory text, with span labeling, entity extraction, and privacy-first self-hosted deployment.
Best practices for annotating medical images in Potato, DICOM display, radiology report labeling, adverse event extraction, and IRB-compliant self-hosted deployment.
Build side-by-side image comparison tasks in Potato for preference ranking, A/B testing, and visual quality assessment, with randomized order and pairwise scoring.
Potato 2.0 ships AI-powered pre-annotation with OpenAI and Claude, multimedia support for audio and video, active learning, bounding box annotation, and a redesigned UI.
Build custom annotation interfaces in Potato using HTML templates, CSS styling, and JavaScript, side-by-side layouts, custom widgets, and embedded media displays.
Build a pronunciation quality annotation task in Potato with audio playback, waveform visualization, Likert rating scales, and per-recording free-text feedback fields.
Calculate and interpret Cohen's Kappa, Fleiss' Kappa, and Krippendorff's Alpha for Potato annotation projects, with Python code examples and interpretation guidelines.
Best practices for ensuring annotation quality in annotation projects, including practical strategies you can implement with and beyond Potato.
An overview of multi-object tracking annotation concepts and how Potato's video annotation capabilities can support basic tracking workflows.
Create an audio emotion classification task in Potato with interactive waveform display, playback speed controls, Likert scales, and configurable emotion label sets.