Skip to content
Potato
Why Potato
Docs
Blog
Showcase
Playground
Tools
Community
Get Started
Open main menu
Showcase
/
Preference Learning
Preference Learning
32 annotation designs you can copy and run. Each ships a
config.yaml
and sample data.
All preference learning designs
AlpacaEval: Instruction-Following Preference Evaluation
AlpacaFarm Preference Simulation
Arena Hard Auto - LLM Pairwise Evaluation
Aya Red-Teaming - Multilingual Global and Local Harm Annotation
BeaverTails Safety Preference
Chatbot Arena: Pairwise LLM Preference Evaluation
CodePRM Code Process Reward
CodeUltraFeedback: Code Preference Evaluation
Conjoint Analysis: Immigrant Admission Preferences
Constitutional AI Harmlessness Evaluation
Conversation Tree
DPO Preference Data Collection
FLASK Skill-based Rubric Evaluation
HelpSteer Multi-Attribute Rating
HH-RLHF Pairwise Preference
InstructGPT Instruction Following
Interpretable Semantic Textual Similarity
Moral Stories Annotation
OpenAssistant Conversation Quality
Pairwise Preference
Pairwise Preference with Rationale
PRM800K Step-by-Step Verification
RewardBench - Reward Model Evaluation
RewardBench: Reward Model Evaluation via Pairwise Preference
SafeRLHF Dual-Dimension Preference
SaGA Gesture-Speech Alignment Multi-Tier Annotation
SPIN Self-Play Preference Annotation
Summary Preference Comparison
SWE-PRM Process Reward Labels for Coding Agents
UltraFeedback Rubric Evaluation
UltraFeedback: Fine-Grained AI Preference Dataset
WebGPT Answer Comparison
Other categories
Text Annotation (212)
Image Annotation (41)
Video Annotation (32)
Audio Annotation (33)
Evaluation Tasks (39)
Comparison Tasks (4)
Surveys (63)