Skip to content
beginnertext

Toxicity Detection

Multi-label classification for identifying various types of toxic content including hate speech, threats, and harassment.

Try it live — no install

Boot the real Potato server in your browser (WebAssembly) and annotate with this exact config. Nothing leaves your machine.

Run live in your browser

Get This Design

This design is available in our showcase. Copy the configuration below to get started.

Quick start:

bash
# Create your project folder
mkdir toxicity-detection
cd toxicity-detection
# Save config.yaml from this page into this folder
potato start config.yaml
Toxicity Detection annotation interface preview

Configuration File

yaml
# Toxicity Detection Configuration
# Generated by Potato Annotation Showcase

port: 8000
annotation_task_name: "Toxicity Detection"
task_dir: "."

# Data configuration
data_files:
  - "data.json"

item_properties:
  id_key: id
  text_key: text

# Output
output_annotation_dir: "annotation_output/"
export_annotation_format: "jsonl"
instance_display:
  fields:
    - key: text
      type: text
      label: "Comment"

# Annotation schemes
annotation_schemes:
  # Multi-label toxicity categories
  - annotation_type: multiselect
    name: toxicity_labels
    description: "Select ALL toxicity categories that apply to this text"
    labels:
      - name: Toxic
        key_value: "1"
        tooltip: "Rude, disrespectful, or unreasonable content likely to make someone leave a discussion"
      - name: Severe Toxic
        key_value: "2"
        tooltip: "Extremely hateful, aggressive, or disrespectful content"
      - name: Obscene
        key_value: "3"
        tooltip: "Lewd, indecent, or profane language"
      - name: Threat
        key_value: "4"
        tooltip: "Content that expresses intention to inflict harm"
      - name: Insult
        key_value: "5"
        tooltip: "Insulting, inflammatory, or provocative content directed at a person"
      - name: Identity Hate
        key_value: "6"
        tooltip: "Hateful content targeting someone's identity (race, religion, gender, etc.)"
    sequential_key_binding: true

  # Overall severity rating
  - annotation_type: radio
    name: overall_severity
    description: "Rate the overall severity of toxicity"
    labels:
      - name: Not Toxic
        key_value: "q"
      - name: Mildly Toxic
        key_value: "w"
      - name: Moderately Toxic
        key_value: "e"
      - name: Severely Toxic
        key_value: "r"
    sequential_key_binding: true

# User configuration
require_password: false

Sample Data

json
[
  {
    "id": "toxicity_detection_1",
    "text": "I disagree with the proposal, and here is why."
  },
  {
    "id": "toxicity_detection_2",
    "text": "Nobody asked for your opinion, as usual."
  }
]

// ... and 3 more items

Details

Annotation Types

multiselectradio

Domain

NLPContent Moderation

Use Cases

Toxicity DetectionContent ModerationHate Speech Detection

Tags

toxicitycontent moderationhate speechmulti-label

Related Designs

AfriHate - Hate and Abusive Language for African Languages

Multilingual content-moderation annotation following the AfriHate scheme (Muhammad et al., NAACL 2025), a collection of hate speech and abusive language datasets for 15 African languages including Amharic, Hausa, Igbo, Yoruba, Swahili, Somali and isiZulu. Native speakers familiar with the regional culture assign each tweet a three-way label - hate, abusive, or neutral - and, when the tweet is hateful, mark the targeted attribute (e.g. ethnicity, religion, politics, gender). Sample items are mild, constructed illustrations that avoid real slurs or targeting of real groups; an English gloss is provided for reference only.

radiomultiselect

HateXplain - Explainable Hate Speech Detection

Multi-task hate speech annotation with classification (hate/offensive/normal), target community identification, and rationale span highlighting. Based on the HateXplain benchmark (Mathew et al., AAAI 2021) - the first dataset covering classification, target identification, and rationale extraction.

radiomultiselect

Toxic Spans Detection

Character-level toxic span annotation based on SemEval-2021 Task 5 (Pavlopoulos et al., 2021). Instead of binary toxicity classification, annotators identify the specific words/phrases that make a comment toxic, enabling more nuanced content moderation.

multiselectradio