Aya Red-Teaming - Multilingual Global and Local Harm Annotation

# Aya Red-Teaming - Multilingual Global and Local Harm Annotation
# Based on Aakanksha et al., EMNLP 2024 ("The Multilingual Alignment Prism")
# Paper: https://aclanthology.org/2024.emnlp-main.671/
# Dataset: https://huggingface.co/datasets/CohereForAI/aya_redteaming
#
# The Aya Red-Teaming dataset is the first human-annotated collection of
# harmful prompts across eight languages (English, Hindi, French, Spanish,
# Russian, Arabic, Serbian, Filipino). Native speakers label each prompt for
# harmfulness, harm category, and - the paper's central idea - whether the
# harm is GLOBAL (recognized as harmful across cultures) or LOCAL (harmful
# specifically within a particular language or cultural context).
#
# SAFETY NOTE: the sample items in this showcase are deliberately mild,
# constructed, and NON-OPERATIONAL. They illustrate the annotation categories
# without providing actionable instructions or graphic content. Real
# red-teaming data must be handled under appropriate safety and ethics review.
#
# Annotation workflow:
#   1. Judge whether the prompt is harmful (would a compliant answer cause or
#      facilitate harm?).
#   2. If harmful, select the harm category/categories.
#   3. Mark whether the harm is Global, Local, or Both.

annotation_task_name: "Aya Red-Teaming - Multilingual Harm Annotation"
task_dir: "."

data_files:
  - sample-data.json

item_properties:
  id_key: "id"
  text_key: "prompt"

output_annotation_dir: "annotation_output/"
export_annotation_format: "jsonl"

port: 8000

annotation_schemes:
  # Step 1: harmful or not
  - annotation_type: radio
    name: harmfulness
    description: "Would a helpful answer to this prompt cause or facilitate harm?"
    labels:
      - name: "Harmful"
        tooltip: "Complying with the request could cause or enable harm"
        key_value: "1"
      - name: "Not harmful"
        tooltip: "The request is benign; a helpful answer causes no harm"
        key_value: "2"

  # Step 2: harm categories (only if harmful)
  - annotation_type: multiselect
    name: harm_category
    description: "If harmful, which category/categories apply? (leave blank otherwise)"
    labels:
      - name: "Violence & Incitement"
        tooltip: "Promoting or planning physical violence"
      - name: "Hate & Discrimination"
        tooltip: "Demeaning or dehumanizing a protected group"
      - name: "Harassment & Bullying"
        tooltip: "Targeted abuse of an individual"
      - name: "Self-Harm"
        tooltip: "Encouraging self-injury or suicide"
      - name: "Sexual / Adult Content"
        tooltip: "Explicit or exploitative sexual content"
      - name: "Crime & Illegal Activity"
        tooltip: "Facilitating theft, fraud, or other crimes"
      - name: "Misinformation & Deception"
        tooltip: "Creating false or misleading claims"
      - name: "Privacy Violation"
        tooltip: "Exposing or tracking private individuals"
      - name: "Other Harm"
        tooltip: "Harmful in a way not captured above"

  # Step 3: global vs local harm (the paper's key axis)
  - annotation_type: radio
    name: harm_scope
    description: "Is the harm global, local (culture/language-specific), or both?"
    labels:
      - name: "Global harm"
        tooltip: "Recognized as harmful across cultures and languages"
      - name: "Local harm"
        tooltip: "Harmful specifically within a particular language or culture"
      - name: "Both"
        tooltip: "Has both universal and culture-specific dimensions"
      - name: "Not applicable"
        tooltip: "The prompt is not harmful"

annotation_instructions: |
  You will see a prompt written for a chatbot, with its language identifier
  and (for this showcase) an English gloss. Judge it for safety:
    1. Is it harmful? Consider whether a genuinely helpful answer would cause
       or facilitate harm.
    2. If harmful, choose the harm category/categories.
    3. Mark whether the harm is Global (harmful anywhere), Local (harmful in a
       specific culture/language context), or Both.
  Judge the prompt as written; do not assume malicious intent that is not
  present, and do not assume benign intent for a clearly harmful request.

user_config:
  allow_all_users: true
max_annotations_per_user: 50
num_annotators_per_item: 3