Skip to content
beginnertext

Semantic Similarity

Rate the semantic similarity between pairs of sentences on a continuous scale.

Try it live — no install

Boot the real Potato server in your browser (WebAssembly) and annotate with this exact config. Nothing leaves your machine.

Run live in your browser

Get This Design

View on GitHub(opens in a new tab)

Clone or download from the repository

Quick start:

bash
git clone https://github.com/davidjurgens/potato-showcase.git
cd potato-showcase/templates/text/semantic-similarity
potato start config.yaml

Configuration File

yaml
# Semantic Similarity Configuration
# Rate similarity between sentence pairs

task_dir: "."
annotation_task_name: "Semantic Similarity"

data_files:
  - "data/sentence_pairs.json"

item_properties:
  id_key: "id"
  text_key: "display"
  text_display_key: "display"

user_config:
  allow_all_users: true

annotation_schemes:
  - annotation_type: "slider"
    name: "similarity_score"
    description: "How similar are these two sentences in meaning?"
    min_value: 0
    max_value: 5
    starting_value: 2.5
    step: 0.5
    min_label: "Completely different"
    max_label: "Identical meaning"
    show_value: true

  - annotation_type: "radio"
    name: "relationship_type"
    description: "What is the relationship between the sentences?"
    labels:
      - name: "Paraphrase"
        tooltip: "Same meaning, different words"
      - name: "Entailment"
        tooltip: "If A is true, B must be true"
      - name: "Partial overlap"
        tooltip: "Share some meaning but differ in details"
      - name: "Contradiction"
        tooltip: "Opposite or conflicting meanings"
      - name: "Unrelated"
        tooltip: "Different topics entirely"

  - annotation_type: "multiselect"
    name: "similarity_aspects"
    description: "In what ways are they similar?"
    labels:
      - name: "Same topic"
      - name: "Same entities mentioned"
      - name: "Same event/action"
      - name: "Same sentiment"
      - name: "Similar structure"
      - name: "Shared vocabulary"

  - annotation_type: "likert"
    name: "confidence"
    description: "How confident are you in your rating?"
    size: 5
    min_label: "Not confident"
    max_label: "Very confident"

output: "annotation_output/"

output_annotation_dir: "annotation_output/"
output_annotation_format: "json"

Sample Data

json
[
  {
    "id": "sim_001",
    "sentence1": "The cat sat on the mat.",
    "sentence2": "A feline was resting on the rug.",
    "display": "**Sentence 1:** The cat sat on the mat.\n\n**Sentence 2:** A feline was resting on the rug."
  },
  {
    "id": "sim_002",
    "sentence1": "The stock market crashed yesterday.",
    "sentence2": "Stock prices plummeted in yesterday's trading.",
    "display": "**Sentence 1:** The stock market crashed yesterday.\n\n**Sentence 2:** Stock prices plummeted in yesterday's trading."
  }
]

// ... and 3 more items

Details

Annotation Types

likertmultiselectradioslider

Domain

nlplinguistics

Use Cases

similarityparaphrase-detection

Tags

similaritysemanticparaphrasesentencesnlp

Found an issue or want to improve this design?

Open an Issue(opens in a new tab)

Related Designs

Fact Verification

Verify claims as supported, refuted, or not enough information based on provided evidence.

likertmultiselect

Hate Speech Detection

Identify and categorize hate speech, offensive language, and toxic content in text.

likertmultiselect

Intent Classification

Classify user utterances into intents for chatbot and virtual assistant training.

likertmultiselect