Skip to content
beginnersurvey

Tweet Intimacy Analysis

Rating the level of intimacy expressed in tweets on a 5-point scale, covering multilingual social media content. Based on SemEval-2023 Task 9 (Pei et al.).

Q1: Rate your experience12345Q2: Primary use case?ResearchIndustryEducationQ3: Additional feedback

Configuration Fileconfig.yaml

This Potato config reproduces the annotation task. Save it as config.yaml and run potato start config.yaml to try it.

yaml
# Tweet Intimacy Analysis
# Based on Pei et al., SemEval 2023
# Paper: https://aclanthology.org/2023.semeval-1.309/
# Dataset (MINT): https://codalab.lisn.upsaclay.fr/competitions/7096
#
# This task asks annotators to rate the level of intimacy expressed in
# tweets on a 5-point Likert scale. Intimacy refers to how personal,
# private, or emotionally close the content of the tweet is.
#
# Scale:
# 1 - Not Intimate: Public, impersonal content (news, facts)
# 2 - Slightly Intimate: Mildly personal opinions or preferences
# 3 - Moderately Intimate: Personal experiences or feelings shared broadly
# 4 - Intimate: Private feelings, personal relationships, or vulnerabilities
# 5 - Very Intimate: Deeply personal disclosures, confessions, or private matters

annotation_task_name: "Tweet Intimacy Analysis"
task_dir: "."

data_files:
  - sample-data.json

item_properties:
  id_key: "id"
  text_key: "text"

output_annotation_dir: "annotation_output/"
output_annotation_format: "json"

port: 8000
server_name: localhost

annotation_schemes:
  - annotation_type: likert
    name: intimacy_rating
    description: "How intimate is the content of this tweet?"
    min_label: "Not Intimate"
    max_label: "Very Intimate"
    size: 5

annotation_instructions: |
  You will see a tweet and its language. Rate the level of intimacy expressed in the tweet
  on a scale from 1 (Not Intimate) to 5 (Very Intimate).
  - 1: Public, impersonal content (news headlines, factual statements)
  - 2: Slightly personal opinions or general preferences
  - 3: Moderately personal experiences or feelings shared broadly
  - 4: Private feelings, personal relationships, or vulnerabilities
  - 5: Deeply personal disclosures, confessions, or private matters

html_layout: |
  <div style="padding: 15px; max-width: 800px; margin: auto;">
    <div style="background: #faf5ff; border: 1px solid #e9d5ff; border-radius: 8px; padding: 12px; margin-bottom: 12px;">
      <strong style="color: #7e22ce;">Language:</strong>
      <span style="font-size: 15px; margin-left: 8px;">{{language}}</span>
    </div>
    <div style="background: #f0f9ff; border: 1px solid #bae6fd; border-radius: 8px; padding: 16px; margin-bottom: 16px;">
      <strong style="color: #0369a1;">Tweet:</strong>
      <p style="font-size: 16px; line-height: 1.7; margin: 8px 0 0 0;">{{text}}</p>
    </div>
  </div>

allow_all_users: true
instances_per_annotator: 50
annotation_per_instance: 3
allow_skip: true
skip_reason_required: false

Sample Datasample-data.json

json
[
  {
    "id": "intimacy_001",
    "text": "Breaking: The Federal Reserve has announced a 0.25% interest rate hike, marking the fourth consecutive increase this year.",
    "language": "English"
  },
  {
    "id": "intimacy_002",
    "text": "I can't stop thinking about what happened between us last night. I don't know if I'll ever feel the same way about anyone again.",
    "language": "English"
  }
]

// ... and 8 more items

Get This Design

View on GitHub

Clone or download from the repository

Quick start:

git clone https://github.com/davidjurgens/potato-showcase.git
cd potato-showcase/semeval/2023/task09-tweet-intimacy
potato start config.yaml

Dataset & paper

Pei et al., SemEval 2023

Official dataset ↗Read the paper ↗

Citation (BibTeX)

bibtex
@inproceedings{pei-etal-2023-semeval,
    title = "{S}em{E}val-2023 Task 9: Multilingual Tweet Intimacy Analysis",
    author = "Pei, Jiaxin  and Silva, V{\'\i}tor  and Bos, Maarten  and Liu, Yozen  and Neves, Leonardo  and Jurgens, David  and Barbieri, Francesco",
    booktitle = "Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023)",
    month = jul,
    year = "2023",
    address = "Toronto, Canada",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2023.semeval-1.309",
    doi = "10.18653/v1/2023.semeval-1.309",
    pages = "2235--2246"
}

Details

Annotation Types

likert

Domain

NLPSocial MediaSemEval

Use Cases

Intimacy DetectionSocial Media AnalysisMultilingual NLP

Tags

semevalsemeval-2023shared-taskintimacytweetsmultilingualsocial-media

Found an issue or want to improve this design?

Open an Issue

Related Designs

Assessing Humor in Edited News Headlines

Rate the funniness of edited news headlines on a 0-3 scale and judge which of two edited versions is funnier, based on SemEval-2020 Task 7 (Hossain et al.). Headlines are minimally edited by replacing a single word to create humorous effect.

likertradio

Clickbait Detection (Webis Clickbait Corpus)

Classify headlines and social media posts as clickbait or non-clickbait based on the Webis Clickbait Corpus. Identify manipulative content designed to attract clicks through sensationalism, curiosity gaps, or misleading framing.

likertmultiselect

Dynamic Hate Speech Detection

Hate speech classification with fine-grained type labels based on the Dynamically Generated Hate Speech Dataset (Vidgen et al., ACL 2021). Classify content as hateful or not, then identify hate type (animosity, derogation, dehumanization, threatening, support for hateful entities) and target group.

likertmultiselect