Skip to content
intermediatetext

Aspect-Based Sentiment Analysis (Original ABSA)

Identify aspect terms in review text and classify their sentiment polarity, based on SemEval-2014 Task 4 (Pontiki et al.). Annotators highlight aspect terms and assign sentiment labels across restaurant and laptop review domains.

About this dataset

SemEval-2014 Task 4, Aspect Based Sentiment Analysis, was organized by Maria Pontiki, Dimitris Galanis, John Pavlopoulos, Harris Papageorgiou, Ion Androutsopoulos, and Suresh Manandhar. ABSA moves past assigning one overall polarity to a sentence and instead identifies the specific aspects of an entity that a review mentions (battery, screen, service, food) and the sentiment expressed toward each. The task attracted 163 submissions from 32 teams.

The organizers released manually annotated reviews in two domains, restaurants and laptops. Four subtasks were defined: aspect term extraction (SB1) and aspect term polarity (SB2) for both domains, plus aspect category detection (SB3) and aspect category polarity (SB4) for restaurants only. Polarity uses four labels: positive, negative, neutral, and conflict, where conflict marks an aspect that carries both positive and negative sentiment. The restaurant training set reuses sentences from Ganu et al. (2009), to which the organizers added aspect-term annotations.

The restaurant data comprises 3,041 training and 800 test sentences. The laptop data comprises 3,045 training and 800 test sentences. Across both domains that is 6,086 training sentences and 1,600 test sentences. Aspect categories and their polarities (SB3, SB4) were annotated only for the restaurant domain.

The Potato config below reproduces this task with a span-highlighting scheme for marking aspect terms in the review text and a radio scheme for sentiment polarity (positive, negative, neutral, conflict). It fits opinion-mining work where annotators locate the evaluated features and label the sentiment. The original SB2 assigns a polarity to each aspect term individually; the config collects one overall polarity per review.

Domains
restaurants, laptops
Restaurant sentences
3,041 train / 800 test
Laptop sentences
3,045 train / 800 test
Total sentences
6,086 train / 1,600 test
Polarity labels
positive, negative, neutral, conflict
Participation
163 submissions from 32 teams
PERORGLOCPERORGLOCDATESelect text to annotate

Configuration Fileconfig.yaml

This Potato config reproduces the annotation task. Save it as config.yaml and run potato start config.yaml to try it.

yaml
# Original ABSA - Aspect-Based Sentiment Analysis
# Based on Pontiki et al., SemEval 2014
# Paper: https://aclanthology.org/S14-2004/
# Dataset: https://alt.qcri.org/semeval2014/task4/
#
# This task requires annotators to identify aspect terms in review text
# and classify the sentiment polarity expressed toward each aspect.
# The task covers two domains: restaurant reviews and laptop reviews.
#
# Aspect terms are specific features or attributes mentioned in the review
# (e.g., "battery life", "screen", "pasta", "service").
#
# Sentiment Labels:
# - Positive: The opinion toward the aspect is favorable
# - Negative: The opinion toward the aspect is unfavorable
# - Neutral: The aspect is mentioned factually without sentiment
# - Conflict: Both positive and negative sentiments are expressed

annotation_task_name: "Original ABSA - Aspect-Based Sentiment Analysis"
task_dir: "."

data_files:
  - sample-data.json

item_properties:
  id_key: "id"
  text_key: "text"

output_annotation_dir: "annotation_output/"
output_annotation_format: "json"

port: 8000
server_name: localhost

annotation_schemes:
  - annotation_type: span
    name: aspect_term_spans
    description: "Highlight all aspect terms (specific features or attributes) mentioned in the review."
    labels:
      - "Aspect Term"
    tooltips:
      "Aspect Term": "A specific feature, attribute, or entity being evaluated (e.g., battery life, pasta, service, screen resolution)"

  - annotation_type: radio
    name: sentiment_polarity
    description: "What is the overall sentiment polarity of this review toward the identified aspects?"
    labels:
      - "Positive"
      - "Negative"
      - "Neutral"
      - "Conflict"
    keyboard_shortcuts:
      "Positive": "1"
      "Negative": "2"
      "Neutral": "3"
      "Conflict": "4"
    tooltips:
      "Positive": "The opinion expressed toward the aspect is clearly favorable or positive"
      "Negative": "The opinion expressed toward the aspect is clearly unfavorable or negative"
      "Neutral": "The aspect is mentioned factually without expressing positive or negative sentiment"
      "Conflict": "Both positive and negative sentiments are expressed toward the aspect"

annotation_instructions: |
  You will be shown review text from the restaurant or laptop domain.

  **Step 1: Aspect Term Identification**
  Highlight all aspect terms in the text. An aspect term is a specific
  feature, attribute, or entity being discussed or evaluated. Examples:
  - Restaurant: food, service, ambiance, price, waiter, pizza, dessert menu
  - Laptop: battery life, screen, keyboard, trackpad, processor, fan noise

  **Step 2: Sentiment Classification**
  Classify the overall sentiment polarity:
  - **Positive**: Favorable opinion (e.g., "excellent sushi", "fast boot time")
  - **Negative**: Unfavorable opinion (e.g., "slow service", "cheap trackpad")
  - **Neutral**: Factual mention without sentiment (e.g., "the menu has 10 items")
  - **Conflict**: Mixed sentiments (e.g., "the food is great but overpriced")

html_layout: |
  <div style="padding: 15px; max-width: 800px; margin: auto;">
    <div style="background: #eff6ff; border: 1px solid #93c5fd; border-radius: 8px; padding: 16px; margin-bottom: 16px;">
      <strong style="color: #1e40af;">Review Text:</strong>
      <p style="font-size: 16px; line-height: 1.7; margin: 8px 0 0 0;">{{text}}</p>
    </div>
    <div style="background: #f5f3ff; border: 1px solid #c4b5fd; border-radius: 8px; padding: 12px; margin-bottom: 16px;">
      <strong style="color: #5b21b6;">Domain:</strong>
      <span style="font-size: 15px; margin-left: 8px; text-transform: capitalize;">{{domain}}</span>
    </div>
  </div>

allow_all_users: true
instances_per_annotator: 50
annotation_per_instance: 2
allow_skip: true
skip_reason_required: false

Sample Datasample-data.json

json
[
  {
    "id": "absa_001",
    "text": "The sushi was excellent but the service was incredibly slow, and we had to wait over 30 minutes for our appetizers to arrive.",
    "domain": "restaurant"
  },
  {
    "id": "absa_002",
    "text": "The battery life on this laptop is outstanding, easily lasting 10 hours on a single charge, but the trackpad feels cheap and unresponsive.",
    "domain": "laptop"
  }
]

// ... and 8 more items

Get This Design

View on GitHub

Clone or download from the repository

Quick start:

git clone https://github.com/davidjurgens/potato-showcase.git
cd potato-showcase/semeval/2014/task04-absa-original
potato start config.yaml

Dataset & paper

Pontiki et al., SemEval 2014

Citation (BibTeX)

bibtex
@inproceedings{pontiki-etal-2014-semeval,
    title = "{S}em{E}val-2014 {T}ask 4: {A}spect {B}ased {S}entiment {A}nalysis",
    author = "Pontiki, Maria and Galanis, Dimitris and Pavlopoulos, John and Papageorgiou, Harris and Androutsopoulos, Ion and Manandhar, Suresh",
    booktitle = "Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014)",
    month = aug,
    year = "2014",
    address = "Dublin, Ireland",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/S14-2004/",
    doi = "10.3115/v1/S14-2004",
    pages = "27--35"
}

Details

Annotation Types

spanradio

Domain

SemEvalSentiment AnalysisAspect-Based

Use Cases

Aspect ExtractionSentiment ClassificationOpinion Mining

Tags

semevalsemeval-2014shared-taskabsaaspect-basedsentimentopinion-miningreviews

Found an issue or want to improve this design?

Open an Issue