Skip to content
Showcase/Semantic Relation Extraction and Classification in Scientific Papers
advancedtext

Semantic Relation Extraction and Classification in Scientific Papers

Extraction and classification of semantic relations between entities in scientific abstracts, covering six relation types. Based on SemEval-2018 Task 7.

PERORGLOCPERORGLOCDATESelect text to annotate

Configuration Fileconfig.yaml

# Semantic Relation Extraction and Classification in Scientific Papers
# Based on Gabor et al., SemEval 2018
# Paper: https://aclanthology.org/S18-1027/
# Dataset: https://lipn.univ-paris13.fr/~gabor/semeval2018task7/
#
# This task asks annotators to identify entity pairs in scientific text
# and classify the semantic relation between them.
#
# Entity Span Labels:
# - Entity 1: The first entity in the relation
# - Entity 2: The second entity in the relation
#
# Relation Types:
# - Usage: Entity 1 is used by/for Entity 2
# - Result: Entity 1 produces/yields Entity 2
# - Model-Feature: Entity 1 is a feature/property of model Entity 2
# - Part-Whole: Entity 1 is a component of Entity 2
# - Topic: Entity 1 is about/relates to topic Entity 2
# - Comparison: Entity 1 is compared with Entity 2

annotation_task_name: "Semantic Relation Extraction in Scientific Papers"
task_dir: "."

data_files:
  - sample-data.json

item_properties:
  id_key: "id"
  text_key: "text"

output_annotation_dir: "annotation_output/"
output_annotation_format: "json"

port: 8000
server_name: localhost

annotation_schemes:
  - annotation_type: span
    name: entity_spans
    description: "Highlight the two entities involved in the semantic relation."
    labels:
      - "Entity 1"
      - "Entity 2"

  - annotation_type: radio
    name: relation_type
    description: "What is the semantic relation between Entity 1 and Entity 2?"
    labels:
      - "Usage"
      - "Result"
      - "Model-Feature"
      - "Part-Whole"
      - "Topic"
      - "Comparison"
    keyboard_shortcuts:
      "Usage": "1"
      "Result": "2"
      "Model-Feature": "3"
      "Part-Whole": "4"
      "Topic": "5"
      "Comparison": "6"
    tooltips:
      "Usage": "Entity 1 is used by, for, or in Entity 2"
      "Result": "Entity 1 produces, yields, or results in Entity 2"
      "Model-Feature": "Entity 1 is a feature or property of model/system Entity 2"
      "Part-Whole": "Entity 1 is a component or part of Entity 2"
      "Topic": "Entity 1 is about or related to the topic Entity 2"
      "Comparison": "Entity 1 is compared or contrasted with Entity 2"

annotation_instructions: |
  You will be shown a sentence from a scientific abstract. Your task is to:
  1. Highlight the two key entities that have a semantic relation.
  2. Classify the relation between them into one of six categories.
  Consider the domain context when determining the relation type.

html_layout: |
  <div style="padding: 15px; max-width: 800px; margin: auto;">
    <div style="background: #fefce8; border: 1px solid #fde68a; border-radius: 8px; padding: 12px; margin-bottom: 12px;">
      <strong style="color: #a16207;">Domain:</strong>
      <span style="font-size: 15px;">{{domain}}</span>
    </div>
    <div style="background: #f0f9ff; border: 1px solid #bae6fd; border-radius: 8px; padding: 16px; margin-bottom: 16px;">
      <strong style="color: #0369a1;">Scientific Text:</strong>
      <p style="font-size: 16px; line-height: 1.7; margin: 8px 0 0 0;">{{text}}</p>
    </div>
  </div>

allow_all_users: true
instances_per_annotator: 50
annotation_per_instance: 2
allow_skip: true
skip_reason_required: false

Sample Datasample-data.json

[
  {
    "id": "scirel_001",
    "text": "We use a convolutional neural network for text classification, achieving state-of-the-art results on sentiment analysis benchmarks.",
    "domain": "Natural Language Processing"
  },
  {
    "id": "scirel_002",
    "text": "The attention mechanism is a key component of the transformer architecture, enabling the model to focus on relevant parts of the input.",
    "domain": "Deep Learning"
  }
]

// ... and 8 more items

Get This Design

View on GitHub

Clone or download from the repository

Quick start:

git clone https://github.com/davidjurgens/potato-showcase.git
cd potato-showcase/semeval/2018/task07-scientific-relations
potato start config.yaml

Details

Annotation Types

spanradio

Domain

SemEvalNLPRelation ExtractionScientific Text

Use Cases

Relation ExtractionScientific Text MiningKnowledge Extraction

Tags

semevalsemeval-2018shared-taskrelation-extractionscientific-textknowledge-extraction

Found an issue or want to improve this design?

Open an Issue

Related Designs

Aspect-Based Sentiment Analysis

Identification of aspect terms in review text with sentiment polarity classification for each aspect. Based on SemEval-2016 Task 5 (ABSA).

spanradio

Causal Medical Claim Detection and PICO Extraction

Detection of causal claims in medical texts and extraction of PICO (Population, Intervention, Comparator, Outcome) elements. Based on SemEval-2023 Task 8 (Khetan et al.).

spanradio

Character Identification on Multiparty Dialogues

Identification and linking of character mentions in TV show dialogue, combining span annotation with entity resolution for the main cast of Friends. Based on SemEval-2018 Task 4.

spanradio