advancedtext
Semantic Relation Extraction and Classification in Scientific Papers
Extraction and classification of semantic relations between entities in scientific abstracts, covering six relation types. Based on SemEval-2018 Task 7.
Configuration Fileconfig.yaml
This Potato config reproduces the annotation task. Save it as config.yaml and run potato start config.yaml to try it.
yaml
# Semantic Relation Extraction and Classification in Scientific Papers
# Based on Gabor et al., SemEval 2018
# Paper: https://aclanthology.org/S18-1111/
# Dataset: https://github.com/gkata/SemEval2018Task7
#
# This task asks annotators to identify entity pairs in scientific text
# and classify the semantic relation between them.
#
# Entity Span Labels:
# - Entity 1: The first entity in the relation
# - Entity 2: The second entity in the relation
#
# Relation Types:
# - Usage: Entity 1 is used by/for Entity 2
# - Result: Entity 1 produces/yields Entity 2
# - Model-Feature: Entity 1 is a feature/property of model Entity 2
# - Part-Whole: Entity 1 is a component of Entity 2
# - Topic: Entity 1 is about/relates to topic Entity 2
# - Comparison: Entity 1 is compared with Entity 2
annotation_task_name: "Semantic Relation Extraction in Scientific Papers"
task_dir: "."
data_files:
- sample-data.json
item_properties:
id_key: "id"
text_key: "text"
output_annotation_dir: "annotation_output/"
output_annotation_format: "json"
port: 8000
server_name: localhost
annotation_schemes:
- annotation_type: span
name: entity_spans
description: "Highlight the two entities involved in the semantic relation."
labels:
- "Entity 1"
- "Entity 2"
- annotation_type: radio
name: relation_type
description: "What is the semantic relation between Entity 1 and Entity 2?"
labels:
- "Usage"
- "Result"
- "Model-Feature"
- "Part-Whole"
- "Topic"
- "Comparison"
keyboard_shortcuts:
"Usage": "1"
"Result": "2"
"Model-Feature": "3"
"Part-Whole": "4"
"Topic": "5"
"Comparison": "6"
tooltips:
"Usage": "Entity 1 is used by, for, or in Entity 2"
"Result": "Entity 1 produces, yields, or results in Entity 2"
"Model-Feature": "Entity 1 is a feature or property of model/system Entity 2"
"Part-Whole": "Entity 1 is a component or part of Entity 2"
"Topic": "Entity 1 is about or related to the topic Entity 2"
"Comparison": "Entity 1 is compared or contrasted with Entity 2"
annotation_instructions: |
You will be shown a sentence from a scientific abstract. Your task is to:
1. Highlight the two key entities that have a semantic relation.
2. Classify the relation between them into one of six categories.
Consider the domain context when determining the relation type.
html_layout: |
<div style="padding: 15px; max-width: 800px; margin: auto;">
<div style="background: #fefce8; border: 1px solid #fde68a; border-radius: 8px; padding: 12px; margin-bottom: 12px;">
<strong style="color: #a16207;">Domain:</strong>
<span style="font-size: 15px;">{{domain}}</span>
</div>
<div style="background: #f0f9ff; border: 1px solid #bae6fd; border-radius: 8px; padding: 16px; margin-bottom: 16px;">
<strong style="color: #0369a1;">Scientific Text:</strong>
<p style="font-size: 16px; line-height: 1.7; margin: 8px 0 0 0;">{{text}}</p>
</div>
</div>
allow_all_users: true
instances_per_annotator: 50
annotation_per_instance: 2
allow_skip: true
skip_reason_required: false
Sample Datasample-data.json
json
[
{
"id": "scirel_001",
"text": "We use a convolutional neural network for text classification, achieving state-of-the-art results on sentiment analysis benchmarks.",
"domain": "Natural Language Processing"
},
{
"id": "scirel_002",
"text": "The attention mechanism is a key component of the transformer architecture, enabling the model to focus on relevant parts of the input.",
"domain": "Deep Learning"
}
]
// ... and 8 more itemsGet This Design
View on GitHub
Clone or download from the repository
Quick start:
git clone https://github.com/davidjurgens/potato-showcase.git cd potato-showcase/semeval/2018/task07-scientific-relations potato start config.yaml
Dataset & paper
Gabor et al., SemEval 2018
Citation (BibTeX)
bibtex
@inproceedings{gabor-etal-2018-semeval,
title = "{S}em{E}val-2018 Task 7: Semantic Relation Extraction and Classification in Scientific Papers",
author = "G{\'a}bor, Kata and Buscaldi, Davide and Schumann, Anne-Kathrin and QasemiZadeh, Behrang and Zargayouna, Ha{\"\i}fa and Charnois, Thierry",
booktitle = "Proceedings of the 12th International Workshop on Semantic Evaluation",
month = jun,
year = "2018",
address = "New Orleans, Louisiana",
publisher = "Association for Computational Linguistics",
pages = "679--688",
doi = "10.18653/v1/S18-1111",
url = "https://aclanthology.org/S18-1111"
}Details
Annotation Types
spanradio
Domain
SemEvalNLPRelation ExtractionScientific Text
Use Cases
Relation ExtractionScientific Text MiningKnowledge Extraction
Tags
semevalsemeval-2018shared-taskrelation-extractionscientific-textknowledge-extraction
Found an issue or want to improve this design?
Open an IssueRelated Designs
Symlink: Linking Mathematical Symbols to their Descriptions
SemEval-2022 Task 12 (Symlink): extracting mathematical symbols and their textual descriptions from scientific documents and linking each symbol to the description it refers to. Annotators mark symbol and description spans, then link them with relation types (Direct, Count, Coref-Symbol, Coref-Description). Documents are drawn from arXiv papers across mathematics, physics, biology, economics, and computer science.
spanradio
Aspect-Based Sentiment Analysis
Identification of aspect terms in review text with sentiment polarity classification for each aspect. Based on SemEval-2016 Task 5 (ABSA).
spanradio
Character Identification on Multiparty Dialogues
Identification and linking of character mentions in TV show dialogue, combining span annotation with entity resolution for the main cast of Friends. Based on SemEval-2018 Task 4.
spanradio