Skip to content
Showcase/Analysis of Clinical Text: Disorder Identification and Normalization
advancedtext

Analysis of Clinical Text: Disorder Identification and Normalization

Identify disorder mentions and their attributes in clinical discharge summaries, based on SemEval-2015 Task 14 (Elhadad et al.). Annotators mark disorder spans, body locations, severity indicators, and classify the assertion status of each disorder.

PERORGLOCPERORGLOCDATESelect text to annotate

Configuration Fileconfig.yaml

This Potato config reproduces the annotation task. Save it as config.yaml and run potato start config.yaml to try it.

yaml
# Clinical Text Analysis - Disorder Identification
# Based on Elhadad et al., SemEval 2015
# Paper: https://aclanthology.org/S15-2070/
# Dataset: https://www.i2b2.org/NLP/DataSets/
#
# This task requires annotators to identify disorder mentions and their
# attributes in de-identified clinical discharge summaries and progress
# notes. Annotators mark disorder spans, body locations, severity
# indicators, disease course markers, and negation cues, then classify
# the assertion status of each disorder.
#
# Span Types:
# - Disorder Mention: A disease, condition, or clinical finding
# - Body Location: An anatomical site associated with a disorder
# - Severity: An indicator of disorder severity (e.g., severe, mild, moderate)
# - Course: A marker of disease progression (e.g., progressive, worsening, stable)
# - Negation Indicator: A cue that negates a disorder (e.g., denies, no, negative for)
#
# Assertion Types:
# - Present: The disorder is currently present in the patient
# - Absent: The disorder has been ruled out or is not present
# - Possible: The disorder is suspected but not confirmed
# - Conditional: The disorder occurs under certain conditions
# - Hypothetical: The disorder is discussed as a future possibility
# - Associated with Someone Else: The disorder is attributed to someone other than the patient

annotation_task_name: "Clinical Text Analysis - Disorder Identification"
task_dir: "."

data_files:
  - sample-data.json

item_properties:
  id_key: "id"
  text_key: "text"

output_annotation_dir: "annotation_output/"
output_annotation_format: "json"

port: 8000
server_name: localhost

annotation_schemes:
  - annotation_type: span
    name: clinical_entity_spans
    description: "Highlight disorder mentions and their associated attributes in the clinical text."
    labels:
      - "Disorder Mention"
      - "Body Location"
      - "Severity"
      - "Course"
      - "Negation Indicator"
    tooltips:
      "Disorder Mention": "A disease, condition, symptom, or clinical finding (e.g., congestive heart failure, pneumonia, diabetes mellitus)"
      "Body Location": "An anatomical site or body part associated with a disorder (e.g., right knee, lower extremity, chest)"
      "Severity": "An indicator of how severe the disorder is (e.g., severe, mild, moderate, significant)"
      "Course": "A marker describing disease progression or trajectory (e.g., progressive, worsening, stable, acute)"
      "Negation Indicator": "A word or phrase that negates the presence of a disorder (e.g., denies, no, negative for, ruled out)"

  - annotation_type: radio
    name: assertion_status
    description: "What is the assertion status of the primary disorder mentioned in this text?"
    labels:
      - "Present"
      - "Absent"
      - "Possible"
      - "Conditional"
      - "Hypothetical"
      - "Associated with Someone Else"
    keyboard_shortcuts:
      "Present": "1"
      "Absent": "2"
      "Possible": "3"
      "Conditional": "4"
      "Hypothetical": "5"
      "Associated with Someone Else": "6"
    tooltips:
      "Present": "The disorder is currently present and confirmed in the patient"
      "Absent": "The disorder has been ruled out or is explicitly stated as not present"
      "Possible": "The disorder is suspected or being considered but not yet confirmed"
      "Conditional": "The disorder is present only under specific conditions or circumstances"
      "Hypothetical": "The disorder is discussed as a future possibility or preventive concern"
      "Associated with Someone Else": "The disorder is mentioned in reference to someone other than the patient (e.g., family history)"

annotation_instructions: |
  You will be shown de-identified clinical text from discharge summaries
  or progress notes.

  **Step 1: Entity Annotation**
  Highlight and classify the following spans in the text:
  - **Disorder Mention**: Any disease, condition, symptom, or finding
    (e.g., "congestive heart failure", "pneumonia", "edema")
  - **Body Location**: Anatomical sites associated with disorders
    (e.g., "right knee", "lower lobe", "bilateral lower extremity")
  - **Severity**: Severity indicators (e.g., "severe", "mild", "moderate")
  - **Course**: Disease progression markers (e.g., "progressive", "worsening", "acute")
  - **Negation Indicator**: Words negating a disorder (e.g., "denies", "no", "negative for")

  **Step 2: Assertion Classification**
  Classify the assertion status of the primary disorder:
  - **Present**: Disorder is confirmed in the patient
  - **Absent**: Disorder is ruled out or explicitly absent
  - **Possible**: Disorder is suspected but unconfirmed
  - **Conditional**: Disorder present only under certain conditions
  - **Hypothetical**: Discussed as a future possibility
  - **Associated with Someone Else**: Attributed to another person

html_layout: |
  <div style="padding: 15px; max-width: 800px; margin: auto;">
    <div style="background: #fef2f2; border: 1px solid #fca5a5; border-radius: 8px; padding: 12px; margin-bottom: 16px;">
      <strong style="color: #991b1b;">Note Type:</strong>
      <span style="font-size: 15px; margin-left: 8px; text-transform: capitalize;">{{note_type}}</span>
    </div>
    <div style="background: #f0f9ff; border: 1px solid #bae6fd; border-radius: 8px; padding: 16px; margin-bottom: 16px;">
      <strong style="color: #0369a1;">Clinical Text:</strong>
      <p style="font-size: 16px; line-height: 1.7; margin: 8px 0 0 0;">{{text}}</p>
    </div>
  </div>

allow_all_users: true
instances_per_annotator: 50
annotation_per_instance: 2
allow_skip: true
skip_reason_required: false

Sample Datasample-data.json

json
[
  {
    "id": "clinical_001",
    "text": "DISCHARGE SUMMARY: The patient is a 67-year-old male admitted with acute exacerbation of congestive heart failure. He presented with progressive dyspnea on exertion and bilateral lower extremity edema over the past two weeks. Chest X-ray revealed bilateral pleural effusions. He was treated with IV furosemide with significant improvement in symptoms.",
    "note_type": "discharge_summary"
  },
  {
    "id": "clinical_002",
    "text": "HOSPITAL COURSE: Patient is a 54-year-old female with a history of type 2 diabetes mellitus and chronic kidney disease stage III. She was admitted for evaluation of worsening renal function. Creatinine on admission was 3.2 mg/dL, up from baseline of 1.8 mg/dL. Renal ultrasound showed no hydronephrosis. Nephrology was consulted.",
    "note_type": "discharge_summary"
  }
]

// ... and 8 more items

Get This Design

View on GitHub

Clone or download from the repository

Quick start:

git clone https://github.com/davidjurgens/potato-showcase.git
cd potato-showcase/semeval/2015/task14-clinical-text-analysis
potato start config.yaml

Dataset & paper

Elhadad et al., SemEval 2015

Citation (BibTeX)

bibtex
@inproceedings{elhadad-etal-2015-semeval,
    title = "{S}em{E}val-2015 Task 14: Analysis of Clinical Text",
    author = "Elhadad, No{\'e}mie and Pradhan, Sameer and Gorman, Sharon and Manandhar, Suresh and Chapman, Wendy and Savova, Guergana",
    booktitle = "Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015)",
    month = jun,
    year = "2015",
    address = "Denver, Colorado",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/S15-2051/",
    doi = "10.18653/v1/S15-2051",
    pages = "303--310"
}

Details

Annotation Types

spanradio

Domain

SemEvalClinical NLPBiomedicalNER

Use Cases

Disorder IdentificationClinical Text MiningMedical NER

Tags

semevalsemeval-2015shared-taskclinicalbiomedicaldisordernernormalization

Found an issue or want to improve this design?

Open an Issue