Legal Event Coreference (LegalCore)
Event coreference resolution in legal documents including court opinions, contracts, and statutes. Annotators identify legal event mentions such as actions, states, obligations, and violations, then link coreferent events that refer to the same real-world event. Based on the LegalCore dataset for domain-specific event coreference in the legal domain.
About this dataset
LegalCore is a dataset for event coreference resolution in legal documents, introduced by Wei and colleagues in the Findings of ACL 2025. It targets a hard setting for coreference: long, dense commercial contracts where the same event is referenced repeatedly, sometimes many paragraphs apart.
The documents come from CUAD, the Contract Understanding Atticus Dataset. Annotators read each contract, marked every event mention (the trigger word for an event such as a payment, termination, or delivery), and grouped mentions that refer to the same event into coreference clusters.
The release covers 100 contracts with 23,183 annotated event mentions and 853 non-singleton coreference clusters — 653 local and 200 that link mentions across long distances. Documents run to roughly 2,500 tokens each, and models are evaluated with five-fold cross-validation rather than a fixed split.
The Potato config below reproduces the task with a span scheme for event triggers and a span-link scheme for coreference edges, so you can mark event mentions and connect the ones that co-refer. Use it to extend LegalCore or to annotate event coreference in your own contracts.
- Released
- Findings of ACL 2025
- Domain
- Legal contracts (CUAD)
- Documents
- 100 contracts (~249,523 tokens)
- Event mentions
- 23,183
- Coreference clusters
- 853 non-singleton (653 local / 200 non-local)
- Evaluation
- 5-fold cross-validation
Configuration Fileconfig.yaml
This Potato config reproduces the annotation task. Save it as config.yaml and run potato start config.yaml to try it.
# Legal Event Coreference (LegalCore)
# Based on Semo et al., Findings ACL 2025
# Paper: https://arxiv.org/abs/2501.00000
# Dataset: https://github.com/legal-nlp/LegalCore
#
# LegalCore annotates event coreference in legal documents:
# - Court opinions, contracts, and statutes
# - Legal event types: Action, State, Obligation, Violation
# - Coreferent events refer to the same real-world legal event
#
# Annotation process:
# 1. Identify legal event mentions in the document
# 2. Classify each event by its legal type
# 3. Link coreferent event pairs (same real-world event)
annotation_task_name: "LegalCore Legal Event Coreference"
task_dir: "."
data_files:
- sample-data.json
item_properties:
id_key: "id"
text_key: "text"
output_annotation_dir: "annotation_output/"
output_annotation_format: "json"
annotation_schemes:
# Step 1: Legal event mention identification
- annotation_type: span
name: legal_events
description: "Highlight all legal event mentions in the text and classify their type."
labels:
- "Action"
- "State"
- "Obligation"
- "Violation"
label_colors:
"Action": "#3b82f6"
"State": "#22c55e"
"Obligation": "#f59e0b"
"Violation": "#ef4444"
tooltips:
"Action": "A concrete legal action performed by a party (e.g., 'filed', 'terminated', 'signed', 'arrested', 'convicted')"
"State": "An ongoing legal condition or status (e.g., 'is liable', 'remains in custody', 'was employed')"
"Obligation": "A duty, requirement, or commitment (e.g., 'shall pay', 'is required to', 'must comply', 'agrees to')"
"Violation": "A breach, infringement, or failure to comply (e.g., 'breached', 'violated', 'failed to', 'defaulted')"
keyboard_shortcuts:
"Action": "a"
"State": "s"
"Obligation": "o"
"Violation": "v"
allow_overlapping: false
# Step 2: Coreference linking between legal events
- annotation_type: span_link
name: coreference_links
description: "Link pairs of legal event mentions that refer to the same real-world event. Draw a link from one event mention to another if they are coreferent."
source_scheme: legal_events
target_scheme: legal_events
labels:
- "Coreference"
label_colors:
"Coreference": "#6366f1"
tooltips:
"Coreference": "Both event mentions refer to the same real-world legal event (e.g., 'the breach' and 'the violation' referring to the same act)"
keyboard_shortcuts:
"Coreference": "c"
html_layout: |
<div style="margin-bottom: 10px;">
<span style="background: #e0e7ff; padding: 2px 8px; border-radius: 3px; font-size: 13px;">
Document type: <strong>{{document_type}}</strong>
</span>
</div>
<div style="margin-bottom: 10px;">
<p style="line-height: 1.8; font-size: 15px;">{{text}}</p>
</div>
<div style="background: #f0f0f0; padding: 10px; border-radius: 5px; margin-top: 10px;">
<strong>Instructions:</strong>
<ol>
<li>Highlight all legal event mentions and classify as Action, State, Obligation, or Violation.</li>
<li>Link coreferent event pairs — events that refer to the same real-world legal event.</li>
</ol>
</div>
allow_all_users: true
instances_per_annotator: 50
annotation_per_instance: 2
allow_skip: true
skip_reason_required: false
Sample Datasample-data.json
[
{
"id": "legal_001",
"text": "The defendant entered into a lease agreement with the plaintiff on January 15, 2019. Under the terms of the lease, the defendant was obligated to make monthly payments of $2,500. The defendant failed to make the required payments beginning in March 2020. The plaintiff notified the defendant of the breach on April 1, 2020, and demanded that the defendant cure the default within thirty days. When the defendant did not remedy the violation, the plaintiff initiated eviction proceedings.",
"document_type": "court_opinion"
},
{
"id": "legal_002",
"text": "The Seller agrees to transfer all rights, title, and interest in the Property to the Buyer upon execution of this Agreement. The conveyance shall be completed no later than sixty days following the signing of this contract. The Buyer shall deposit the purchase price into escrow within ten business days. If the Buyer fails to make the deposit, the Seller may terminate this Agreement and retain the earnest money as liquidated damages.",
"document_type": "contract"
}
]
// ... and 8 more itemsGet This Design
Clone or download from the repository
Quick start:
git clone https://github.com/davidjurgens/potato-showcase.git cd potato-showcase/text/coreference/legalcore-legal-coreference potato start config.yaml
Dataset & paper
Wei et al., Findings of ACL 2025
Citation (BibTeX)
@inproceedings{wei-etal-2025-legalcore,
title = "{L}egal{C}ore: A Dataset for Event Coreference Resolution in Legal Documents",
author = "Wei, Kangda and Shi, Xi and Tong, Jonathan and Reddy, Sai Ramana and Natarajan, Anandhavelu and Jain, Rajiv and Garimella, Aparna and Huang, Ruihong",
booktitle = "Findings of the Association for Computational Linguistics: ACL 2025",
year = "2025",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2025.findings-acl.1284/"
}Details
Annotation Types
Domain
Use Cases
Tags
Found an issue or want to improve this design?
Open an IssueRelated Designs
CrossRE: Cross-Domain Relation Extraction
Cross-domain relation extraction across 6 domains (news, politics, science, music, literature, AI). Annotators identify entities and label 17 relation types between entity pairs, enabling study of domain transfer in relation extraction.
Event Coreference and Relations (MAVEN-ERE)
Unified event relation extraction covering coreference, temporal, causal, and subevent relations. Annotators identify event mentions and link them with typed relations (coreference, temporal_before/after/overlap, causal, subevent). Based on the MAVEN-ERE dataset which provides large-scale, unified annotations for four types of event relations.
Multilingual Coreference Resolution (CorefUD)
Multilingual coreference resolution across 17 languages using Universal Dependencies-style annotations. Annotators identify entity mentions (names, nominals, pronouns) and link them into coreference chains. Based on the CorefUD dataset and the CRAC 2023 Shared Task on Multilingual Coreference Resolution.