Multiplayer Rooms

New in v2.7.0

Multiplayer Rooms turn annotation from a solitary task into a live, shared session. A group of annotators works on the same item at the same time, with the social dynamics measured rather than merely enabled. No LLM, no external services, no new dependencies.

Three room types:

Why norming rooms

Annotation teams already run norming meetings over screen-share, and the results live in someone's notes. Rooms make the session part of the tool and instrument it.

• Blind first. Before the reveal, members see who has voted but never what. This is enforced server-side, so first impressions are genuinely independent.
• The reveal is a measurement. Blind votes are immutable once revealed. Every post-reveal change is logged with what the member switched from, what they switched to, and what the majority was at that moment — a per-member conformity record.
• The α meter answers "was this worth it?" Krippendorff's α is computed twice over revealed items: on blind votes, and on current votes. The gap is the session's norming lift, on screen while it happens.

Configuration

rooms:
  enabled: true
  who_can_create: any      # "any" logged-in user, or "admin" only
  persist_votes: true      # final votes write into members' annotation state
  poll_interval_ms: 1500   # client sync cadence (500–30000)
  max_members: 12          # per room (2–100)
  schema: sarcasm          # optional; defaults to the first radio/likert scheme

Rooms vote on one single-choice scheme (radio or likert). If schema is not set, the first such scheme in annotation_schemes is used.

Running a session

1. Start the server, log in, and open /rooms.
2. Create a room (type and item count). It gets a six-letter code like MKT4QP. Codes avoid ambiguous glyphs so you can read them aloud.
3. Others join from the lobby list, or by opening /rooms/<CODE>.
4. Vote, the host reveals, discuss in the side chat, change votes if convinced, and the host advances. After the last item the room closes with a session summary, and the host can download the full event log.

Huddles

Choosing Huddle at creation seeds the room with every item where annotators currently disagree on the room's schema, up to 200. The original annotations appear above the vote panel as context. Disagreements are computed live from annotation state, so the adjudication subsystem does not need to be enabled.

Shadow sessions

In a shadow room, everyone except the host joins as an observer. Observers cannot vote, but they see the host's votes, reveals, and text selections — the host's current selection is highlighted in amber on the observers' screens — with roughly 1.5 seconds of latency.

Persistence and crash recovery

Every room is event-sourced. Each mutation (join, vote, reveal, change, message, advance, close) appends one JSON line to:

<output_annotation_dir>/rooms/room-<CODE>.jsonl

Room state is a pure replay of that log, so live rooms survive a server restart. The log doubles as the session's audit trail: blind votes, timestamps, discussion, and every post-reveal change with its majority context. GET /rooms/api/<CODE>/export (host or admin) returns the log plus computed metrics as JSON.

With persist_votes: true (the default), each member's final vote on a revealed item is written into their regular annotation state when the host advances. Room work counts as real annotations and flows into every existing export path.

Clients synchronize by polling /events with a cursor. There are no WebSockets, so rooms work under the threaded dev server and any WSGI deployment alike.

API summary

Further reading

• Psychometrics — per-annotator ability and item difficulty; rooms are where you fix the codebook problems it flags
• Adjudication and disagreement — the asynchronous counterpart to huddles
• Writing annotation guidelines
• Source documentation