Analytic Memos for Qualitative Coding

Memos in Potato are free-text notes attached to an item, to a text selection inside an item, or to a shape drawn on an image. Any user who can read a memo can reply to the memo, and a memo raised as an issue stays open until the author, the assignee, or an adjudicator resolves the issue. Memos work in any Potato project, with or without QDA Mode, and Potato keeps memos in the project database rather than in the annotation files. Memos arrived in Potato 2.6.0, and replies and issues arrived in 2.12.0.

An analytic memo is a note a researcher writes during coding about the data and the codes, e.g., why a code fits a passage or where the codebook needs a new rule. Memo writing is most often described as part of grounded theory, and Birks, Chapman, and Francis (2008) recommend memo writing across qualitative methods. In the authors' account, memos help the analyst move from raw data to the abstractions that explain the data. In a coding team, memos also record the decisions behind the codes, and the export below turns those notes into an audit trail.

Each memo records the following:

• Anchor. A memo sits on the whole item, on a character range of the item's text (shown with a quote badge), or on one drawn shape (shown with the shape's label and type, e.g., car box).
• Visibility. A memo is private (the default) or shared.
• Thread. Replies sit under the first note and take the first note's visibility.
• Issue status. An issue is open or resolved, can have one assignee, and records who resolved the issue and when.
• Storage. A memo is saved as soon as you click Add note, in the memos and memo_threads tables of <task_dir>/project.sqlite.

Turning memos on

Memos are off in a standard project and on by default under QDA Mode and Solo Mode. A project turns memos on with the annotation_ui block. The config below sets up a team coding interview excerpts with the span scheme and private memos. The adjudication block names a lead analyst, and an adjudicator can read every memo in the project:

annotation_task_name: Clinic access interviews
task_dir: .
output_annotation_dir: annotation_output/
data_files:
- data/excerpts.json
item_properties:
  id_key: id
  text_key: text
 
annotation_ui:
  memos: true
  visibility: private
 
adjudication:
  enabled: true
  adjudicator_users: [lead_analyst]
 
annotation_schemes:
- annotation_type: span
  name: codes
  description: Highlight a passage and apply a code
  labels: [transport barriers, cost concerns, provider trust]

The span scheme matters for export because the quotation report needs a span scheme (see Exporting memos). Four keys control memos:

Outside QDA Mode the Notes panel starts closed, and annotators open the panel with the Notes button on the right edge of the page. Closing the panel keeps the panel closed on later items for the rest of the browser tab's session. With memos off, Potato renders no Notes button and the /api/memos endpoints return 503. The composer's visibility menu starts on Private in every project, and annotation_ui.visibility does not change the menu. A team that wants shared memos asks coders to pick Shared for each new memo.

Potato files each memo under the project's annotation_task_name. Renaming the task in the config hides the memos written under the old name. The rows stay in project.sqlite, and restoring the old name brings the memos back.

Writing a memo in the Notes panel

The Notes panel lists the memos on the current item, oldest first, with each thread's replies under the first note. To add a memo:

1. Click Notes on the right edge of the annotation page.
2. To anchor the memo to a passage, select text in the item first and tick Attach to selected text. To anchor the memo to a drawn shape, select the shape on the image canvas and tick Attach to the selected shape.
3. Type the note in the composer.
4. To raise the note as an issue, tick Raise as an issue to resolve. Optionally, enter the username of the person who should resolve the issue under Assign to.
5. Choose Private or Shared under Visibility and click Add note.

Each memo shows the buttons the current user may use. Reply appears on every thread, Resolve or Reopen and Assign appear on issues the user may change, and Edit and Delete appear on the user's own memos. An adjudicator also sees Delete on other users' memos. Editing a first note can change the note's visibility as well as the text.

A text anchor records the start and end character offsets of the selection within the item's displayed text, with the end offset exclusive. Selections count only inside the item's main text display. A shape anchor records the shape's scheme, type, label, and normalized coordinates because drawn shapes have no stored ID. Clicking the shape's badge on the memo selects the shape again by matching the stored coordinates. If the shape has since been moved or deleted, the panel reports that the shape is no longer on the image as drawn.

Who can read a memo

A private memo is readable by the author and by adjudicators. A shared memo is readable by every user on the project. The assignee of an issue can read and reply to the issue's thread even when the issue is private. The remaining rules for memos are:

• Only the author edits a memo's text or visibility.
• The author or an adjudicator deletes a memo. Deleting a thread's first note deletes the replies.
• Anyone who can read a thread can reply. A reply follows the first note's visibility, and a later change to the first note's visibility applies to the replies too.
• For a memo a user cannot read, every action returns 404. Potato gives the same response for a memo that does not exist.

For memos, an adjudicator is a username listed in adjudication.adjudicator_users while adjudication.enabled is true. Potato checks only that list for memos. A user who holds the RBAC admin role but is not on the list reads the same memos as any annotator.

Private memos are private only inside the annotation interface. Anyone who runs the quotation report export or opens project.sqlite reads every memo, private ones included. Tell coders about the export before coding starts.

Shared memos let coders read each other's reasoning while coding. When two coders code the same excerpts independently for an intercoder reliability check, shared memos can anchor the second coder's judgments and raise agreement for reasons that have nothing to do with the codebook. Keep memos private during the reliability round. After the round, a coder can open each private memo with Edit and switch the memo to Shared for the reconciliation meeting.

Replies and issues

Every memo starts a thread. A reply has no visibility or status separate from the first note. Ticking Raise as an issue to resolve gives the first note the status open. Issues suit questions that need an answer before coding moves on, e.g., a code boundary the codebook does not settle. The permissions for issues are:

Resolving an issue records who resolved the issue and when, and reopening the issue clears both fields. Clearing an assignment removes the assignee's access to a private issue. When open issues are assigned to an annotator, the Notes button shows the count, e.g., Notes · 2 assigned to you, and the panel lists the issues at the top. An issue on an item in the annotator's own queue gets a Go to item button, and an issue on an item outside the queue is labeled as not in the queue. Setting jumping_to_id_disabled: true removes the Go to item buttons.

To send assignments to a chat channel or a tracker, subscribe a webhook to the memo.issue_assigned event. The event fires whenever an issue gets a new assignee and sends the memo ID, the item ID, the assignee, the user who assigned the issue, and the status. The payload leaves out the memo's text because an issue can be private.

Issues in the admin dashboard

When memos are on, the Overview tab of the admin dashboard shows an Open issues tile with the number of open issues and the number resolved. The admin API lists the issues themselves, private ones included:

curl -H "X-API-Key: $POTATO_ADMIN_KEY" \
  "http://localhost:8000/admin/api/memo_issues?status=open"

The response has one entry per issue, newest first, with the memo's text, author, item ID, anchor, assignee, status, and resolver. Use ?status=resolved for resolved issues, or leave the parameter out for both. The endpoint lists issues only. Plain memos reach a project lead through the export or an adjudicator account.

Exporting memos

The quotation_report exporter writes one CSV row per coded span. With include_memos=true, the exporter adds one row per memo and per reply:

python -m potato.export -c config.yaml -f quotation_report \
  --option include_memos=true -o ./export/

The command writes ./export/quotations.csv. Memo rows use the same nine columns as span rows:

The exporter runs only when the config has at least one span scheme. A study coded only with radio or multiselect questions gets the error No span annotation schema found in config. A memo row also has no timestamp and no assignee, and a shape-anchored memo exports with empty offsets.

For an audit trail with dates, query the two memo tables directly. The sqlite3 command below lists every memo in the project with the creation time in UTC, the thread status, the assignee, and the resolver:

sqlite3 -header -csv project.sqlite "
  SELECT m.instance_id, m.created_by, datetime(m.created_at, 'unixepoch') AS created,
         m.visibility, t.parent_id, t.status, t.assignee, t.resolved_by, m.body
  FROM memos m LEFT JOIN memo_threads t ON t.memo_id = m.id
  WHERE m.project = 'Clinic access interviews'
  ORDER BY m.created_at" > memos.csv

A non-empty parent_id marks a reply and names the thread's first note. Potato stores times as Unix seconds, and resolved_at and updated_at convert the same way as created_at.

Memos do not go into the REFI-QDA .qdpx export. When the config sets annotation_ui.memos: true, the .qdpx exporter warns that the memos were left out. Under QDA Mode or Solo Mode without that key, the export leaves the memos out with no warning. When a project moves to another QDA program, export the memos separately with the quotation report.

The archive that potato snapshot create builds includes project.sqlite, and the archive from potato deploy pull includes the file as well. Both archives therefore hold every memo, private ones included. The snapshot's content hash covers the config, the study's files, and the annotations. The hash leaves the databases out, and a new memo does not change the hash. See Project Snapshots.

Memo API

The Notes panel uses a REST API under /api/memos, authenticated by the annotator's login session. Scripts can call the same endpoints:

A text anchor is {"start": 48, "end": 98, "field": "text"}, and a shape anchor is {"shape": {"schema": ..., "type": ..., "label": ..., "coordinates": ...}}. A POST for an item ID that is not in the project returns 400.

Example

Potato's repository includes a memos example. Run the example from a Potato checkout:

python potato/flask_server.py start examples/advanced/memos-example/config.yaml -p 8000

The example turns on private memos over four short product reviews with one sentiment question. The example has no span scheme, and the quotation report cannot export the example's memos. The QDA Mode example has a codebook-backed span scheme. To try the export, run examples/advanced/qda-mode-example/config.yaml.

Further Reading

• QDA Mode for the codebook, cases, and search that QDA Mode turns on with memos
• Intercoder Reliability in Qualitative Research for when to keep memos private
• Export Formats for the full exporter reference
• Admin Dashboard for the API key and the Overview tab
• Webhooks for delivering memo.issue_assigned events

For implementation details, see the source documentation.

References

Birks, M., Chapman, Y., and Francis, K. (2008). Memoing in qualitative research: Probing data and processes. Journal of Research in Nursing, 13(1), 68–75. doi:10.1177/1744987107081254