7. Store operations

Runnable companion: examples/04_operations.py

Chapters 4–6 each performed one operation. This chapter reads them back: the store's operation log is its structure and its history in one list.

The log

ops = client.list_store_operations("voice_corpus_v1")["operations"]
for op in ops:
    print(op["kind"], op["state"], op["summary"], op["id"])

source_import  committed  demo-batch-001                 src_9f2c…
add            ready      sample_store:voice_corpus_v0   add_41ab…
attach         committed  f0_v001: f0.npy                att_77de…

Oldest first, one entry per operation, with two ids that do different jobs:

• operation_id is yours — the id you passed (or the SDK generated) when you started the operation. It is compared exactly, never derived, and it is how you find your own operation again to replay or wait on it.
• id is the system id of the node. It is what removal takes.

Running things twice

This is the first chapter that shows all three operation kinds at once, so it is the place to name the two mechanisms that make re-running safe. They answer different questions and both are live:

	Replay	Convergence
Asks	has this operation already happened?	is this work already done?
Keyed on	operation_id, compared exactly	(producer_id, key), from content
Applies to	source import, add, attach	attach only
You get	the existing operation, no new work	a workset of only what is missing
Resumes partial progress	no	yes

You met replay in chapter 4 (importing the same batch twice) and again in chapter 5 (the add's stable id). You met convergence in chapter 6 (the attach workset draining to empty).

On an attach both apply, and neither substitutes for the other: replay stops a retried job opening a second ingest, convergence stops it recomputing what it already computed. Two more rules worth knowing — a failed operation releases its operation_id so a retry under a stable id works, and the source-import archive hash rejects duplicate bytes rather than replaying them. Both, in full, in idempotency.

state is that operation's own lifecycle (planning → committing → ready / committed, or failed), summary is a human hint — an operation_id, an add's source, a producer and its keys — and display_name / metadata are whatever you attached when you created it.

The log is deliberately cheap: it is a bounded union over the store's operations, so it is fine to poll (that is exactly what wait_for_add does).

A store accumulates one entry per grow or attach, so a long-lived store's log is long. Read a page of it, and read it newest-first when the question is "what happened last":

page = client.list_store_operations("voice_corpus_v1", limit=20, order="newest")
page["total"]        # the whole log; the entries are the page

datavo sample-store operations list voice_corpus_v1 --order newest --limit 20

Both default to the whole log, oldest-first — the answer above — so nothing you already wrote reads differently.

Removing one

client.remove_store_operation("voice_corpus_v1", op_id)     # the system `id`

Removal is dependency-gated, and it does different work per kind:

• A grow operation (source_import, add) is removable while nothing has enriched the samples it created. Removing it deletes those samples. The first attach freezes every prior grow operation — removing one then returns 409, because an attach may have written onto its samples and the log would stop explaining the data. Enriched data is discarded by deleting the store, not by unpicking its middle.
• An attach operation is reverted: removal invalidates exactly the coverage that op wrote — its producer presence and revisions — and drops it from the log. Not(AttachedBy(producer, keys)) then re-selects exactly those samples, so a normal producer re-run recomputes only them; the producer and every other sample's coverage stay intact. This is the surgical way to redo one bad run on a subset of a store. To undo a producer's contribution across the whole store instead, remove the producer — client.remove_producer(store, producer_id).

A protected downstream reference — a dataset that pins this data — blocks removal (of the producer, or of an attach op that wrote it) with a 409.

Reading the store's shape

The log tells you how the store was built; two more calls tell you what it holds now:

detail = client.get_store("voice_corpus_v1")
detail.sample_count
[k.key for k in detail.keys]                       # keys present in the store

client.get_store_stats("voice_corpus_v1")          # aggregate counts and durations
client.list_store_attribute_names("voice_corpus_v1")   # searchable facets
client.list_store_attribute_values("voice_corpus_v1", "language", value_type="text")

None of this is sample bytes — it is the store's current shape, which is what the UI shows and what you inspect before selecting anything. Reading actual samples is the next chapter.

Pitfalls

• Passing operation_id to removal. Removal takes the system id; the two are different fields on the same entry.
• Reading state as store health. It is per operation: a failed import next to a committed one means that batch did not land. Re-import that batch; the samples the other operations wrote are unaffected.
• Expecting removal to undo an attach. Attach removal routes you to producer-level removal by design.

---

Next: 8. Read samples — the read side starts here.