7. Store operations

Runnable companion: examples/04_operations.py

Chapters 4–6 each performed one operation. This chapter reads them back: the store's operation log is its structure and its history in one list.

The log

ops = client.list_store_operations("voice_corpus_v1")["operations"]
for op in ops:
    print(op["kind"], op["state"], op["summary"], op["id"])

source_import  committed  demo-batch-001                 src_9f2c…
add            ready      sample_store:voice_corpus_v0   add_41ab…
attach         committed  f0_v001: f0.npy                att_77de…

Oldest first, one entry per operation, with two ids that do different jobs:

• operation_id is yours — the id you passed (or the SDK generated) when you started the operation. It is compared exactly, never derived, and it is how you find your own operation again to replay or wait on it.
• id is the system id of the node. It is what removal takes.

Running things twice

This is the first chapter that shows all three operation kinds at once, so it is the place to name the two mechanisms that make re-running safe. They answer different questions and both are live:

	Replay	Convergence
Asks	has this operation already happened?	is this work already done?
Keyed on	operation_id, compared exactly	(producer_id, key), from content
Applies to	source import, add, attach	attach only
You get	the existing operation, no new work	a workset of only what is missing
Resumes partial progress	no	yes

You met replay in chapter 4 (importing the same batch twice) and again in chapter 5 (the add's stable id). You met convergence in chapter 6 (the attach workset draining to empty).

On an attach both apply, and neither substitutes for the other: replay stops a retried job opening a second ingest, convergence stops it recomputing what it already computed. Two more rules worth knowing — a failed operation releases its operation_id so a retry under a stable id works (except a large add that failed partway: replaying that one resumes it), and the source-import archive hash rejects duplicate bytes rather than replaying them. Both, in full, in idempotency.

state is that operation's own lifecycle (planning → committing → ready / committed, or failed), summary is a human hint — an operation_id, an add's source, a producer and its keys — and display_name / metadata are whatever you attached when you created it.

An add also reports its progress, materialized_sample_count of sample_count. Every entry reports reclaim_count: how many of its attempts ended without an outcome because the worker running them was killed (out of memory, a scale-in, a restart). A retried attempt is normal; an operation whose attempts keep dying ends failed after four, with that reason in error_message, instead of being retried forever.

The log is deliberately cheap: it is a bounded union over the store's operations.

When you are waiting for one operation, ask for that one — operation_id narrows the log to it, served by the per-store uniqueness index, and that is what wait_for_add polls:

client.list_store_operations("voice_corpus_v1", operation_id="seed-from-corpus")

A store accumulates one entry per grow or attach, so a long-lived store's log is long. Read a page of it, and read it newest-first when the question is "what happened last":

page = client.list_store_operations("voice_corpus_v1", limit=20, order="newest")
page["total"]        # the whole log; the entries are the page

datavo sample-store operations list voice_corpus_v1 --order newest --limit 20

Both default to the whole log, oldest-first — the answer above — so nothing you already wrote reads differently.

Removing one

client.remove_store_operation("voice_corpus_v1", op_id)     # the system `id`

You can remove an operation you added, and a store's owner can remove any operation in their store. Removal is dependency-gated, and it does different work per kind:

• A grow operation (source_import, add) is removed with everything written onto its samples: removing it deletes the samples it created and every row on them, keys attached onto them included. An attach operation that wrote only onto those samples leaves the log with them; one that also wrote onto other samples stays, with its coverage of those intact.
• An attach operation is reverted: removal invalidates exactly the coverage that op wrote — its producer presence and revisions — and drops it from the log. Not(AttachedBy(producer, keys)) then re-selects exactly those samples, so a normal producer re-run recomputes only them; the producer and every other sample's coverage stay intact. This is the surgical way to redo one bad run on a subset of a store. To undo a producer's contribution across the whole store instead, remove the producer — client.remove_producer(store, producer_id).

A grow operation's removal returns 409, naming every blocker, while:

• a dataset holds any of its samples — delete the dataset first;
• work in the store is still in flight — an import, an attach or an add that has not finished, or a workset still planning;
• another add brought keys onto the samples an add created — remove that add first;
• for an add that brought keys onto samples already in the store, an attach has written onto those samples since — they outlive the removal, so revert that attach first.

A protected downstream reference — a dataset that pins this data — blocks removal (of the producer, or of an attach op that wrote it) with a 409 too.

Preview a grow operation's removal before running it. A dry run returns what the removal would delete and every blocker, and changes nothing:

preview = client.remove_store_operation("voice_corpus_v1", op_id, dry_run=True)
preview["counts"], preview["blockers"]

datavo sample-store operations remove voice_corpus_v1 <op-id> --dry-run

A removal is one transaction. A large one can outlast the client's read timeout and still commit, so confirm against the operation log rather than retrying.

Reading the store's shape

The log tells you how the store was built; two more calls tell you what it holds now:

detail = client.get_store("voice_corpus_v1")
detail.sample_count
[k.key for k in detail.keys]                       # keys present in the store

client.get_store_stats("voice_corpus_v1")          # aggregate counts and durations
client.list_store_attribute_names("voice_corpus_v1")   # searchable facets
client.list_store_attribute_values("voice_corpus_v1", "language", value_type="text")

None of this is sample bytes — it is the store's current shape, which is what the UI shows and what you inspect before selecting anything. Reading actual samples is the next chapter.

Pitfalls

• Passing operation_id to removal. Removal takes the system id; the two are different fields on the same entry.
• Reading state as store health. It is per operation: a failed import next to a committed one means that batch did not land. Re-import that batch; the samples the other operations wrote are unaffected.
• Expecting removal to undo an attach. Attach removal routes you to producer-level removal by design.

---

Next: 8. Read samples — the read side starts here.