Idempotency¶

Running the same thing twice is normal: a job is retried, a response is lost to a timeout, a pipeline re-runs over a store that grew. Datavo answers "what happens then" two ways, and they are not interchangeable.

	Replay	Convergence
The question	has this operation already happened?	is this work already done?
Keyed on	`operation_id`, compared exactly	`(producer_id, key)`, derived from content
Applies to	source import, add, attach	attach only
You get	the existing operation, no new work	a workset containing only what is missing
Resumes partial progress	no	yes

Both are live on an attach at the same time. Neither replaces the other: replay stops a retried job from opening a second ingest; convergence stops it from recomputing what it already computed.

The guide teaches replay in chapter 4, convergence in chapter 6, and names them side by side in chapter 7. This page is the full rules.

Replay¶

operation_id is yours. Datavo compares it by exact string equality and never derives, normalizes, or hashes it. Two ids differing by one character are two operations.

client.create_source_import(SourceImportCreateRequest(
    sample_store="voice_corpus_v1",
    operation_id="studio-session-4821",     # yours; the same one replays
))
client.add_to_store("voice_corpus_v1", operation_id="seed-from-corpus", from_store="...")
client.attach_keys("voice_corpus_v1", "f0.tar", keys=["f0.npy"],
                   producer_id="f0_v001", operation_id="f0-run-42")

Submitting an operation whose operation_id already exists in that store returns the existing operation and materializes nothing.

Omit it and you never replay¶

The SDK generates a fresh UUID when you pass no operation_id, so an interactive call opens a new operation every time — replay is something you opt into by passing a deterministic id, never something that surprises you. The server always receives an id; it only ever compares, never invents.

Pass a deterministic id when a retry of the same logical work should collapse into one operation — a nightly import keyed on the session it imports.

Where each kind resolves it¶

Source import — at create. A replay returns the existing import, so an interrupted upload resumes into it.
Add — at commit. A replay returns the existing node.
Attach — at reserve, deliberately: by commit the shard is already uploaded and the duplicate work is done. A replay of an ingest still awaiting its shard hands the upload URL back so the upload resumes; a replay of one whose shard was already accepted returns no upload URL, and the caller polls that ingest to its terminal state instead.

A failed operation releases its id¶

An operation in a terminal failed state does not hold its operation_id against a retry. Submitting the same id again creates a new operation.

Without this, a deterministic operation_id would be a trap: the first failure would make that id permanently unusable, and a retried pipeline job could never run again. The failed operation stays in the store's operation log under its own system id, so nothing is lost.

`source_ref` is the deprecated spelling¶

source_ref was the source import's replay key before operation_id existed on every kind. It still works:

operation_id given → used.
only source_ref given → it is the operation_id.
both given with different values → 400. Neither is silently preferred; which one won would be a rule you never read.

Lookups follow the same split: get_source_import_by_operation_id and list_source_import_samples_by_operation_id are canonical, and the by_source_ref verbs remain as deprecated aliases (see deprecations). A label for the batch — a session id, an export job — belongs in metadata, which every operation carries.

Convergence¶

An attach workset subtracts what the producer has already written:

client.create_workset(filters=[
    HasKeys(["audio.wav"]),                                    # inputs you need
    Not(AttachedBy(producer_id="f0_v001", output_keys=["f0.npy"])),  # minus what's done
])

Two properties follow, and they are why an attach loop selects rather than enumerates:

Doneness is an empty workset. sample_count == 0 means there is nothing to compute. A re-run after a successful pass is a no-op, not a recompute.
Partial progress resumes. An attach that covered half a store leaves a workset containing exactly the other half. This is the property replay does not give you: replay is all-or-nothing per operation, convergence is per sample.

The boundary is (producer_id, key). A second producer writing the same key sees every sample as its own work — that is what makes two producers of one key comparable rather than competing. The same producer sees none.

producer_coverage(store, producer_id) reports the same thing out of loop; it agrees with the workset for the same producer, store and keys.

Not idempotency: the duplicate-content guard¶

Datavo hashes a source import's committed parts. Re-committing byte-identical parts into the same store fails the import — source import archive already committed as <id> — whatever operation_id you used.

That is a rejection, not a replay: the second import ends in failed and wait_for_source_import raises, rather than the first import being handed back. The two are easy to confuse because both stop duplicate data, so the rule is worth stating plainly:

same operation_id → replay → you get the first operation back, no error.
same bytes, new operation_id → rejection → a second operation is opened and then fails.

Look an existing archive up with get_source_import_by_archive(store, "<sha256>").

One word per concept¶

Write	Not
replay	"dedup", "idempotent create", "deduplicate"
convergence	"drain", "skip", "incremental", "already-done check"
operation id	"source_ref", "idempotency key", "request id", "replay key"