CLI reference¶

The datavo CLI covers configuration, catalog inspection, dataset and split authoring from spec files, downloads, and the archive/delete lifecycle. Data-plane work — selecting, iterating, attaching — is the SDK (guide).

Everything below runs as datavo <command> when the CLI is installed, or uv run datavo <command> from this repo. Most commands take --format json for machine-readable output.

Auth¶

datavo auth login                                    # interactive Entra login
datavo auth status                                   # local credential state
datavo auth logout                                   # clear token and MSAL cache

The CLI targets https://prod.datavo.io unless you tell it otherwise, so a fresh install needs nothing but auth login.

Profiles — only for other instances¶

datavo config set dev --url https://dev.datavo.io   # create/update a profile
datavo config use dev                                # make it active
datavo config list                                   # profiles, active one marked
datavo config show                                   # the active profile + server config
datavo config remove dev

Profiles live in ~/.datavo/config.json; omitting the profile name on config set names it after the URL's hostname. auth login takes no flags against an Entra server — --token, --tenant-id, --client-id and --scope are overrides for unusual setups.

Every command targets the active profile — and with none set that is production. Check datavo config list before anything that writes; DATAVO_PROFILE=<name> datavo … overrides for one invocation, as does DATAVO_API_BASE_URL=<url>.

datavo version prints the CLI's own version; it does not call the server. The server's version and public settings come from GET /config, which config set fetches and caches.

On a failure the CLI prints the request method, URL and status; a validation error (422) is summarized field-by-field, and --verbose (or DATAVO_VERBOSE=1) adds the full response body. When the CLI's version differs from the server's, the error ends with an upgrade line — a stale CLI is the usual cause of a 422 or an unparseable response, and uv tool upgrade datavo-cli is the fix.

Stores¶

datavo sample-store create voice_corpus_v1
datavo sample-store list
datavo sample-store list --query "spirelight trial" --sort samples   # every word, any order
datavo sample-store get voice_corpus_v1
datavo sample-store archive voice_corpus_v1         # required before delete
datavo sample-store restore voice_corpus_v1         # undo an archive
datavo sample-store delete voice_corpus_v1          # archived stores only

# operation log — inspect and revert
datavo sample-store operations list voice_corpus_v1
datavo sample-store operations remove voice_corpus_v1 <op-id> --dry-run   # what it would delete, and its blockers
datavo sample-store operations remove voice_corpus_v1 <op-id>

# the source imports whose samples the store holds, wherever each ran (the ImportedBy names)
datavo sample-store imports voice_corpus_v1
datavo sample-store imports voice_corpus_v1 --prefix studio:spirelight:session: --limit 50

# seed a disposable store from real data (chapter 5 of the guide)
datavo sample-store copy eval-scratch --source-store voice_corpus_v1 --limit 50
datavo sample-store copy ds-scratch --source-dataset voice_train_v1 --limit 30

datavo sample-store transfer <source> <target>      # move a whole store's contents

imports lists one row per source import whose samples the store holds — imported into it or brought in by adds — with its operation_id and how many of the store's samples ImportedBy would select. --prefix narrows by operation id, exactly; --offset and --limit page.

copy requires a source and --limit; --key projects keys, --has-key narrows to samples that have one, and --randomize (with --seed) takes a repeatable random subset. The full archive→delete lifecycle runs from the CLI: archive first, then delete. You archive and delete what you created — a store, a dataset, a split set — and a write grant on someone else's does not change that. You can remove the operations and producers you added, and a store's owner can remove any of them in their store. Permanent deletion is blocked while a protecting dataset depends on the store — see lifecycle and retention.

operations list shows the store's grow/attach operations oldest-first with their ids; operations remove <op-id> reverts one — an attach operation invalidates exactly the samples it wrote (so a producer re-run recomputes only them), an add or source_import removes the samples it introduced with every row on them, keys attached onto them included. It reports HTTP 409, naming every blocker, while a dataset holds those samples, work in the store is in flight, or a protected reference pins the operation. --dry-run previews an add or source_import removal, counts and blockers, and changes nothing. A removal is one transaction: a large one can outlast the client's read timeout and still commit, so confirm against operations list rather than retrying.

Samples¶

datavo sample get <sample_id>            # detail + available keys
datavo sample history <sample_id>        # key revision history

Datasets¶

Dataset and split authoring is spec-file first, which is what makes definitions reviewable and checked in:

datavo dataset list                                # names, state, sample counts, split sets
datavo dataset list --query "hw32 train" --sort relevance   # every word, any order; best match first
datavo dataset list --store voice_corpus_v1        # the datasets that read a store
datavo dataset preview --spec voice_train.yaml     # dry run: counts, no writes
datavo dataset create  --spec voice_train.yaml
datavo dataset get voice_train_v1
datavo dataset add-key-group voice_train_v1 --spec conditioning_group.yaml
datavo dataset pull voice_train_v1 --output-dir ./downloaded/train
datavo dataset archive voice_train_v1              # required before delete
datavo dataset restore voice_train_v1
datavo dataset delete  voice_train_v1              # archived datasets only

list takes --query, --include-archived, and --offset/--limit. As with stores, deletion runs archive→delete from the CLI and is blocked while a consumer depends on the dataset.

get on a ready dataset also prints its key provenance: for each key, the producers it reads with their member counts, and whether the key is pinned or reads the newest revision when the dataset froze. An imported key shows as imported xN for its N source imports. --format json prints the dataset alone; the provenance is client.get_dataset_key_provenance(name).

A spec is the create request in YAML. The canonical form keeps a workset you selected with the SDK (guide chapter 11):

name: voice_train_v1
from_workset: ws_7f3c…
shard_plan:
  shard_size: 1000
scalars:
  sample_rate: 24000

Add --wait to create to poll until the dataset is ready instead of returning the planning response. pull writes one tar per shard under <output-dir>/<key-group>/, every key group unless --key-group narrows it. Re-running a pull resumes it — each shard tar is named by its content, so a tar already on disk is skipped and only the missing shards download; --overwrite forces a fresh download of every shard.

Split sets¶

datavo split-set preview voice_train_v1 --spec voice_split.yaml   # dry run + diagnostics
datavo split-set create  voice_train_v1 --spec voice_split.yaml
datavo split-set get     voice_train_v1 voice_v1
datavo split-set list    voice_train_v1        # split sets under a parent dataset
datavo split-set list    --query voice_train   # search split sets by their or their dataset's name
datavo split-set splits  voice_train_v1        # the child datasets they generated
datavo split-set archive voice_train_v1 voice_v1   # required before delete
datavo split-set restore voice_train_v1 voice_v1
datavo split-set delete  voice_train_v1 voice_v1   # archived split sets only

name: voice_v1
assignment_seed: 7
splits:
  - name: train
    ratio: 0.6
  - name: val
    ratio: 0.2
  - name: test
    ratio: 0.2
rules:
  - kind: exclusivity
    keys: [speaker.txt]
    enforcement: hard
shard_plan:
  shard_size: 256

The spec's name is the split-set name and becomes the prefix of the generated datasets (voice_v1/train, …). Every split command takes the parent dataset as its first argument. Pass --split-set <name> to splits to scope the listing when a dataset has several.

Those children are ordinary datasets, so datavo dataset pull voice_v1/train --output-dir ./data downloads one split's tars the same as any other dataset.

Cache¶

datavo warm voice_v1/train                       # pre-populate the local cache
datavo warm voice_v1/train --key audio.wav       # only these keys
datavo warm voice_v1/train --concurrency 16      # parallel installs (default: the read prefetch pool)

warm installs shards ahead of a run by content hash — every key unless --key narrows it — and never opens a tar or decodes a sample, so it runs at the speed of the tier it reads. It reports what it did on stderr:

warmed 4032 shards / 4032 tars, 324.3 GB (4032 fetched, 0 already cached) in 812s — 399.4 MB/s for 'voice_v1/train'

--concurrency bounds the parallel installs and --cache-config points at a shards.yaml other than the default. See the shard cache and its Getting maximum read throughput section.

datavo prune-cache                               # clear stale locks and staging orphans
datavo prune-cache --dry-run                     # report only

prune-cache sweeps every folder tier of the resolved hierarchy: it removes lock files and staging orphans past their TTL, brings entries written before the tier had modes up to them, and reports what it cleared plus each tier's occupancy against max_bytes. It touches local disk only — no server, no credentials. Run it as the owner of the cache directories; entries owned by someone else are counted and named rather than skipped silently.