Shard Cache Hierarchy¶

This document explains how Datavo caches shard tars on the consumer side and how to shape that cache for fast IO.

The shard cache hierarchy is a property of the machine running the consumer code. Datavo dataset consumers (SampleStream.from_dataset(...)) read from it transparently. They never need to know which tiers exist, where they live, or how big they are.

Goals¶

The hierarchy is designed around one boundary:

Consumers open a dataset and iterate samples. They do not configure storage.
The local IO topology is described in a single YAML file — no tier needs pre-warming.

Specifically:

A consumer-side iteration over a Datavo split should hit local disk on the second pass, even when several users share one machine.
You can add a shared scratch dir, a per-user SSD, or both by editing one config file. No code change on the consumer side.
The cache should fill itself naturally as consumers read. There is no separate prefetch or bulk-import step.

Identity And Cache Key¶

Datavo workset shards are addressed by content_hash, a deterministic function of exactly what determines the shard's bytes:

the ordered sample ids of the shard and its index,
the key group and its keys, and the group's coverage (mode, optional keys),
the composition the keys resolve through — each source's store or dataset, split, row filter, key_map and producer pins, but not the requested key list,
which revisions the cells resolve to: the dataset's revision watermark when a dataset is replayed, and otherwise a digest of the revisions currently presented for the shard's own cells.

Two reads that would stream the same bytes share one cache entry — a superset warm, in any key order, serves a single-key run. And a read whose bytes changed gets a new hash: re-attaching a key a store-scoped read uses (a second producer, or the same producer after a revert) makes its shards miss once instead of serving the old bytes, while attaching some other key leaves them hits. A dataset's hash never moves, because a dataset is frozen.

Files are stored as <content_hash>.tar at the top level of each tier directory. Tier directories also contain a .staging/ subdirectory for in-flight downloads and a .locks/ subdirectory for per-shard advisory locks.

Tiers¶

A tier is a backend that can hold shards by content hash. Two kinds of tier ship today; both implement the same small interface so the hierarchy can treat them uniformly.

Folder tiers¶

A folder tier is a directory on a local filesystem.

ShardCacheTier(
    path="/scratch/datavo/shards",
    max_bytes=1_099_511_627_776,   # 1 TiB
    name="scratch",
)

Folder tier semantics:

path is created on first use. ~ and environment variables are expanded.
max_bytes is optional. Missing or zero means unbounded.
Eviction is LRU based on file mtime. Reads update mtime so popular shards stay resident.
A tier does not measure itself on every install. Walking a tier to total its bytes is its dominant cost on a network filesystem, and below the high-water mark (90% of max_bytes) no achievable total could make eviction necessary. A tier measures itself exactly on one of three triggers: its estimate reaches the high-water mark; the installing process has itself installed more than its share of the headroom (the last measurement's headroom divided by 8 assumed concurrent writers); or free space on the filesystem falls below one eviction batch. Everything else costs no directory listing at all. Measurements are shared between processes through an .occupancy marker in the tier, so a short-lived container reads one small file rather than stat'ing every entry.
Eviction frees a band, not one shard's worth. A measurement that finds the tier above the mark evicts LRU down to 75% of max_bytes. A tier that stopped at its cap would measure itself again on the very next install; the band it frees is what makes the installs that follow free.
Each tier locks per content_hash (POSIX fcntl.flock) so concurrent processes on the same machine do not double-fetch or stomp on installs.
The tier is directly readable: the consumer iterates the tar file in place, no copying involved.

Cloud tiers¶

Three cloud-backed tiers ship today, one per major cloud:

AzureBlobShardCacheTier — Azure Blob container
S3ShardCacheTier — AWS S3 bucket
GcsShardCacheTier — Google Cloud Storage bucket

All three behave the same way from the hierarchy's perspective:

Shards are stored at <content_hash>.tar inside the bucket/container. An optional prefix extends that to <prefix>/<content_hash>.tar for sharing one bucket across deployments.
The tier is not directly readable. A cloud-tier hit is materialized into a folder tier first; the consumer always reads from local disk. See Cloud topology below.
max_bytes is informational. Cloud buckets do not evict locally — eviction is expected to be managed via cloud-side lifecycle policies (Azure blob rules, S3 lifecycle, GCS lifecycle/autoclass). Setting max_bytes still lets the hierarchy skip the tier for shards that exceed the limit.
The cloud SDK is imported lazily. A tier is constructable without the SDK installed; the import only fires when you actually exercise the tier. Concrete extras:
Azure: pip install 'datavo-sdk[azure]' (azure-storage-blob, plus azure-identity for managed-identity auth).
AWS: pip install 'datavo-sdk[aws]' (boto3).
GCP: pip install 'datavo-sdk[gcp]' (google-cloud-storage).
All three: pip install 'datavo-sdk[all-clouds]'.

AzureBlobShardCacheTier(
    container_url="https://datavoeastus.blob.core.windows.net/shards",
    name="region-eastus",
    # sas_token: SAS string for ad-hoc auth, otherwise DefaultAzureCredential
    # managed_identity_client_id: pin to a specific user-assigned MI
)

S3ShardCacheTier(
    bucket="datavo-cache-us-east-1",
    region="us-east-1",
    name="region-us-east-1",
    # boto3's default credential chain — env vars, EC2 instance profile,
    # ECS task role, EKS service account...
)

GcsShardCacheTier(
    bucket="datavo-cache-us-central1",
    project="datavo-prod",
    name="region-us-central1",
    # Application Default Credentials — workload identity on GKE, service
    # account on GCE, gcloud auth locally...
)

Tiers are independent of each other. Each tier evicts on its own schedule against its own capacity. The hierarchy never coordinates eviction across tiers.

Hierarchy¶

A ShardCacheHierarchy is an ordered list of tiers, fastest first. The default order is the order they appear in YAML.

The hierarchy enforces one invariant: at least one folder-backed tier must be present. That tier is the "local landing zone" where any remote-tier hit is materialized so the consumer can read a regular file. A hierarchy made up of only remote tiers would have nothing to hand back to the consumer.

Reads¶

hierarchy.get_or_fetch(content_hash, fetcher) walks tiers fastest to slowest looking for the shard. The first tier that has it serves the read.

If no tier has it, the caller-supplied fetcher writes the tar into a staging file. The fetcher is opaque to the cache. For a Datavo consumer the fetcher issues a workset stream ticket and downloads the shard.

Writes: populate on miss¶

On a complete miss the staged tar is installed into every tier that can hold it. Each tier respects its own capacity:

if a tier's max_bytes is smaller than the shard, the tier is skipped with a warning;
if at least one tier accepts the shard, the call succeeds and returns the topmost installed path;
if no tier accepts, the call raises.

This is what makes the cache self-warming. You do not need to pre-fill the scratch tier. The first consumer to read a shard puts it everywhere.

Writes: promote on hit¶

When a slower tier serves a read, the shard is also copied into the faster tiers above it that did not have it. After promotion the call returns the topmost installed path so the caller reads from the fastest available tier even on a first-time slow-tier hit.

Promotion is per-tier subject to capacity:

tiers too small for the shard are skipped with a warning;
if every faster folder tier rejects, the read is served from the original slower-tier hit (no error) — unless the hit was on a remote tier, in which case the call raises because there is no local landing zone for the consumer.

Promotion happens once per call. After a promote the shard is in the faster tier; subsequent calls find it there directly.

Reads from remote tiers¶

A hit on a blob tier (or any tier with supports_direct_read = False) is materialized through the topmost folder tier's .staging/ area on its way to the consumer:

The hierarchy downloads the blob into a staging file inside the topmost folder tier.
The staging file is then installed into every faster folder tier that can hold it (the same promote-on-hit path used for slow-folder hits).
The hierarchy returns the path inside whichever folder tier accepted first — the consumer reads from a regular local file.

That single staging step is what couples the local tier and the regional blob tier into a real hierarchy: the consumer doesn't notice the blob exists, and the blob's network cost is paid once per shard per VM.

Hits do not refetch¶

The fetcher is never called when any tier has the shard, including the promote-on-hit path. There is no revalidation step. A content hash uniquely identifies the bytes; if a tier has content_hash.tar, it is by definition the correct bytes.

Getting maximum read throughput¶

A read is bounded by whichever is slower: the tier that serves the bytes, or the process that consumes them. The tiers are rarely the limit — a local NVMe tier serves gigabytes per second and an NFS tier on a 10 GbE link close to a gigabyte per second — so when a run pulls far below that, the cause is almost always on the consumer side. In order of payoff:

Warm the key groups you will read, before the run. A key group is the cache unit, so a warm of inputs has not cached features. Within that, the list you pass does not matter: a shard's cache identity does not include the requested keys, so a warm may pass a superset of what the run reads, in any order, and every shard the run touches is a hit. One warm therefore serves every consumer of the dataset — training, evaluation and QC share the group tars they have in common, whatever each of them asks for.
Warm with warm_cache or datavo warm, never with an iterate-and-discard loop. Iterating a stream parses every tar and decodes every sample on the interpreter lock; a loop that throws the samples away pays that cost for nothing. warm_cache(stream) installs every shard by content hash through the same per-shard fetch the read uses and never opens a tar, so it runs at the tier's speed and reports shards, tars and bytes:

from datavo_sdk import SampleStream, warm_cache

stream = SampleStream.from_dataset("voice_demo_v1/train", keys=["audio.wav"])
print(warm_cache(stream).summary_line())
# warmed 4032 shards / 4032 tars, 324.3 GB (4032 fetched, 0 already cached) in 812s — 399.4 MB/s

datavo warm voice_demo_v1/train --key audio.wav --concurrency 16

concurrency bounds the parallel installs; it defaults to the read path's prefetch pool so a warm and the read that follows exert the same pressure on a shared tier. 3. Decode only what you consume. decode=True (the default) runs torch.load / np.load / text decoders per sample. A pass that ignores payloads — counting, listing ids, checking coverage — should construct the stream with decode=False. 4. Put decoding in DataLoader worker processes, not prefetch threads. The prefetch pool overlaps downloads with consumption; its threads share the interpreter lock with decoding, so raising prefetch does not speed up a consumer that is CPU-bound — it only adds contention. Use DataLoader(num_workers=N) so each worker decodes in its own process and the main process stays I/O-only. 5. On NFS tiers, raise the kernel readahead. Linux defaults NFS mounts to a 128 KB readahead (/sys/class/bdi/<major:minor>/read_ahead_kb; find the device in /proc/self/mountinfo), so one sequential stream issues 128 KB reads regardless of rsize=1M, capping a single stream near 150–200 MB/s. Setting it to a few MB lets one stream approach the link rate. Mount with noatime: every hit already touches mtime for LRU, an atime update on top is a second metadata write per read. 6. Prefer fewer, larger shards on network tiers. Every hit costs metadata operations — the per-hash lock file and the mtime touch — and on NFS each is a round trip. Tens of thousands of 1 MB tars amplify that; a shard plan with larger shards amortises it. 7. Expect the first pass on a host to run at the next tier's copy rate. A shard not yet in the fastest tier is served from the tier below and promoted (copied) on the way; a dataset new to a host is therefore bounded by that tier's throughput once, and by local disk on every later pass. Size the top tier above the dataset you warm, or LRU eviction turns every pass into a first pass.

Telling a CPU-bound consumer from a slow tier¶

Three checks, none of which need a profiler:

CPU shape. A CPU-bound Python consumer shows one to two cores of usage spread thinly across many threads (top -H: a dozen threads at ~10 % each) with one or two runnable at any instant. A tier-bound consumer shows threads sleeping in I/O and a total well under one core.
NFS wait time. /proc/self/mountstats on the worker gives per-operation totals for the mount; the READ line's cumulative round-trip time over a 10 s window divided by the number of reader threads is the fraction of time they spent waiting on the tier. A few percent means the tier is idle.
Raw tier rate. dd if=<tier>/<hash>.tar of=/dev/null bs=1M iflag=direct (O_DIRECT bypasses the page cache) on one file, then eight in parallel. If that is an order of magnitude above what the run pulls, the run is the limit.

The measurement behind this section: a prewarm that iterated a 4,032-shard dataset pulled 21 MB/s from an NFS tier for hours with the GPU idle, while the same tier served 182 MB/s to one dd stream and ~950 MB/s to eight; the process sat at 1.9 cores across 17 threads, waiting on NFS 1 % of the time. The hierarchy's own promotion path on the same worker, fed the same way warm_cache feeds it (16 concurrent get_or_fetch calls, no decoding), promoted 3,000 tars / 84 GB from that NFS tier into the SSD tier at 493 MB/s.

Configuration¶

The hierarchy is resolved when a read opens. The resolution order is:

An explicit cache= hierarchy passed to SampleStream.from_dataset(...) or SampleStream.from_workset(...).
The client's own cache — client.cache. A client that was handed a hierarchy when it was built carries it, so a read on that client reaches that cache by passing client= and nothing else. A plain DatavoClient carries none and falls through to the next step.
An explicit cache_config_path argument to the same constructors or to get_default_shard_cache(...).
/etc/datavo/shards.yaml (system-wide).
~/.config/datavo/shards.yaml (per-user override).
A built-in default: a single user-local tier at $DATAVO_SHARD_CACHE_DIR or ~/.cache/altavo/datavo/shards with 50 GiB LRU capacity.

System-first is intentional. The shared IO topology is a property of the machine, not the user. Per-user overrides exist for laptops and developer workstations.

Every open logs one INFO line naming the resolved tier roots and which of those steps supplied them, so a read that fell through to the last one says so. Pass require_cache=True to either constructor to fail instead of opening on that built-in tier — worth doing in a job that only makes sense on a real cache disk.

YAML schema¶

tiers:
  - name: scratch
    type: folder                  # default; can be omitted
    path: /scratch/datavo/shards
    max_bytes: 1TB
  - name: home
    path: ~/.cache/altavo/datavo/shards
    max_bytes: 50GB
  - name: region-eastus
    type: azure_blob
    container_url: https://datavoeastus.blob.core.windows.net/shards
    # sas_token: "..."            # optional; otherwise DefaultAzureCredential
    # prefix: "team-a"            # optional namespacing inside the container
    # managed_identity_client_id: "..."   # optional; specific UAMI client id
  - name: region-us-east-1
    type: s3
    bucket: datavo-cache-us-east-1
    region: us-east-1
    # prefix: "team-a"
  - name: region-us-central1
    type: gcs
    bucket: datavo-cache-us-central1
    # project: "datavo-prod"      # optional
    # prefix: "team-a"

Common fields:

tiers (required, non-empty list). Order is fastest first. At least one tier must be folder-backed.
tiers[].type (optional). One of folder, azure_blob, s3, gcs. Defaults to folder.
tiers[].name (optional). Display name used in logs.
tiers[].max_bytes (optional). Integer or string with a SI-style suffix (B, KB, MB, GB, TB, 1024-based). Missing or zero means unbounded.

Folder-tier fields:

tiers[].path (required). Local filesystem path. ~ and $VAR are expanded.

Azure-blob-tier fields:

tiers[].container_url (required). Azure Blob container URL.
tiers[].sas_token (optional). When present, used for SAS-based auth. Otherwise the tier falls back to DefaultAzureCredential.
tiers[].prefix (optional). Path-style prefix prepended to each blob name.
tiers[].managed_identity_client_id (optional). Passed through to DefaultAzureCredential for pinning to a specific user-assigned managed identity.

S3-tier fields:

tiers[].bucket (required). S3 bucket name.
tiers[].region (optional). AWS region; boto3 figures it out from the default chain if omitted but explicit is usually clearer.
tiers[].prefix (optional). Key prefix prepended to each object name.

GCS-tier fields:

tiers[].bucket (required). GCS bucket name.
tiers[].project (optional). GCP project id; ADC usually resolves it.
tiers[].prefix (optional). Object name prefix.

The legacy root field from the pre-Datavo cache hierarchy is accepted for backward compatibility but ignored, with a warning. The source of truth in the Datavo model is the server itself; the hierarchy only describes local tiers.

Recommended deployment: shared on-prem workstation¶

For a shared GPU workstation with a fast NVMe and a slow shared NFS scratch share:

# /etc/datavo/shards.yaml
tiers:
  - name: local
    path: /var/cache/datavo/shards
    max_bytes: 200GB
  - name: scratch
    path: /scratch/datavo/shards
    max_bytes: 5TB

The first consumer on this machine to read a shard downloads it once and populates both tiers. Subsequent reads on the same shard hit local. A different user reading the same shard later still hits scratch and gets the shard promoted into local on their first read.

Single-user laptop¶

The built-in default is usually enough; no YAML required.

Cloud topology¶

On a cloud cluster of GPU VMs the latency and egress profile of the origin Datavo server is unfavorable to read from on every iteration. The recommended pattern is to put one bucket per region in between the per-VM SSD and the origin: every VM in that region shares the cache, and the cache fills naturally as users in the region consume.

The same shape works on all three clouds, swapping the bucket-side tier:

# /etc/datavo/shards.yaml on each GPU VM in eastus (Azure)
tiers:
  - name: nvme
    path: /mnt/cache/datavo/shards
    max_bytes: 200GB
  - name: region-eastus
    type: azure_blob
    container_url: https://datavoeastus.blob.core.windows.net/shards
    # managed identity picks up the VM's identity; no secrets in this file

# us-east-1 on AWS
tiers:
  - name: nvme
    path: /mnt/cache/datavo/shards
    max_bytes: 200GB
  - name: region-us-east-1
    type: s3
    bucket: datavo-cache-us-east-1
    region: us-east-1

# us-central1 on GCP
tiers:
  - name: nvme
    path: /mnt/cache/datavo/shards
    max_bytes: 200GB
  - name: region-us-central1
    type: gcs
    bucket: datavo-cache-us-central1

What happens (identical across clouds):

First VM in the region opens a dataset. Both tiers miss. The origin server streams the tar to the VM, the hierarchy stages it under /mnt/cache/datavo/shards/.staging/, then installs into nvme and uploads to the region bucket. The shard is now warm for the whole region.
Second VM in the same region opens the same dataset. nvme misses on that VM but the region bucket hits. The hierarchy materializes the object into the VM's nvme.staging/, installs into nvme, and returns the local path. The origin server is never contacted.
A VM in a different region opens the same dataset. That region's bucket misses (different bucket), so the call falls through to the origin server and warms up the new region. Cross-region egress is paid once per region, not once per VM.

For the consumer:

Nothing changes in code. SampleStream.from_dataset(...) reads the local YAML, walks the hierarchy, and hands back a local path.

The hierarchy treats all three cloud tiers identically; you can in principle mix providers in one config (e.g. an Azure deployment that also reads from a shared GCS bucket of legacy datasets), though that's an unusual setup.

Why not stream straight from the bucket?¶

The hierarchy could in principle read the tar straight out of the bucket over the network, skipping the local copy. The current design instead materializes the object into the local tier on first read because:

Iterating a Datavo dataset is usually a long-running training loop that reads each shard several times across epochs; the local copy pays for itself after one re-read.
Local mtime-driven LRU then becomes the per-VM working-set policy without any bucket-side coordination.
The consumer's read path stays a single tarfile.open(local_path) call; there is no separate "remote shard" iteration mode.

If a future workload changes that tradeoff (pure shuffle-once, very large shards) a streaming read mode can be added as an opt-in flag without touching the rest of the design.

Concurrency¶

Folder-tier installs are guarded by per-content-hash advisory file locks (tier.path/.locks/<content_hash>.lock). Two processes on the same machine that race on the same shard:

both stage their own download into the same tier's .staging/,
one acquires the lock first, finishes its rename, releases the lock,
the second acquires the lock, observes the file already exists, discards its own staged copy, and returns the existing path.

The lock is per-shard, not per-tier, so concurrent fetches of different shards do not block each other. The hierarchy itself holds no locks; the locking lives entirely inside each tier.

When the lock file cannot be opened at all — its directory is not writable by this process, the mount is read-only, the disk is full — the read continues unlocked after one warning per lock directory. Two processes may then fetch the same shard at once, which is the same outcome the race above already has: both write their own staging file and one atomic rename wins, and since the name is a content hash both wrote the same bytes. Every other error opening the lock still raises.

Cloud-tier installs are race-free at the storage layer: uploads of the same content hash from two VMs may both succeed, but they write the same bytes (same content hash means same payload), so the resulting object is correct either way. The cost is one redundant upload, which is negligible compared to the network fetch from the origin server that both VMs would otherwise do.

Operational Considerations¶

Disk usage. Each tier independently respects its max_bytes. If you set the top tier too small, large shards are skipped there and shared tiers do the work. This is intentional: a too-tight top tier does not break reads, it just degrades them.
Filesystem requirements. Tier directories must support os.rename atomically within themselves (any POSIX filesystem does). Cross-tier installs use shutil.copyfile and tolerate cross-filesystem layouts.
Cleanup. A tier prunes itself. After a fetch — never on a hit — it may sweep its own .locks/ and .staging/: lock files unused for 7 days and .part files untouched for 24 hours are removed. The sweep is rate-limited to once every 6 hours per tier by a marker file, and serialized by its own lock, so several processes sharing one disk perform one sweep between them. datavo prune-cache runs the same sweep on demand. Cached tars are never pruned: capacity is max_bytes and LRU eviction, and the sweep is not a second eviction policy.

Removing a lock file takes two gates, because deleting one another process holds open would break exclusion rather than corrupt anything: the file must be older than the TTL (its mtime is refreshed on every acquire, so age means unused since) and lockable non-blocking, with the unlink performed while holding it. In the small race that remains, the loser fetches a shard someone else is already fetching — one redundant download of content-addressed bytes, never a damaged entry.

.staging/ orphans are pruned rather than counted: a tier's byte accounting skips dotfiles, so orphaned .part bytes count toward neither total_bytes() nor eviction, and without the TTL a tier would exceed max_bytes by their size indefinitely. - Capacity is a bound, not a promise. max_bytes is a disk-full guard, not a correctness constraint: every entry is addressed by content hash and can be refetched, so losing one costs a refetch. A tier enforces it exactly in the band where it binds and approximately below it, and can therefore sit above max_bytes for a while. Three independent backstops bound how far: the byte budget, sized so 8 concurrent writers cannot between them add more than the headroom before one re-measures and republishes for the rest; the 10% high-water margin above that; and the free-space trigger beneath everything, which fires on the disk actually filling regardless of what any estimate says. A tier that has drifted self-corrects on the first install that reaches any of the three.

These are not configuration and are not settable in shards.yaml — they are derived from the tier's own max_bytes. Note that the free-space trigger self-tunes: it never fires on a tier whose cap is a small share of a large filer, and fires exactly when room is needed on a tier whose cap is its disk — the case a byte budget cannot serve on its own, because such a tier fills by promote-on-hit at the speed of the tier below it.

datavo prune-cache reports exact occupancy against max_bytes on demand, and reseeds the shared measurement while it is there. - Cache poisoning. Files in a tier are trusted because they are addressed by content hash. Anyone who manually drops tars into a tier directory must name them with the correct hash. - Observability. A tier logs one INFO line per exact measurement, naming the trigger that caused it (top-band, budget-exhausted, free-space, trust-ceiling, no-measurement), what it measured, how long it took, and what it evicted. The measurement is rare by construction, so the line is not noise — and a tier that starts measuring on every install says so instead of showing up as an unexplained metadata storm on the filer. The hierarchy logs skipped tiers, promotion attempts, and install failures at WARNING. There is no per-tier metrics emission today; capacity and hit rates are surfaced via the standard logging stream.

Permissions¶

A tier is shared infrastructure — one cache disk, several containers, and not necessarily the same uid in each. So the mode of every entry is a property of the tier, set explicitly when the entry is created:

Entry	Default mode
Tier root, namespace dir, `.locks/`, `.staging/`	`0777`
`<content_hash>.tar`	`0666`
`.locks/*.lock`	`0666`

Directories are writable because every sharer must be able to create (staging and lock files), rename into (the atomic install), and unlink from (LRU eviction) them — all three are directory-write operations. There is deliberately no sticky bit: eviction unlinks whatever is coldest regardless of owner, and +t would restrict that to the owner, so the tier would quietly stop evicting other uids' entries and overrun max_bytes.

Files are writable, not merely readable, for a reason worth stating: eviction is ordered by mtime and every cache hit refreshes it with utime, which needs write permission. A world-readable-but-not-writable tar would be untouchable by anyone who did not write it, so a shard read constantly by a second uid would look like the coldest entry in the tier and be evicted while it was the hottest. Under directories anyone can already write, restricting the file buys nothing.

Taking a lock is likewise a write open, so a lock file another uid may need must be writable by them. Where locking is unavailable anyway — an unwritable .locks/, a read-only mount, a full disk — the read proceeds without a lock after one warning, rather than failing: the lock prevents duplicate work, not corruption.

Both modes are configurable per tier, for a machine that should not have a world-writable cache:

tiers:
  - name: personal
    type: folder
    path: ~/.cache/altavo/datavo/shards
    dir_mode: "0700"      # octal, as a string
    file_mode: "0600"

A mode that cannot be set — a directory created earlier by another uid, which this process may use but not chmod — is reported once, naming the path, the mode it has, the mode it wanted, and the owner. Entries already on disk when a tier first gains modes are brought up to them by datavo prune-cache, run as their owner; a read never repairs one, it simply treats an unusable entry as a miss.

Programmatic Access¶

The hierarchy and its tiers are public SDK surface:

from datavo_sdk import (
    AzureBlobShardCacheTier,
    SampleStream,
    GcsShardCacheTier,
    S3ShardCacheTier,
    ShardCacheHierarchy,
    ShardCacheTier,
    get_default_shard_cache,
    warm_cache,
)

# Default consumer path.
ds = SampleStream.from_dataset("voice_demo_v1/train", keys=["audio.wav"])

# Fill the cache ahead of a run without decoding a sample (see "Getting maximum
# read throughput" above).
print(warm_cache(ds).summary_line())

# Inspect or override:
cache = get_default_shard_cache()
for tier in cache.tiers:
    print(tier.name, tier.max_bytes, tier.supports_direct_read)

# Custom on-prem topology, e.g. for tests.
custom = ShardCacheHierarchy(
    [
        ShardCacheTier("/tmp/fast", max_bytes=10 * 1024**3, name="fast"),
        ShardCacheTier("/tmp/slow", name="slow"),
    ]
)
ds = SampleStream.from_dataset("voice_demo_v1/train", keys=["audio.wav"], cache=custom)

# Cloud topology — pick the tier class that matches the region's cloud.
cloud_azure = ShardCacheHierarchy(
    [
        ShardCacheTier("/mnt/cache/datavo/shards", max_bytes=200 * 1024**3, name="nvme"),
        AzureBlobShardCacheTier(
            container_url="https://datavoeastus.blob.core.windows.net/shards",
            name="region-eastus",
        ),
    ]
)

cloud_aws = ShardCacheHierarchy(
    [
        ShardCacheTier("/mnt/cache/datavo/shards", max_bytes=200 * 1024**3, name="nvme"),
        S3ShardCacheTier(
            bucket="datavo-cache-us-east-1",
            region="us-east-1",
            name="region-us-east-1",
        ),
    ]
)

cloud_gcp = ShardCacheHierarchy(
    [
        ShardCacheTier("/mnt/cache/datavo/shards", max_bytes=200 * 1024**3, name="nvme"),
        GcsShardCacheTier(
            bucket="datavo-cache-us-central1",
            name="region-us-central1",
        ),
    ]
)

The same ShardCacheHierarchy can be reused across multiple SampleStream instances. It carries no per-dataset state.

Importing a cloud tier class does not require the cloud SDK; the SDK is imported lazily the first time the tier hits the wire. That keeps the cost of from datavo_sdk import S3ShardCacheTier to zero when boto3 is not installed.