15. Shard caching and the cache hierarchy

Runnable companion: examples/12_warm_and_read_cache.py

Every shard datavo streams is content-addressed — named by a content_hash over its bytes. So the first time you read a dataset or workset, its shards are fetched from the server and dropped into a local cache tier; the second time — same process, a new process, or another machine sharing a tier — they are served from local disk and the server is never touched. That is what makes multi-epoch training and re-opening a frozen dataset cheap. You do not turn it on: every read in chapters 8–14 already went through it.

Quick start (local)

On your own machine there is nothing to configure. The first read populates a default cache under ~/.cache/altavo/datavo/shards — a single folder tier, 50 GiB, least-recently-used eviction — and every later read of the same bytes is a local hit:

from datavo_sdk import SampleStream

# First pass: shards fetched from the server and cached.
for sample in SampleStream.from_dataset("voice_demo_v1/train", keys=["audio.wav"]):
    ...

# Second pass, even in a new process: served from local disk, no server traffic.
for sample in SampleStream.from_dataset("voice_demo_v1/train", keys=["audio.wav"]):
    ...

Three levers cover almost every local need:

You want	Do this
The cache on a bigger disk	set DATAVO_SHARD_CACHE_DIR=/data/datavo-cache
To pre-fill it before a run	datavo warm <dataset> --key audio.wav
A specific size or a multi-tier layout	drop a shards.yaml (below)

A minimal shards.yaml — looked up at ~/.config/datavo/shards.yaml, then /etc/datavo/shards.yaml — is just one folder tier:

tiers:
  - type: folder
    path: /data/datavo-cache
    max_bytes: 200GB

That is all a single user needs. Cloud tiers, shared-workstation layouts, and the full schema live in the shard cache reference.

Tiers and the hierarchy

A cache is a hierarchy: an ordered list of tiers, fastest first. A tier is either a folder tier (local disk — the required landing zone every hit is materialized into) or a cloud tier (Azure Blob, S3, GCS — a shared, read-through layer many machines populate once). A single laptop has one folder tier; a shared cluster puts a fast local folder tier in front of a cloud tier, so the first machine to need a shard pulls it from the server and every other machine pulls it from the bucket.

A read walks the hierarchy by content hash:

• Hit on the fastest tier → returned straight away.
• Hit on a slower tier → served, and promoted into the faster tiers above it, so the next read hits the top.
• Miss everywhere → fetched from the server once and populated into every tier that can hold it.

The hierarchy is self-warming from both directions — you never pre-fill it by hand, though datavo warm lets you do it deliberately before a job.

from datavo_sdk import get_default_shard_cache

cache = get_default_shard_cache()          # the hierarchy a read would use
for tier in cache.tiers:                    # fastest first
    print(tier.name, tier.supports_direct_read, tier.max_bytes)

Eviction is per tier, and it is not retention

A folder tier is bounded by max_bytes and evicts least-recently-used by file mtime: a read touches a shard, so hot shards stay and cold ones fall out when space is needed. A cloud tier's max_bytes is informational — you let the bucket's own lifecycle rules reclaim it.

This is cache eviction — dropping a local copy you can always refetch. It is not the server-side deletion of the origin bytes, which is a separate, deliberate act with its own rules (lifecycle and retention). Evicting a cached shard loses nothing; deleting a store's blobs is what loses data.

Instances never share a key

The on-disk key is only the server's content_hash, and nothing guarantees that value is unique across datavo servers. So each tier is scoped by the instance it serves — dev and prod land in sibling subdirectories under the same tier, and a du -sh or rm -rf on one leaves the other alone. You get this for free; it is why a cache built against dev.datavo.io never serves a shard to a client talking to prod.

Pitfalls

• Expecting a warm cache after changing the requested keys. A key group is the cache unit (chapter 8): a run that fetched only inputs has not cached features. Warm what you will read.
• A cache tier smaller than one shard. A folder tier whose max_bytes is below a single shard can hold nothing, so every read misses; size the tier above your shard size.
• Treating eviction as data loss. An evicted shard refetches on the next read. Only origin-blob deletion (retention) removes data.

---

That is the whole guide. From here, the reference has the exhaustive surfaces — the shard cache schema and deployments, the CLI, idempotency, and retention — and the internals trace the read path tier by tier.