Cache expiration and invalidation

DataCaches provides first-class support for cache invalidation: time-to-live (TTL), stale detection, bulk invalidation, fine-grained purging, and automatic post-write purge policies.

TTL and staleness

Per-entry TTL

Pass a ttl keyword to write! to record a time-to-live for an entry. The value must be a Dates.Period (e.g. Dates.Hour(6), Dates.Day(30)).

using DataCaches, Dates

dc = DataCache(:myproject)

# Cache a result with a 6-hour TTL
write!(dc, result; label = "query1", ttl = Hour(6))

# Check staleness before re-using
if isstale(dc, "query1")
    result = fetch_live()
    write!(dc, result; label = "query1", ttl = Hour(6))
end
data = read(dc, "query1")

read does not enforce staleness — stale entries are readable until explicitly removed. This enables a stale-while-revalidate pattern: serve from the cache while refreshing in the background.

Cache-level default TTL

Set a default TTL on the cache itself so every write! inherits it automatically:

# All writes to this cache expire after 24 hours unless overridden per-entry
dc = DataCache(:myproject; default_ttl = Day(1))

write!(dc, result; label = "query1")          # inherits Day(1)
write!(dc, result; label = "query2", ttl = Hour(1))  # overrides to Hour(1)

The default_ttl is persisted in the cache's TOML index, so it is remembered when the cache is reloaded in a future session.

`isstale`

isstale checks whether an entry has exceeded its TTL:

isstale(dc, "query1")           # by label
isstale(dc, entry)              # by CacheEntry
isstale("query1")               # uses active_autocache()
isstale(entry)                  # uses active_autocache()

Returns false when no TTL applies (neither per-entry nor cache default). Legacy entries whose datecached is unknown are never considered stale.

Stale-while-revalidate pattern

function get_data(dc, label, fetch_fn; ttl = Hour(6))
    # Return cached value even if stale; trigger background refresh when stale
    if haskey(dc, label) && isstale(dc, label)
        @async begin
            result = fetch_fn()
            write!(dc, result; label = label, ttl = ttl)
        end
    elseif !haskey(dc, label)
        result = fetch_fn()
        write!(dc, result; label = label, ttl = ttl)
    end
    return read(dc, label)
end

`invalidate!` — bulk invalidation

invalidate! removes entries matching a set of criteria in a single batched index rewrite. When called without a cache argument, uses active_autocache().

By staleness

# Remove all entries that have exceeded their TTL
invalidate!(dc; stale = true)
invalidate!(; stale = true)    # default cache

By label pattern

# Remove entries whose label/description matches a pattern
invalidate!(dc; pattern = r"^temp_")
invalidate!(dc; pattern = "old_run_")          # String is converted to Regex

By date

# Remove entries cached more than 30 days ago
invalidate!(dc; before = now() - Day(30))

# Remove entries from a specific window
invalidate!(dc; after  = DateTime("2026-01-01T00:00:00"),
                before = DateTime("2026-02-01T00:00:00"))

By file format

# Remove all JLS (binary serialization) entries
invalidate!(dc; format = "jls")
invalidate!(dc; format = r"jls|json")   # multiple formats via Regex

By predicate function

A predicate function receives a CacheEntry and returns true if the entry should be removed:

# Remove entries whose description starts with a particular function name
invalidate!(dc; predicate = e -> startswith(e.description, "my_func("))

# Remove large files
invalidate!(dc; predicate = e -> isfile(e.path) && stat(e.path).size > 50_000_000)

Combining criteria

All filter kwargs (same as entries / CacheAssets.ls) are accepted first to scope the candidate set, then stale and predicate are applied as additional restrictions:

# Remove stale, labeled-only entries cached before a date
invalidate!(dc; labeled = true, before = now() - Day(7), stale = true)

Dry-run preview

invalidate!(dc; stale = true, dry_run = true)
# Prints: "invalidate! (dry run): N entries would be removed: …"
# No entries are deleted.

`CacheAssets.purge!` — power tool

DataCaches.CacheAssets.purge! is the full-featured purge function with LRU eviction, size limits, and all the filter kwargs accepted by CacheAssets.ls.

Remove by age

using DataCaches.CacheAssets

# Remove entries older than 30 days
purge!(dc; max_age = Day(30))

Remove by idle time (LRU)

# Remove entries not accessed in the last 7 days
purge!(dc; max_idle = Day(7))

Note

max_idle requires track_access = true (the default) on the cache. Entries that have never been accessed are treated as most-idle.

Keep only the N most recently accessed

# Keep the 20 most recently accessed entries; delete the rest
purge!(dc; keep_count = 20)

Limit total cache size

# Purge LRU entries until total cache size is under 500 MiB
purge!(dc; max_size_bytes = 500 * 1024 * 1024)

Protect labeled entries

# Purge only unlabeled entries older than a week; leave labeled entries alone
purge!(dc; max_age = Day(7), keep_labeled = true)

Combine with format or pattern filters

# Delete JLS entries not accessed in the last 30 days
purge!(dc; format = "jls", max_idle = Day(30))

# Delete entries matching a label pattern that are older than a month
purge!(dc; pattern = r"^temp_", max_age = Day(30))

Dry-run

purge!(dc; max_age = Day(7), dry_run = true)
# Prints: "purge! (dry run): N entries would be removed: …"
# No entries are deleted.

Remove all stale entries via purge!

purge!(dc; stale = true)

Auto-purge policies

set_autopurge! attaches a PurgePolicy to a cache. The policy runs automatically at the end of every write!, keeping the cache within configured limits without manual intervention.

LRU eviction

# Keep only the 50 most recently accessed entries
set_autopurge!(dc; keep_count = 50)

# Write a new result — oldest entries are pruned automatically
write!(dc, new_result; label = "latest")

Age-based cleanup

# Entries older than 14 days are automatically removed on each write
set_autopurge!(dc; max_age = Day(14))

Size-based cleanup

# Cache stays under 1 GiB; LRU entries are pruned when the limit is exceeded
set_autopurge!(dc; max_size_bytes = 1024^3)

Combined policy

# Remove entries older than 30 days OR keep only the last 100,
# but never delete labeled entries
set_autopurge!(dc; max_age = Day(30), keep_count = 100, keep_labeled = true)

Disabling auto-purge

set_autopurge!(dc; enabled = false)

Using the autocache store

All functions accept a zero-argument form that operates on active_autocache():

set_autopurge!(; keep_count = 20)
invalidate!(; stale = true)
CacheAssets.purge!(; max_age = Day(30))

Quick-reference table

Function	Description
`write!(dc, data; ttl = Hour(6))`	Write with per-entry TTL
`DataCache(store; default_ttl = Day(1))`	Cache-level default TTL
`isstale(dc, label)` / `isstale(entry)`	Check if entry is past TTL
`invalidate!(dc; stale, pattern, format, before, after, labeled, predicate, dry_run)`	Bulk removal by criteria
`CacheAssets.purge!(dc; max_age, max_idle, keep_count, max_size_bytes, stale, keep_labeled, dry_run, …ls filters…)`	Power tool with LRU/size limits
`set_autopurge!(dc; max_age, max_idle, keep_count, max_size_bytes, keep_labeled, enabled)`	Configure automatic post-write purging

API reference

DataCaches.isstale — Function

isstale(cache::DataCache, entry::CacheEntry) → Bool
isstale(cache::DataCache, label::AbstractString) → Bool
isstale(entry::CacheEntry) → Bool
isstale(label::AbstractString) → Bool

Return true if entry (or the entry identified by label) has exceeded its time-to-live and is considered stale.

The effective TTL for an entry is its own entry.ttl if set, otherwise the cache's default_ttl. When no TTL applies (both are nothing), returns false.

Staleness is purely informational — read does not enforce it. The caller decides whether to refresh (enabling stale-while-revalidate patterns).

Examples

dc = DataCache(:myproject; default_ttl = Dates.Hour(1))
write!(dc, result; label = "query1")

isstale(dc, "query1")   # false immediately after write
# … one hour later …
isstale(dc, "query1")   # true

# Per-entry TTL overrides the cache default:
write!(dc, result2; label = "short", ttl = Dates.Minute(5))
isstale(dc, "short")    # true after 5 minutes

source

DataCaches.invalidate! — Function

invalidate!(cache::DataCache; kwargs...) → DataCache
invalidate!(; kwargs...) → DataCache

Remove entries from cache that match the given criteria. All removals are batched into a single index rewrite.

When called without a cache argument, uses active_autocache().

Filtering keyword arguments

Entries are first selected by the standard filter set (same as entries):

pattern — Regex or String; matched against label, description, or UUID
filepath_pattern — Regex or String; matched against full file path
filename_pattern — Regex or String; matched against filename only
format — String or Regex; matched against the entry's format tag ("csv", "jls", …)
before::DateTime — entries cached before this time
after::DateTime — entries cached after this time
accessed_before_date::DateTime — entries last accessed before this time
accessed_after_date::DateTime — entries last accessed after this time
labeled::Union{Nothing,Bool} — true = labeled only; false = unlabeled only

Invalidation criteria

stale::Bool = false — further restrict to entries that are past their TTL
predicate::Union{Nothing,Function} = nothing — further restrict to entries where predicate(entry) returns true

Options

dry_run::Bool = false — print matching entries to io without deleting
io::IO = stdout — output stream for dry_run messages

Examples

dc = DataCache(:myproject; default_ttl = Dates.Hour(24))

# Remove all stale entries
invalidate!(dc; stale = true)

# Remove entries matching a label pattern
invalidate!(dc; pattern = r"^temp_")

# Remove CSV entries older than a week
invalidate!(dc; format = "csv", before = Dates.now() - Dates.Day(7))

# Remove entries matching a custom predicate
invalidate!(dc; predicate = e -> startswith(e.description, "my_func("))

# Preview what would be removed
invalidate!(dc; stale = true, dry_run = true)

source

DataCaches.set_autopurge! — Function

set_autopurge!(cache::DataCache; enabled=true, kwargs...) → DataCache
set_autopurge!(; enabled=true, kwargs...) → DataCache

Configure an automatic purge policy on cache. After each write!, the policy is applied to keep the cache within the specified limits.

When called without a cache argument, uses active_autocache().

Pass enabled = false to remove the current policy (disabling auto-purge).

Keyword arguments

max_age::Union{Nothing,Period} — remove entries older than this age (by datecached)
max_idle::Union{Nothing,Period} — remove entries not accessed within this period
keep_count::Union{Nothing,Int} — keep only the N most-recently-accessed entries
max_size_bytes::Union{Nothing,Int} — purge LRU entries until total cache size ≤ limit
keep_labeled::Bool = false — when true, labeled entries are never auto-purged

Examples

dc = DataCache(:myproject)

# Keep only the 20 most recently accessed entries
set_autopurge!(dc; keep_count = 20)

# Remove entries older than 30 days, never delete labeled entries
set_autopurge!(dc; max_age = Dates.Day(30), keep_labeled = true)

# Disable auto-purge
set_autopurge!(dc; enabled = false)

source

DataCaches.CacheAssets.purge! — Function

DataCaches.CacheAssets.purge!([cache::DataCache]; kwargs...) → DataCache

Bulk-delete cache entries matching the given criteria. All standard ls filtering keyword arguments are accepted to scope which entries are candidates for purging. When cache is omitted, the active default cache is used (see active_autocache()).

All removals are batched into a single index rewrite.

Standard filter kwargs (same as ls)

pattern, filepath_pattern, filename_pattern — label/path/filename pattern
format::Union{Nothing,String,Regex} — format tag filter (e.g. "jls", r"csv|json")
before, after — filter by datecached
accessed_before_date, accessed_after_date — filter by dateaccessed
labeled::Union{Nothing,Bool} — true = labeled only; false = unlabeled only

Purge criteria (applied after filtering)

stale::Bool = false — delete entries past their TTL (requires TTL to be configured)
max_age::Union{Nothing,Period} — delete entries whose age (by datecached) exceeds this
max_idle::Union{Nothing,Period} — delete entries that have not been accessed within this period (entries never accessed are treated as least-recently-used and are eligible)
keep_count::Union{Nothing,Int} — keep only the N most-recently-accessed entries among the candidates; delete the rest
max_size_bytes::Union{Nothing,Int} — purge LRU entries until the total size of the remaining candidates is at or below this limit

Options

keep_labeled::Bool = false — when true, labeled entries are never deleted, even if they match all other criteria
dry_run::Bool = false — print what would be deleted to io without deleting
io::IO = stdout — output stream for dry_run messages

Examples

dc = DataCache(:myproject)

# Remove all stale entries (requires TTL on entries or cache)
CacheAssets.purge!(dc; stale = true)

# Remove entries older than 30 days
CacheAssets.purge!(dc; max_age = Dates.Day(30))

# Keep only the 10 most recently accessed entries
CacheAssets.purge!(dc; keep_count = 10)

# Purge LRU entries until cache is under 100 MiB, keep labeled entries
CacheAssets.purge!(dc; max_size_bytes = 100 * 1024 * 1024, keep_labeled = true)

# Preview without deleting
CacheAssets.purge!(dc; max_age = Dates.Day(7), dry_run = true)

# Purge using default cache
CacheAssets.purge!(; keep_count = 50)

source