DataCaches.jl

A lightweight, file-backed key-value cache for Julia workflows that make frequent expensive function calls (remote API queries, long-running computations) and need results available across Julia sessions.

DataCaches selects a transparent, inspectable storage format automatically based on the data type, falling back to Julia's binary serialization for types without a dedicated format:

The format used is recorded per entry in the cache index so the correct deserializer is always selected on read. Custom serializers can be registered for additional types via register_serializer!.

Three levels of caching are provided, from lightest-weight to most manual:

Installation

At the Julia REPL, press ] to enter package manager mode:

pkg> add DataCaches

Quick Start

Cache store

First, create a DataCache. A name or path is always required:

dc = DataCache(:myproject)                               # named, no path management
dc = DataCache(joinpath(homedir(), ".datacaches", "p1")) # explicit path

Caching approaches

using DataCaches

dc = DataCache(:myproject)

# Memoized — works with any function, just wrap the call
set_active_autocache!(dc)
result = @filecache some_expensive_query(; taxon = "Dinosauria")  # fetches + stores
result = @filecache some_expensive_query(; taxon = "Dinosauria")  # from cache
result = @filecache! some_expensive_query(; taxon = "Dinosauria") # force refresh

# Explicit — dict-style, full control over labels
dc["dinosaurs"] = some_expensive_query(; taxon = "Dinosauria")
df = dc["dinosaurs"]

# Automatic — zero call-site changes, but requires instrumented functions
# (see Pattern 3 in the README for setup)
set_autocaching!(true; cache = dc)
result = some_expensive_query(; taxon = "Dinosauria")   # fetches + stores
result = some_expensive_query(; taxon = "Dinosauria")   # from cache, unchanged call
set_autocaching!(false)

Further reading

• API Reference — full DataCaches function and type reference
• Caches — filesystem-style management of cache stores in the scratchspace
• Cache Assets — inspect, copy, move, and remove individual cache entries
• Library Integration — embedding DataCaches.jl in a package