"Cached" means three different things
The word "cached" gets used loosely, but Databricks actually maintains three independent caching layers with different scopes and different rules:
Result cache
The exact output of a previous query. A local copy lasts for the cluster's lifetime (cleared on stop/restart); on serverless SQL warehouses, a remote copy also lasts 24 hours and survives a warehouse stop and restart.
Disk cache
Raw data files already read from remote storage, held on the worker's local SSD in a faster format so later reads skip remote storage. Formerly called the Delta cache and the DBIO cache. Shares the local result cache's lifecycle.
UI cache
Per-user caching of query results and SQL editor visualizations in the Databricks SQL UI, with at most a 7-day lifecycle. Doesn't apply to AI/BI dashboards, which have their own separate caching.
Which cache actually should have helped?
Answer three questions about the run
Walk through the same query's two runs and see which cache was actually eligible to help.
1. Was the cluster or SQL warehouse stopped or restarted between the two runs?
2. Has the underlying table been written to (inserted, updated, deleted, or overwritten) since the first run?
3. Is it a serverless SQL warehouse?
Why the result cache didn't apply
The local result cache lives for the lifetime of the cluster that produced it and is cleared the moment that cluster is stopped or restarted -- including an automatic stop after the configured idle timeout. If the "same" query ran an hour ago but the cluster stopped in between, the second run starts from scratch and has to re-execute fully, regardless of how identical the query text is. The disk cache is cleared on the exact same schedule, so the first run after a restart also loses the local file-level caching benefit it would otherwise have gotten.
Why the remote result cache specifically didn't apply
Databricks documents the remote result cache as serverless-only. A classic or Pro SQL warehouse has no equivalent -- it only gets the local, per-cluster result cache and disk cache, both of which disappear on stop or restart. So two runs of the exact same query on a classic or Pro warehouse, separated by a restart, get no caching benefit from either layer at all. On a serverless warehouse, the same two runs can still hit the remote result cache, which survives the stop, because it isn't tied to any one cluster's lifecycle in the first place.
When it's not a caching bug at all
If the underlying table was written to between the two runs, Databricks correctly invalidates the cached result -- that's the cache doing its job, not failing at it. The disk cache behaves the same way: it's documented as automatically detecting changes to the underlying data files and invalidating itself when it finds them. Re-running the full query in either case is the correct behavior, and "why didn't this stay cached" isn't the right question to ask about it.
Key takeaways
- Result cache, disk cache, and UI cache are three separate systems with three separate lifecycles -- "it's cached" needs to specify which one.
- A cluster or warehouse stop clears the local result cache and the disk cache on the same schedule -- only the serverless-only remote result cache survives it.
- Classic and Pro warehouses never get the remote result cache's 24-hour, restart-surviving benefit -- only serverless does.
- A query hitting one cache layer says nothing about whether the other two also applied -- diagnose them independently.
Diagnostic checklist
Check from_result_cache in system.query.history to see directly whether a statement's result came from the cache, and cache_origin_statement_id to trace it back to the query that first populated that entry. Check read_io_cache_percent separately to measure disk-cache benefit on the file-read side -- a query can show 0% there while from_result_cache is still TRUE, because a full result-cache hit never reads data files at all. The download below runs both checks, grouped by warehouse.