构建于对象存储之上,从缓存提供服务

那些真正到达对象存储的读取十分罕见,而且都是一次性的。它们来自旧分支、恢复操作,或闲置数周后重新唤醒的数据库;而一旦这些数据被缓存,后续的读取又回到本地。
反对在对象存储上运行 Postgres 的常见论点是这样的:
Postgres 需要处理大量小规模、对延迟敏感的读取
对象存储是为更大规模的请求、更高的延迟而设计的,从它那里读取一次可能耗时数百毫秒
因此,由对象存储支撑的 Postgres 必然是缓慢的 Postgres
这个前提没错,但结论不对:它假设构建在对象存储之上的数据库必须从对象存储读取数据才能响应查询,而 Lakebase Postgres 几乎从不这样做。
先看一些数据
为了说明我们所说的"几乎从不"是什么意思,我们查看了生产机群中一部分实例的页面读取统计。在 Lakebase 架构中,读取按层级拆分,每一次页面读取都由第一个持有该页面的层级来响应:
计算节点的本地缓存提供最快的读取,读取过程完全不必离开该计算节点
本地未命中时,读取会跨越网络到达 pageserver,由后者从其本地 NVMe 中提供
只有当 pageserver 本地没有该数据时,读取才会到达对象存储。
计算节点的本地缓存每秒处理约 11 亿次页面访问,约占总量的 99.46%
pageserver 的本地 NVMe 每秒处理约 600 万次读取,约占总量的 0.54%
对象存储每秒仅处理约 39 次请求
Lakebase Postgres 的绝大多数读取都由最快的层级提供。99.46% 的页面访问由计算节点的本地缓存应答,延迟为内存级或微秒级,与传统 Postgres 从热缓存中读取的方式相同。
需要 pageserver 的读取由 NVMe 提供。当页面访问超出计算节点范围时,pageserver 会直接以其本地 NVMe 高效应答这些访问。
需要访问对象存储的读取是罕见且特殊的场景。大约每 2800 万次页面访问中只有 1 次需要从对象存储中取数据,因此超过 99.99999% 的读取无需访问对象存储即可完成。穿透到对象存储的读取通常是长期休眠项目的唤醒、恢复操作,或是过去创建的分支。
构建在对象存储之上并不意味着要从对象存储读取
构建在对象存储之上为以下方面开辟了道路
持久性:像 S3 这样的对象存储设计目标是 99.999999999% 的持久性,因此持久化状态不依赖于任何单台机器
低成本存储且无需预置容量:存储随数据增长,无需规划磁盘容量,也不存在空间耗尽的风险
不可变、可寻址的历史:由于历史以不可变的层文件保存,分支和即时恢复都是元数据操作,无需复制数据
无状态计算:持久化状态存放在计算节点之外,因此计算可以缩容到零,并在不迁移数据的情况下重新启动
事务与分析共用一份副本:一旦运营数据存放在湖中,分析引擎就可以通过 LTAP 读取同一份副本
lakebase 架构
当你构建在对象存储之上时,速度就是你必须在设计中重点考虑的部分。为了让 Postgres 在其上依然保持快速,我们在设计存储层时确保对象存储永远不会挡在查询前面——读取会流经分层结构,将热数据保留在本地缓存中,包括计算节点和 pageserver 上的缓存,而对象存储只为它们做后端支撑。
下面我们仔细看看一次读取如何流经这些分层:
一次读取如何在 lakebase 中流转
lakebase 架构将 Postgres 的计算与存储解耦:
计算层运行标准 Postgres。它将热页保留在计算节点的本地缓存中。它不持有任何持久状态,因此可以启动、停止、扩缩容或发生故障,而不会让数据面临风险。
存储层负责正确性、持久性和历史记录。它由三个组件构成:
Safekeeper 负责复制 WAL。一旦多数 Safekeeper 确认该记录,事务即告提交。基于 Paxos 的协议负责领导者选举和故障切换。
Pageserver 把 WAL 转换成页面。它们在本地 NVMe 上构建 LSM 树,并异步地将物化后的历史持久化到对象存储。
对象存储保存不可变的长期历史数据。它为 Pageserver 提供底层支撑,而 Postgres 从不直接读取它。
这套架构与传统 Postgres 的差异足够大,以至于我们为它单独划出了一个类别:lakebase。Lakebase Postgres 遵循 lakebase 原则。
当 Lakebase Postgres 需要一个页面时,
它首先检查计算节点的本地缓存:如果页面就在那里,这次读取就在计算节点上结束,和任何带有热缓存的 Postgres 一样。这就是每秒约 11 亿次页面访问
本地未命中时,它会向 Pageserver 发送 GetPage@LSN:请求指定一个页面和一个 LSN,Pageserver 返回该页面在该 LSN 时的状态。几乎在所有情况下,Pageserver 自己的 NVMe 上已经有了所需的层(这就是我们之前分享的每秒约 600 万次读取)
只有当 Pageserver 本地没有这些层时,它才会从对象存储中获取。这就是每秒约 39 次请求
在这套设计中,每一层都会缓存自己从下一层获取的数据。Pageserver 返回的页面会落到计算节点的本地缓存中——从对象存储获取的层则落到 Pageserver 的 NVMe 上。更深层的成本只付一次,之后对同一数据的下一次读取都留在本地。
当一次读取离开计算节点:pageserver 内部
那么,正如我们刚刚看到的,离开计算节点的读取只占很小一部分,而且几乎所有这些读取都会止步于 pageserver。
pageserver 将数据组织成两种层文件:
镜像层(image layer)保存某个键范围在一个 LSN 上所有页面的快照
增量层(delta layer)保存某个键范围在一段 LSN 区间内的 WAL 记录
为了响应 GetPage@LSN,pageserver 会:
从请求的页面和 LSN 开始
逐层向下遍历,收集该页面的 WAL 记录(增量)
在页面的第一个镜像处停止
在该镜像之上回放收集到的增量,并返回结果
应用的增量数量衡量了一次读取在 pageserver 上需要多少工作量。增量为零时,页面直接来自镜像;增量很多时,pageserver 必须回放很长的 WAL 链才能返回。
这听起来工作量很大,但实际并非如此。以下数据说明了这一点,覆盖我们在 us-east-1 的机群:
p90 保持在每次 GetPage 不足 1 个增量。这意味着大多数请求直接由镜像提供,完全无需回放
p99 约为 6。这一水平是在我们推出一项更积极地创建基础页面(镜像)的改动后达到的。镜像与读取请求的 LSN 越接近,需要回放长 WAL 链的读取就越少。
2026 年 10 月 13 日至 19 日,us-east-1 每次 GetPage 读取所应用的增量数。p90 略低于 1;p99 下降并稳定在 6 左右
最不常见的路径:那些到达对象存储的未命中
到达 pageserver 的读取请求中,只有极小一部分会再往前走一步。每秒约 39 个抵达对象存储的请求有一个共同点——它们索取的都是近期无人读取过的数据。实际上,这几乎每次都落在以下两种情况之一:
在旧时间点做分支或恢复。Lakebase 的一个分支是指向某个 LSN 的指针。时间点恢复与快照恢复使用同一机制:它们在该 LSN 处创建一个新分支,而不是重写当前分支。当恢复点距今很久远时,那段历史可能并不在 pageserver 的 NVMe 上,因此首批读取会从对象存储获取。
长时间休眠后唤醒的非活跃数据库。一个已经缩容至零达数周的 Lakebase 数据库不再缓存其数据,因此唤醒后的首批读取会走对象存储。
值得注意的是,即便在这两种情况下,也只有首批读取需要付出这份代价。此后,数据会被缓存在最快的存储层级中,读取走的路径与任何活跃工作负载的本地路径相同。在活跃查询执行中,对象存储根本不在 Lakebase 的关键路径上。
构建在对象存储之上的 Postgres 不必为了回答查询而从对象存储读取。在 Lakebase 中,一次读取会先到计算节点的本地缓存,再到 pageserver,而每一层都会缓存它从下一层取到的数据。
Image layers 让 pageserver 的重放链保持简短。这些机制共同使对象存储远离读取路径——超过 99.99999% 的页面访问由计算节点和 pageserver 上的本地缓存服务,这正是 Lakebase Postgres 在保持快速的同时获得对象存储优势的原因。
在收件箱中获取最新文章
译文由英文原文机器翻译,术语按行业惯例统一,可在文末展开原文核对。
查看英文原文(Built on object storage, served from cache)
The rare reads that do reach object storage are one-offs. They come from old branches, restores, or databases waking up after weeks idle, and once that data is cached, the next reads are local again.
The common argument against running Postgres on object storage goes like this:
Postgres processes many small, latency-sensitive reads
Object storage is built for larger requests at higher latency, and a single read from it can take hundreds of milliseconds
Therefore, a Postgres backed by object storage must be a slow Postgres
The premise is right, but the conclusion isn't: it assumes that a database built on object storage has to read from object storage to answer queries, and Lakebase Postgres almost never does.
Let's start with some numbers
To show what we mean by "almost never," we looked at page read stats across a slice of our production fleet. In the lakebase architecture, reads are split into tiers, and every page read is served by the first tier that has the page:
The compute's local cache serves the fastest reads, without the read ever leaving the compute node
On a local miss, the read crosses the network to a pageserver, which serves it from its own local NVMe
Only if the pageserver doesn't have the data locally does the read reach object storage.
The compute's local cache served ~1.1 billion page accesses per second, about 99.46% of the total
Pageserver local NVMe served ~6 million reads per second, about 0.54% of the total
Object storage served only ~39 requests per second

The vast majority of Lakebase Postgres reads are served by the fastest tier. 99.46% of page accesses are answered from the compute node's local cache, at memory or microsecond-level latency, the same way a traditional Postgres reads from a warm cache.
Reads that need the pageserver are served from NVMe. When page accesses extend beyond the compute node, the pageserver efficiently answers them directly from its own local NVMe.
Reads that need object storage are rare and special case. Only about 1 in 28 million page accesses needs a fetch from object storage, so more than 99.99999% are served without it. Reads that pass through to object storage are typically wakeups of long dormant projects, restores, or branches created in the past.
Building on object storage doesn't mean reading from it
Building on object storage opens the path for
Durability: object storage like S3 is designed for 99.999999999% durability, so durable state doesn't depend on any single machine
Cheap storage with no provisioning: storage grows with the data, with no disks to size and no risk of running out of space
Immutable, addressable history: because history is kept as immutable layer files, branching and instant restores are metadata operations and don't need data copies
Stateless compute: durable state lives outside the compute node, so compute can scale to zero and start again without moving data
One copy for transactions and analytics: once operational data lives in the lake, analytical engines can read the same copy through LTAP
The lakebase architecture
When you build on object storage, speed is the part you have to design around. To keep Postgres fast on top of it, we designed the storage layer so that object storage never sits in front of a query - reads go through tiers that keep hot data in local caches, on the compute node and on the pageservers, and object storage only backs them.
Here's a closer look at how a read moves through those tiers:
How a read moves through the lakebase
The lakebase architecture decouples compute and storage for Postgres:
The compute layer runs standard Postgres. It keeps hot pages in a local cache on the compute node. It holds no durable state, so it can start, stop, scale, or fail without putting data at risk.
The storage layer owns correctness, durability, and history. It is built from three components:
Safekeepers replicate the WAL. A transaction is committed once a quorum of safekeepers acknowledges the record. A Paxos-based protocol handles leader election and failover.
Pageservers turn WAL into pages. They build an LSM tree on local NVMe and persist materialized history to object storage asynchronously.
Object storage holds the immutable, long-term history. It backs the pageservers, and Postgres never reads from it directly.
This architecture differs enough from conventional Postgres that we've given it its own category: the lakebase. Lakebase Postgres follows the lakebase principles.
When Lakebase Postgres needs a page,
It first checks the compute's local cache: if the page is there, the read ends on the compute node, as in any Postgres with a warm cache. That's the ~1.1 billion page accesses per second
On a local miss, it sends GetPage@LSN to a pageserver: the request names a page and an LSN, and the pageserver returns the page as it existed at that LSN. In almost every case, the pageserver already has the layers it needs on its own NVMe (that's the ~6 million reads per second we shared earlier)
Only if the pageserver doesn't have those layers locally does it fetch them from object storage. That's the ~39 requests per second
In this design, every tier caches what it fetches from the tier below. A page returned by the pageserver lands in the compute's local cache - layers fetched from object storage land on pageserver NVMe. The cost of a deeper tier is paid once, and the next read of the same data stays local.
When a read leaves the compute node: inside the pageserver
So, as we just saw, reads that leave the compute node are a small minority, and almost all of them stop at the pageserver.
The pageserver organizes data into two kinds of layer files:
An image layer holds a snapshot of every page in a key range at one LSN
A delta layer holds the WAL records for a key range across a range of LSNs
To answer GetPage@LSN, the pageserver:
Starts at the requested page and LSN
Walks down through the layers, collecting the WAL records (deltas) for that page
Stops at the first image of the page
Replays the collected deltas on top of that image and returns the result
The number of deltas applied is the measure of how much work a read takes on the pageserver. With zero deltas, the page comes straight from an image. With many, the pageserver has to replay a long chain of WAL before it can return.
This sounds like a lot of work, but it is not so much in practice. Here's data that illustrates this, across our us-east-1 fleet:
p90 stayed under 1 delta per GetPage. This means most requests are served straight from an image, with no replay at all
p99 was around 6. This dropped to that level after we rolled out a change that creates base pages (images) more eagerly. With images closer to the LSNs that reads ask for, fewer reads have a long chain of WAL to replay.
Deltas applied per GetPage read across us-east-1, Oct 13 to 19, 2026. p90 stays just under one; p99 drops and settles at around 6
The least frequent path: misses that reach object storage
Only a tiny fraction of the reads that reach a pageserver go one step further. The ~39 requests per second that reach object storage have one thing in common - they ask for data that nobody has read recently. In practice, this means one of these two cases practically every time:
Branches or restores at old points in time. A Lakebase branch is a pointer to an LSN. Point-in-time restore and snapshot restore use the same mechanism: they create a new branch at that LSN rather than rewriting the current one. When the restore point is far in the past, that history may not be on pageserver NVMe, so the first reads fetch it from object storage.
Inactive databases waking up after a long time sleeping. A Lakebase database that has scaled to zero for weeks no longer has its data cached, so its first reads after waking up go to object storage.
It's important to note how even in both these cases, only the first reads pay the cost. After that, the data is cached in the fastest tiers, and reads follow the same local path as any active workload. In active query execution, object storage is simply not in the critical path in the lakebase.
A Postgres built on object storage doesn't have to read from object storage to answer queries. In Lakebase, a read goes to the compute's local cache, then the pageserver, and every tier caches what it fetches from the one below.
Image layers keep the pageserver's replay chains short. Together, these keep object storage off the read path - more than 99.99999% of page accesses are served from local caches on compute and pageservers, which is how Lakebase Postgres gets the benefits of object storage while staying fast.
Get the latest posts in your inbox
原文为英文,正文由自动抓取并保留原文。
原文发布于 Databricks Blog(Built on object storage, served from cache),标题、摘要与正文由机器翻译,术语按行业惯例统一,未经人工逐句校订。