Cloudflare R2 SQL Architecture Diagram: Distributed Iceberg Query Workers

The September 25, 2025 post R2 SQL: a deep dive into our new distributed query engine is the architecture source: a planner prunes Apache Iceberg metadata, then workers scan Parquet in R2. The Basin SQL docs, updated October 1, 2026, say Basin SQL, formerly R2 SQL, is generally available, and that old R2 SQL APIs still work and will be deprecated. That docs page does not restate DataFusion, Arrow, or the prune steps.

What this Cloudflare Basin SQL distributed query architecture diagram shows

A SQL request hits a coordinator that plans and merges. The planner walks Iceberg metadata and emits Parquet row groups, not whole tables. Workers run SQL on those ranges and return Apache Arrow over gRPC. The deep dive’s opening says the work spreads across “Workers and R2.” The execution section then calls the executors servers chosen through an internal API. Both sentences are in that post. The post uses both names, and the diagram does not collapse them into one label.

The problem this architecture is solving

The deep dive says one logical table can be millions of files, so a full scan is not viable, and compute should exist only for the query. R2 SQL there is retrieval SQL on Iceberg without standing up Spark or Trino. “Results in seconds” and “petabytes” are that post’s claims. The October 2026 docs page does not repeat them. It shows wrangler basin sql query against a warehouse and a table such as default.transactions.

Main components and trust boundaries

Iceberg gives the planner snapshots and statistics instead of a rewrite of the data files. The deep dive defines two stat levels. Partition stats live in the manifest list: for a day(event_timestamp) partition, the earliest and latest day in that manifest. Column stats live in the manifest for each Parquet file: minimum, maximum, and a null count. A file whose http_status spans 200 to 404 can be skipped for http_status = 500. A file whose column is entirely null can be skipped for IS NOT NULL. Row-group stats sit in the Parquet footer and let the planner skip groups inside a file that already survived.

The coordinator streams a unit as soon as it exists and picks healthy servers from an internal API. Links to workers use Argo Smart Routing, stated as the connectivity path, not as a percentile. Each worker gets row groups, the SQL, and Parquet metadata with byte offsets so it need not rediscover them from R2. Apache DataFusion runs the scan. The Sept 25 2025 post says a DataFusion in-memory partition and an Iceberg partition differ in purpose and do not always correspond, and that one row group is one DataFusion partition. It says row groups usually contain at least 1000 rows. That is one sentence, not a guarantee for every file.

Results use Arrow in memory and Arrow IPC inside the gRPC response. The coordinator deserializes back to Arrow arrays and aggregates. Filter pushdown and column projection are DataFusion behaviors the post claims: ranged R2 reads fetch only the columns the query names, and filters apply as values are read.

Request or data path, step by step

The planner fetches current table metadata, a JSON file of schema, partition spec, and snapshot log, then the latest snapshot. That snapshot points at one manifest list. Partition stats drop manifests whose ranges cannot match. Remaining manifests yield Parquet files, and column stats drop files. Footer stats drop row groups. Matching row groups become the queue.

Planning streams. The post says waiting for a full plan on millions of files would add too much latency, so units leave while later manifests are still open. Order follows ORDER BY using stats. For ORDER BY timestamp DESC LIMIT 5, a five-row heap stops the scan once its oldest timestamp is newer than any unread file’s high-water mark. As of September 25, 2025, that ordering works only for partition-key columns. The GA docs page does not say whether the limit remains.

The diagram: labeled boxes

One solid chain: SQL, coordinator and planner, Iceberg snapshot, manifest list (partition prune), manifests (column prune), Parquet footer (row-group prune), work units, DataFusion query workers, ranged reads on R2, Arrow IPC/gRPC, then merge. There is no separate client box, no catalog-metadata box, no dashed prune arrow, no split to many workers, no return edge to the coordinator, and no stop line.

• Prune before read: manifest, file, then row group. A skipped object has no R2 read.
• Unhealthy server: the coordinator asks an internal API and does not send work to a server in maintenance. The post does not describe a retry policy beyond that selection.
• Early stop: only the ordered, partition-key case in the 2025 post. It is not general aggregation on this diagram.
• Name split: the intro says Workers and the execution section says servers. The diagram keeps both names.

What the source does not claim (preview, case study, or limits)

The docs page calls Basin SQL generally available in its October 1, 2026 update. The deep dive’s future list — distributed aggregations, visibility tools, more Iceberg options, dashboard queries, full-text and geospatial indexes — is not GA just because the name is. The docs page does not confirm that DataFusion and gRPC survived the rename. No price or concurrency quota is stated. “Seconds” is the deep dive’s wording only.

How this differs from a nearby pattern on ByteDiagram

The Basin platform diagram is Pipelines into Catalog on R2, with SQL as one tier of the product. This diagram is only the query: metadata prune, work units, workers, merge. Logpush, transforms, and snapshot maintenance belong to the Basin platform picture, not this query-worker diagram. Use the platform picture when the review is the whole lake, and this one when the review is how a single statement avoids reading every Parquet file.

FAQ

What did Cloudflare rename R2 SQL to, on the pages cited here?

The Basin SQL docs, updated October 1, 2026, say Basin SQL was formerly R2 SQL, is generally available, and that existing R2 SQL APIs continue to work and will be deprecated later. The September 25, 2025 architecture post still uses the R2 SQL name and does not mention Basin.

Which Iceberg structures does the deep dive say the planner can skip?

Partition stats in the manifest list, column min, max, and null counts in manifests, and row-group statistics in Parquet footers. Surviving row groups are the work units. The October 2026 docs page does not restate this prune order.

Does the deep dive say query executors are Cloudflare Workers or servers?

It says both, in different sections. The introduction says work is distributed across Workers and R2. The execution section says the coordinator picks healthy servers through an internal API, and those servers run as query workers using DataFusion, with Arrow IPC over gRPC. The docs page fetched here names neither.

Conclusion

The picture is one solid chain: the coordinator and planner, Iceberg prune from snapshot to row group, work units, DataFusion query workers, ranged R2 reads, and Arrow IPC/gRPC into a merge. Nothing returns to the coordinator. Label the engine R2 SQL in the 2025 deep dive and Basin SQL, formerly R2 SQL, on the October 1, 2026 docs page. Workers versus servers, and the partition-key ORDER BY limit, are not settled by the GA note. Sources: the R2 SQL deep dive and the Basin SQL docs. More diagrams are on the ByteDiagram blog.