← All pins

Pins tagged “databases”

DuckDB Internals: Why is DuckDB Fast?

A practical walk through DuckDB internals, starting with why running in process avoids client protocol overhead and then following SQL through parsing, binding, optimization, physical planning, pipelines, and storage.

There is also a second part from the series which goes further into DuckDB's vectorized execution engine, precompiled inner loops, SIMD, and push based pipelines.

databasesduckdbperformance

Offloading I/O to Dedicated Cores: An Asymmetric io_uring Backend for Seastar and ScyllaDB

An engineering writeup from the ScyllaDB team on reworking Seastar's io_uring backend so application shards stop spending their own CPU cycles on I/O. Instead of every shard issuing syscalls, shards are grouped and each group hands its asynchronous I/O to a dedicated networking core; the chosen design gives each shard a private io_uring instance while sharing kernel worker pools through IORING_SETUP_ATTACH_WQ, with a single SQPOLL thread polling the group's submission queues so nothing blocks on the compute cores. The post is candid about the tradeoffs, walking through three architectures the team weighed and measuring where the asymmetric backend actually wins on compute heavy request handling versus where one networking core becomes the bottleneck on copy_to_user for pure I/O workloads.

networkingio_uringdatabases

Aurora DSQL: Scalable, Multi-Region OLTP

A paper from the AWS team describing the design of Aurora DSQL, a distributed SQL database that aims to scale OLTP workloads across regions without giving up strong consistency. It explains how the system decouples the transaction path from storage, uses quorum based replication and a coordination layer to preserve ACID semantics globally, and keeps latency reasonable by letting regions operate with local autonomy while still agreeing on a consistent order. The evaluation is the interesting part, with measurements across different transaction types and region configurations that show where the tradeoffs actually land rather than just asserting they work.

databasesdistributed-systemspapers

Introducing Meerkat: an experiment in global consensus

Cloudflare Research's writeup on Meerkat, an experimental consensus service that keeps control-plane state strongly consistent across their 330+ data centers. The interesting part is that it drops the leader entirely: instead of Raft-style leader election through timeouts, which the post argues are hard to tune on the wide-area internet and have caused availability incidents, Meerkat builds on the QuePaxa algorithm where any replica can accept writes at any time and concurrent proposals interfere constructively rather than fighting. It maintains a linearizable log split into slots, and this is the first industrial deployment of QuePaxa at global scale. Worth reading if you care about how consensus actually behaves across unpredictable wide-area links rather than in a single datacenter.

distributed-systemsconsensusdatabases

A Decade of Dynamo: Powering the next wave of high-performance apps

A retrospective USENIX ATC paper on how DynamoDB evolved from the original Dynamo design into a fully managed service. It's a rare, candid look at the operational lessons behind a system running at enormous scale: predictable performance, admission control, durability, and the trade-offs made to keep tail latencies flat. Good reading if you care about how distributed storage behaves in production rather than on paper.

distributed-systemsdatabasespapers