Pins

A practical walk through DuckDB internals, starting with why running in process avoids client protocol overhead and then following SQL through parsing, binding, optimization, physical planning, pipelines, and storage.

There is also a second part from the series which goes further into DuckDB's vectorized execution engine, precompiled inner loops, SIMD, and push based pipelines.

An engineering writeup from the ScyllaDB team on reworking Seastar's io_uring backend so application shards stop spending their own CPU cycles on I/O. Instead of every shard issuing syscalls, shards are grouped and each group hands its asynchronous I/O to a dedicated networking core; the chosen design gives each shard a private io_uring instance while sharing kernel worker pools through IORING_SETUP_ATTACH_WQ, with a single SQPOLL thread polling the group's submission queues so nothing blocks on the compute cores. The post is candid about the tradeoffs, walking through three architectures the team weighed and measuring where the asymmetric backend actually wins on compute heavy request handling versus where one networking core becomes the bottleneck on copy_to_user for pure I/O workloads.

A paper from the AWS team describing the design of Aurora DSQL, a distributed SQL database that aims to scale OLTP workloads across regions without giving up strong consistency. It explains how the system decouples the transaction path from storage, uses quorum based replication and a coordination layer to preserve ACID semantics globally, and keeps latency reasonable by letting regions operate with local autonomy while still agreeing on a consistent order. The evaluation is the interesting part, with measurements across different transaction types and region configurations that show where the tradeoffs actually land rather than just asserting they work.

Cloudflare Research's writeup on Meerkat, an experimental consensus service that keeps control-plane state strongly consistent across their 330+ data centers. The interesting part is that it drops the leader entirely: instead of Raft-style leader election through timeouts, which the post argues are hard to tune on the wide-area internet and have caused availability incidents, Meerkat builds on the QuePaxa algorithm where any replica can accept writes at any time and concurrent proposals interfere constructively rather than fighting. It maintains a linearizable log split into slots, and this is the first industrial deployment of QuePaxa at global scale. Worth reading if you care about how consensus actually behaves across unpredictable wide-area links rather than in a single datacenter.

Ulrich Drepper's classic long-form guide to how modern memory hardware actually works: CPU caches, cache coherency, virtual memory, NUMA, and the concrete techniques you can use to write cache-friendly code. Dated in a few specifics but still the single best foundation for reasoning about memory performance on today's machines.

A retrospective USENIX ATC paper on how DynamoDB evolved from the original Dynamo design into a fully managed service. It's a rare, candid look at the operational lessons behind a system running at enormous scale: predictable performance, admission control, durability, and the trade-offs made to keep tail latencies flat. Good reading if you care about how distributed storage behaves in production rather than on paper.

A clear walkthrough of the Generic Cell Rate Algorithm (GCRA), the mechanism behind the leaky-bucket rate limiter. Instead of tracking a counter that has to be refilled on a timer, GCRA stores a single timestamp — the theoretical arrival time of the next conforming request — and decides admission with a bit of arithmetic. The post builds the intuition step by step and shows why this is both memory-cheap and pleasant to implement.