Sergey's Blog

Git at any scale

2026-08-22T00:00:00+00:00

A post from Cursor about Continuity, the Git storage system they built to host repositories at real scale. Instead of treating each repo as one precious copy on one disk, every push goes to a write ahead log in S3 first, and the client only hears back once that write is durable. Local disks become warm caches that can be rebuilt at any time, and replicas stay in sync with gossip over UDP plus conditional reads against S3. The post also covers repacking without making every replica redo the same expensive work, and gives real numbers: about 120 pushes a second on standard S3 and over 300 on S3 Express One Zone.

Rendezvous hashing

2026-08-22T00:00:00+00:00

An overview of rendezvous hashing, a simple way to assign objects to servers without a central coordinator. Each server gets a score from the object and server names, and the highest score wins. Every client reaches the same result, and adding or removing a server moves only the objects affected by that change. The article also compares the method with consistent hashing and covers replication, weights, and faster variants.

The when, why and how of waiting and backoff in multi-threaded applications on Arm

2026-08-08T00:00:00+00:00

An Arm post on what a thread should do while it waits, whether on a lock or after a failed atomic. Spinning in a tight loop is the obvious move, but it floods memory with traffic and slows every other core touching the same location, so backing off helps both throughput and fairness. The post covers backoff strategies that space out the checks, and the Arm specific tools for them: the counter timer for timed waits, the WFET instruction on Armv8.7 and later that lets a core sleep for a set duration instead of burning power, and barriers like ISB and SB that control how far ahead the processor looks. It also lists patterns that look correct but are not, such as empty loops and simple LDXR plus WFE combinations, and explains why they fall apart as thread counts grow.

uringscope: Portable, Low-Overhead Observability for io_uring

2026-08-08T00:00:00+00:00

A paper on why io_uring workloads are so hard to see into. Submission and completion happen in rings shared with the kernel, so strace catches only the setup call and you are left guessing when something goes wrong. uringscope is a single binary eBPF tool that watches those rings and rebuilds the life of each request from raw kernel events, using CO-RE, BTF probes, and flexible field lookups so one build survives the tracepoint churn across kernel versions. On real NVMe workloads it costs about 0.7 to 9.9 percent of throughput, cheaper than the other tools measured at the same level of detail. The same data feeds a doctor mode that turns raw measurements into named problems with the evidence behind them, aimed at someone chasing a tail latency bug rather than browsing histograms.

DuckDB Internals: Why is DuckDB Fast?

2026-07-26T00:00:00+00:00

A walk through why DuckDB, an in process analytical database, runs SQL so fast. It follows a query from parsing to execution and shows how each step avoids extra work: running as a library instead of a server cuts out network and serialization overhead, the optimizer pushes filters down and picks join order with dynamic programming, and columnar storage with zone maps lets whole chunks of data be skipped when they cannot match a filter. It also explains how a query is split into pipelines so the work can run in parallel across threads, each with its own local state. The result is a full picture of the system rather than a list of buzzwords.

There is also a second part from the series which covers how those plans actually run: vectorized execution in batches of 2048 rows, selection vectors that filter without copying, and a push based model that spreads work across CPU cores.

Offloading I/O to Dedicated Cores: An Asymmetric io_uring Backend for Seastar and ScyllaDB

2026-07-23T00:00:00+00:00

A ScyllaDB engineering post about a new io_uring backend for Seastar that breaks the usual shared nothing rule, where every core does its own I/O and compute. The asymmetric backend instead sets aside a few cores as dedicated networking workers while the rest run only application logic, and routes I/O syscalls to those workers through io_uring queues. To make it work the team had to remove a speculative fast path that let a shard skip io_uring and issue a plain syscall on its own core, since that shortcut defeats the point of offloading. The numbers are honest about the tradeoff: raw I/O throughput trails the older linux aio backend, but compute shards get back the CPU time they used to spend on sockets and disk calls.

Aurora DSQL: Scalable, Multi-Region OLTP

2026-07-18T00:00:00+00:00

A paper from Amazon engineers on how Aurora DSQL works inside. DSQL is a serverless SQL database that runs active active across regions, so any region can take reads and writes at once. The design splits the two apart: reads use multiversion concurrency control with precise timestamps and never coordinate with other nodes, while writes use optimistic concurrency control and coordinate only at commit time, through components called adjudicators and a replication layer called the Journal. Query processors run in small Firecracker microVMs and keep no local state, which lets compute, storage, and coordination scale on their own, from idle up to millions of transactions per second. The paper shows how all of this keeps full ACID transactions even when an availability zone or a whole region fails.

Introducing Meerkat: an experiment in global consensus

2026-07-11T00:00:00+00:00

A Cloudflare post about Meerkat, a consensus system they built to keep control plane state consistent across their 330 plus data centers. It runs on QuePaxa, an algorithm that needs no leader. In Raft a dead leader or a slow network stalls writes until a new one is elected, and the timeouts are hard to tune across the wide area internet. QuePaxa instead lets any replica propose a write at any time, and concurrent proposals help each other reach agreement rather than block each other. The post is honest about the cost, since each write still takes one to three round trips, so Meerkat suits data that changes rarely and must stay correct rather than a busy database. It is also the first time QuePaxa has run at production scale.

How /dev/null and Friends Work Inside the Linux Kernel

2026-03-08T00:00:00+00:00

If you list all device files with major number 1 on a Linux machine, you'll see something like this:

$ ls -la /dev/ | grep ' 1, '
crw-rw-rw-  1 root root      1,     7 Jan 18 20:01 full
crw-r--r--  1 root root      1,    11 Jan 18 20:01 kmsg
crw-r-----  1 root kmem      1,     1 Jan 18 20:01 mem
crw-rw-rw-  1 root root      1,     3 Jan 18 20:01 null
crw-r-----  1 root kmem      1,     4 Jan 18 20:01 port
crw-rw-rw-  1 root root      1,     8 Jan 18 20:01 random
crw-rw-rw-  1 root root      1,     9 Jan 18 20:01 urandom
crw-rw-rw-  1 root root      1,     5 Jan 18 20:01 zero

Every Linux system has them. You've used /dev/null a thousand times. You've probably read from /dev/urandom or /dev/random. But have you ever looked at what backs these files inside the kernel?

What makes these eight devices special is that they are among the simplest real drivers in Linux. There's no hardware to initialize, no DMA buffers to manage, no firmware to load. Just pure logic: "when userspace calls read(), do this." That makes them the perfect starting point for understanding how any Linux character device driver works.

We'll walk through their implementations line by line. Along the way, you'll see the interface every character device must conform to, the patterns that the kernel uses to dispatch I/O to the right handler, and the design choices that separate a minimal driver from a production-ready one. By the end, you'll have a concrete blueprint for writing your own.

All the code in this post comes from Linux 6.19 (drivers/char/mem.c and drivers/char/random.c). Let's start with open("/dev/null", ...) and trace it all the way down to the two-line C function that makes it work.

Memory Barriers: a Hardware View for Software Hackers

2025-03-21T00:00:00+00:00

Paul McKenney's paper on why memory barriers exist at all, built from the hardware up. It starts with how a CPU cache is laid out, then how the MESI protocol keeps caches agreeing on the value of each location, then how store buffers and invalidate queues quietly break that agreement in exchange for speed. Once you see those two queues, read and write barriers stop looking arbitrary and start looking like the obvious fix.

What Every Programmer Should Know About Memory

2025-03-20T00:00:00+00:00

Ulrich Drepper's long guide from 2007 on how memory hardware really works and why memory, not the CPU, is often what makes a program slow. It covers how RAM chips work, how CPU caches are built and why they exist, and how virtual memory and NUMA change the picture, then spends a large middle section on concrete advice for writing code that uses caches well. There are plenty of diagrams and real numbers measured on real hardware, along with practical topics like data layout, cache line size, and tools that help you find memory related slowdowns. Some of the hardware details have aged, but the core ideas still hold up.

A Decade of Dynamo: Powering the next wave of high-performance apps

2025-01-15T00:00:00+00:00

A USENIX ATC paper that looks back at how DynamoDB grew from the original Dynamo design into a managed service. It shares what the team learned from running the system at a very large scale and shows how distributed storage works in production, not just in theory.

GCRA: a simple and elegant rate-limiting algorithm

2024-09-01T00:00:00+00:00

A clear guide to the Generic Cell Rate Algorithm (GCRA), which is used for leaky bucket rate limiting. Instead of tracking a counter and refilling it on a timer, GCRA stores one timestamp and uses a simple calculation to decide whether to allow a request. The post explains the idea step by step and shows why it needs little memory and is easy to build.

Introduction to JIT Compilation

2012-10-04T00:00:00+00:00

The Java HotSpot VM (which Oracle acquired after purchasing Sun Microsystems) forms the basis for both the Java Virtual Machine (JVM) and OpenJDK (an open-source project). Like all Java virtual machines, the Java HotSpot VM provides the necessary environment for executing bytecode. In practice, it is responsible for three main functions:

bytecode interpretation
class loading and type checking
memory management

This article focuses on bytecode interpretation, specifically the optimizations performed by the virtual machine.…

About Sizes in Java

2012-09-26T00:00:00+00:00

Recently, I've been pondering the question: "How can one accurately assess the amount of memory allocated for objects in Java?" There are several articles on Habrahabr [1], [2] dedicated to this topic. However, I wasn't quite satisfied with the approach used by the authors. Therefore, I decided to delve into the internals of the OpenJDK Hotspot VM (hereafter referred to as Hotspot) and try to understand how things really work.

Data Types in Java

Primitives (byte, short, char, int, float, long, double, boolean).
Objects. The size of an object depends on the specific VM implementation and the processor architecture, so a definitive answer is not possible. Nonetheless, it's interesting to explore (using a specific VM as an example) what size of memory is allocated for a Java object.
Arrays. One-dimensional linear structures that can contain all listed types (including other arrays). Arrays are also objects but with a specific structure.…