← All pins

Pins tagged “performance”

Git at any scale

A post from Cursor about Continuity, the Git storage system they built to host repositories at real scale. Instead of treating each repo as one precious copy on one disk, every push goes to a write ahead log in S3 first, and the client only hears back once that write is durable. Local disks become warm caches that can be rebuilt at any time, and replicas stay in sync with gossip over UDP plus conditional reads against S3. The post also covers repacking without making every replica redo the same expensive work, and gives real numbers: about 120 pushes a second on standard S3 and over 300 on S3 Express One Zone.

distributed-systemsconsensusperformance

The when, why and how of waiting and backoff in multi-threaded applications on Arm

An Arm post on what a thread should do while it waits, whether on a lock or after a failed atomic. Spinning in a tight loop is the obvious move, but it floods memory with traffic and slows every other core touching the same location, so backing off helps both throughput and fairness. The post covers backoff strategies that space out the checks, and the Arm specific tools for them: the counter timer for timed waits, the WFET instruction on Armv8.7 and later that lets a core sleep for a set duration instead of burning power, and barriers like ISB and SB that control how far ahead the processor looks. It also lists patterns that look correct but are not, such as empty loops and simple LDXR plus WFE combinations, and explains why they fall apart as thread counts grow.

armperformanceconcurrency

DuckDB Internals: Why is DuckDB Fast?

A walk through why DuckDB, an in process analytical database, runs SQL so fast. It follows a query from parsing to execution and shows how each step avoids extra work: running as a library instead of a server cuts out network and serialization overhead, the optimizer pushes filters down and picks join order with dynamic programming, and columnar storage with zone maps lets whole chunks of data be skipped when they cannot match a filter. It also explains how a query is split into pipelines so the work can run in parallel across threads, each with its own local state. The result is a full picture of the system rather than a list of buzzwords.

There is also a second part from the series which covers how those plans actually run: vectorized execution in batches of 2048 rows, selection vectors that filter without copying, and a push based model that spreads work across CPU cores.

databasesduckdbperformance

What Every Programmer Should Know About Memory

Ulrich Drepper's long guide from 2007 on how memory hardware really works and why memory, not the CPU, is often what makes a program slow. It covers how RAM chips work, how CPU caches are built and why they exist, and how virtual memory and NUMA change the picture, then spends a large middle section on concrete advice for writing code that uses caches well. There are plenty of diagrams and real numbers measured on real hardware, along with practical topics like data layout, cache line size, and tools that help you find memory related slowdowns. Some of the hardware details have aged, but the core ideas still hold up.

memoryperformancehardware