https://segflow.github.io/tags/concurrency/Recent content in Concurrency on SegflowHugoen-usSun, 19 Feb 2023 16:00:00 +0100https://segflow.github.io/post/backpressure-patterns/Sun, 19 Feb 2023 16:00:00 +0100https://segflow.github.io/post/backpressure-patterns/<p>A pipeline has backpressure when the consumer can make the producer slow down, or refuse the work, or both. A bigger buffer does neither. It just defers the same problem, with more memory held hostage in the meantime. The two get confused a lot, and the difference shows up unmistakably in the tail latency numbers, so this post does the comparison directly.</p> <p>When load exceeds capacity you have three sane options:</p> <ol> <li><strong>Block the caller</strong> until a slot frees. Throughput pins to whatever the consumer sustains; tail latency degrades cleanly.</li> <li><strong>Reject the caller</strong> so they retry, fall back, or shed. Latency stays low for accepted work; some work never happens.</li> <li><strong>Rate-limit the caller</strong> so they never get close to overloading you. Predictable throughput, the rest is queued or dropped.</li> </ol> <p>Go has a direct idiom for each: a <strong>bounded channel</strong> for option 1, the <strong><code>golang.org/x/sync/semaphore</code></strong> package for option 1 with per-item weighting, and a <strong><code>time.Ticker</code> token bucket</strong> for option 3 (and trivially option 2). They are not interchangeable. They produce three very different latency curves under the same workload, which is what we will see below.</p>