The 10x Rule in CI: Why Build Caches Are the Real Bottleneck
Archi4 min read
Contents
The 10x Rule in CI: Why Build Caches Are the Real Bottleneck#
When developers talk about build and test speed, they tend to talk about the tools. A compiler gets faster. A test runner adds parallelism. A pipeline gains a new stage. The vendor blog post promises a 10x improvement, and the engineering team adds it to the roadmap.
Then the pipeline still takes 40 minutes.
The surprising number in continuous integration is not a benchmark score. It is a ratio. Across a wide range of real pipelines, the time your team spends waiting on a build is often closer to 10 times the time the compiler or test runner actually spends doing work. The rest goes to fetching dependencies, restoring caches, downloading container layers, and moving artifacts between stages. Fix the cache and you get a real speedup. Fix the compiler and you might save 30 seconds.
What the 10x Actually Measures#
A typical pipeline has three cost centers. The first is compute: compiling code, running tests, packaging binaries. The second is transfer: pulling package registries, container images, and caches. The third is orchestration: scheduling jobs, waiting on runners, passing artifacts downstream.
Most tooling vendors optimize the first. Most pipeline time, in my experience and in public postmortems, is spent in the second and third. A cold CI run on a mid-sized repo can spend more time downloading a base image than compiling the application on top of it. A test suite that finishes in 90 seconds can sit idle for several minutes because the runner waited for a cache miss before it could start.
That ratio is the number worth watching. If your pipeline takes 30 minutes, and 25 of those are waiting on I/O, a faster compiler changes almost nothing.
The Cache Is the Build#
A build cache stores the outputs of previous work so the next run can skip repeated steps. That includes compiled objects, dependency trees, test results, and container layers. When the cache hits, the pipeline behaves like an incremental build. When it misses, the pipeline behaves like a fresh install on a laptop that has never seen your project.
Caching is usually treated as an optimization. In practice, it is the pipeline. The build system itself is often fast enough. The expensive part is reconstructing the environment from scratch every time someone pushes a commit.
Three signs that the cache, not the tool, is your bottleneck: , The first job spends minutes downloading before it runs any code. Package registry latency is doing more harm than your compiler., Cache hit rate is low or unmeasured. If you cannot state the hit rate for your main pipeline, you are guessing about where the time goes., Pipelines are slower on Mondays. Cold caches after a weekend of eviction are a tell. Expensive transfers are hiding behind the usual build log.
Where the Time Goes Missing#
There are several common reasons a cache fails to help. Cache keys may include a hash of the lockfile, so any dependency bump invalidates everything. Cache storage may be scoped per branch, so every feature branch starts cold. Container layers may be rebuilt because the base image tag moved. Artifacts may be re-uploaded between jobs instead of shared through a mounted volume.
None of these are tool bugs. They are configuration decisions that quietly turn a fast build into a slow one. The fix is rarely a new compiler flag. It is a cache policy that is explicit about what gets stored, for how long, and under what keys.
A Practical Checklist#
Start with measurement. Instrument the pipeline so you know how much time goes to compute, to dependency fetch, to cache restore, and to artifact upload. Most CI systems expose this in job logs, but almost nobody reads it carefully.
Then narrow the cache scope. Use a lockfile hash as the key for dependency caches, but keep the base image and toolchain caches separate. A toolchain upgrade should not invalidate the package cache.
Share caches across branches where it is safe. A merged main branch is a better source of cache entries than an isolated feature branch. Many systems support this; it is usually off by default.
Keep container layers warm. Pin base images by digest, not by a moving tag. Rebuild layers on a schedule instead of on every commit.
Track the hit rate. A cache with no metrics is a cache you cannot manage. If the hit rate drops, treat it like a failing test.
What This Buys You#
A pipeline that spends its time on transfer is a pipeline that responds well to infrastructure changes. A pipeline that spends its time on compute responds only to code changes. Most teams spend months optimizing the second and never measure the first.
The 10x number is not a promise. It is a warning about where to look. Before you swap the build tool, check the cache. The compiler is probably not the reason you are waiting.
More articles