The Patch That Did Not Apply: Root-Causing a Silent Build Failure

Archi4 min read

open-sourcepackagingbuilds

Contents
  1. The Build That Succeeded and Shipped Nothing
  2. First stop: the patch file
  3. The Context Matching Rule
  4. What the exit code hid
  5. Reproducing the Failure on Purpose
  6. The three checks that matter
  7. Why This Keeps Happening in Distros

The Build That Succeeded and Shipped Nothing#

I spent a Friday afternoon chasing a package that installed cleanly, reported success, and then failed to work. The fix was a two-line patch from upstream. The distro's build system said it applied the patch. The binary disagreed.

This is a field note about that afternoon, because the root cause was not in the patch and not in the code. It was in the rule that decides whether a patch applies at all, and in what the build script does when the rule says no.

The package was a small text-processing utility. Version 2.4.1 had a known bug where a locale change broke its sort order. Upstream shipped a patch on a mailing list. I downloaded the tarball, dropped the patch into debian/patches/, added it to the series file, and ran the build. Exit code zero. Package installed. Sorting still wrong.

First stop: the patch file#

The patch looked fine. It was a unified diff against sort.c, context lines included, no trailing whitespace weirdness that I could see at a glance. I applied it by hand to a fresh extraction with patch -p1 and it worked. So the patch was not corrupt and the source was not wrong. Something between the two was eating the change.

I reran the build with verbose logging. The build log showed the patch tool being invoked, then a line I had skimmed past the first time: Hunk #1 succeeded at 47 (offset -3 lines). That is not a failure message. It is the patch tool telling you it found the context a few lines away from where the diff expected it and applied it anyway. That is the normal, helpful behavior of patch. It is also the behavior that hides drift.

The Context Matching Rule#

Unified diffs do not identify lines by number. They identify them by surrounding context: three lines above and three below, give or take. When the target file has moved, patch searches nearby for matching context and shifts the hunk. This is why patches survive small unrelated changes upstream.

It is also why a patch can apply to the wrong place. On a file that has been reformatted, the context can match a block of code that looks similar but sits in a different function. The hunk lands, the build compiles, and the behavior is unchanged or subtly broken. The tool has no idea what the code means. It only knows whether the lines around the target look familiar.

In my case the context matched a comment block that had been duplicated elsewhere in the file by an earlier distro patch. The fix landed in a block of dead code. The sort order was never touched.

What the exit code hid#

The deeper problem was the build script. It called the patch tool and checked only that the process exited without a crash. With set -e not in play and no check on the tool's own status, a mismatch between the applied result and the expected result went unreported. The build continued, compiled the unpatched logic, and produced a package. Nobody downstream had a reason to look.

Patch tools signal trouble in more than one way. A rejected hunk gets written to a .rej file and bumps the exit status. A hunk applied with an offset does not. If you want the build to fail on drift, you have to ask the tool to be strict. In patch, that means flags like --fuzz=0 (no fuzzy line matching) and a check that no .rej files appeared. Without those, offset application is silent.

Reproducing the Failure on Purpose#

To confirm the theory I built a small test. I took a file with two nearly identical comment blocks, added a patch whose context matched the second block, and ran it with default settings. The hunk applied to the wrong block at an offset of 18 lines. Exit code zero. Then I reran with strict fuzz and a .rej check, and the build stopped as it should have.

The difference between the two runs was one flag and one line of shell. That is the whole cost of catching this class of bug at build time.

The three checks that matter#

, Confirm the hunk landed where the patch author meant it, not just that it applied. Compare a post-patch hash of the target file against a known-good build, or grep for the changed line., Treat .rej files as build failures. A single find . -name '*.rej' after the patch step costs nothing., Pin patch fuzz to zero in CI, so offset application fails loudly instead of quietly.

None of this is exotic. It is the difference between a build that proves the patch is present and a build that assumes it.

Why This Keeps Happening in Distros#

Downstream packaging lives on patches. Upstream source moves, distro patches lag, and every release drags a stack of diffs that were written against an older tree. Maintainers do not always get to rebase, and the tooling is designed to be forgiving. Forgiving tooling plus a build script that only checks for a crash is a silent failure generator.

There is a second layer too. Some patch series are applied in order, and a later patch depends on context created by an earlier one. If the first applies at an offset, the second can match the wrong block with no warning. The failure compounds down the series.

The fix in my case was three lines of build script and a comment explaining why the strict flags are there. The patch now fails the build when it drifts. The next time upstream moves, someone gets a red build instead of a bug report six weeks later from a user who cannot sort their files.

More articles