The Real Reason OS Updaters Break: File Locking, Not Bad Code

Archi4 min read

operating-systemsupdatesfile-systems

Contents
  1. The update that bricked 400 laptops was not a code quality problem
  2. Why in-use file replacement is harder than it looks
  3. The three failure modes you will actually see
  4. What a correct design looks like
  5. The numbers that matter

The update that bricked 400 laptops was not a code quality problem#

In early 2025, a mid-sized manufacturer pushed an OS update to 400 field laptops. Within hours, 38 percent of them failed to boot. The postmortem blamed a "buggy driver" and a "rushed QA cycle." Both were symptoms. The root cause was file locking: the update mechanism could not safely replace a driver file while the system held it open, so it scheduled the swap for the next reboot. The reboot happened at the wrong moment, mid-write.

This pattern repeats across Windows, macOS, and Linux. The technical fault is not that updaters are poorly written. It is that the file locking primitives they rely on were designed for a single-user, single-boot world. Modern fleets run dozens of background services, antivirus hooks, and container runtimes that hold handles on the same files an update needs to touch. The updater's job is to verify that a file is not in use, then replace it. That verification is a race condition by design.

Why in-use file replacement is harder than it looks#

On Unix-like systems, the usual trick is to write the new file to a temporary path and rename it over the old one. The rename is atomic. Processes that already opened the old file keep reading from the old inode until they close it. New opens get the new file. This works well for applications that tolerate a mix of old and new versions running at the same time. It fails for kernel modules, shared libraries loaded by long-running daemons, and anything that mmaps a file and expects the mapping to stay valid.

Windows is more restrictive. You cannot delete or replace a file that has an open handle without sharing delete access. The standard workaround is to move the file to a pending rename registry key and let the session manager finish the operation at boot. That boot-time window is where many failures happen. If the pending rename list is corrupt, or if a driver loads earlier than expected, the replacement never completes. The system boots with a half-updated driver set.

The deeper issue is that file locking is advisory on Unix and mandatory on Windows, but neither model gives the updater a reliable way to ask: "Is any process going to need this file in the next 90 seconds?" The updater can only observe the present.

The three failure modes you will actually see#

First, the check-then-act gap. The updater checks that no process has the file open, then writes. Between the check and the write, a service starts and opens the file. The write corrupts the in-memory mapping. On Linux this often shows up as a SIGBUS when the process touches a page that no longer has a backing file.

Second, the reboot race. The updater stages a replacement for next boot. Another update or a user action triggers an earlier reboot. The staged operation runs against a different kernel version than the one it was prepared for. The driver loads, but the ABI has changed.

Third, the antivirus interposition. Security software opens files with exclusive access to scan them. The updater sees the lock, backs off, and retries on a schedule. If the scan runs every 15 minutes and the updater retries every 10, the update can loop indefinitely. The user sees a progress bar stuck at 94 percent.

What a correct design looks like#

The updater should not try to replace files in place. It should build a new system image, or at least a new set of immutable snapshots, and switch to it atomically at boot. This is how ChromeOS, Fedora Silverblue, and Android's A/B partitions work. The running system is never modified. The update writes to an inactive slot, verifies the slot, and flips a boot flag. If the new slot fails to boot, the bootloader falls back to the old one after a watchdog timeout.

For systems that cannot use A/B slots, the next best option is to stop all writers before the update begins. That means a proper maintenance mode: quiesce services, unmount filesystems, and perform the replacement from a minimal environment. The problem is that desktop and server OSes resist quiescing because they assume a user is present and will not tolerate a full stop.

My view is that the industry should treat file replacement during a live update as a legacy hazard, the way we now treat running as root. It should be disabled by default and allowed only with an explicit opt-in flag. The cost is a reboot. The benefit is that the updater never has to guess whether a file is safe to overwrite.

The numbers that matter#

A fleet of 400 laptops with a 38 percent failure rate is 152 devices. If each device costs four hours of IT time to recover, that is 608 hours. At a loaded rate of $60 per hour, the bill is $36,480. The update itself probably cost less than $5,000 to develop. The economics favor a slower, safer update path almost every time.

The next time an OS update fails in the field, look at the updater log for pending file rename operations. Count how many files were staged for boot-time replacement. If the number is above zero, the failure is not a mystery. It is a locking problem that the design chose not to solve.

More articles