The Metadata Field That Sank an AI Procurement Audit

Archi3 min read

aipolicyprocurement

Contents
  1. One Empty Column, Six Months Lost
  2. What the column was supposed to hold
  3. Why nobody caught it earlier
  4. The cost of the fix
  5. The smaller detail underneath
  6. What procurement teams are changing
  7. The general point

One Empty Column, Six Months Lost#

The failure surfaced in a spreadsheet, not a courtroom. A county health department had spent four months evaluating a vendor's claims triage system. The model performed well on paper. Then the compliance officer opened the vendor's data provenance workbook and found a column labeled "source approval" with 340 of 412 rows blank.

That column, roughly 40 characters wide, ended the procurement. Not the accuracy numbers. Not the price. Not the integration timeline.

What the column was supposed to hold#

Federal guidance and most state AI procurement templates now ask vendors to document, for each dataset used in training or fine-tuning, who authorized its collection and under what legal basis. Health records are the obvious case. But the workbook also covered call transcripts, web-scraped public health notices, and a 90,000-row spreadsheet of prior claims decisions that the vendor had received from a subcontractor.

The vendor's counsel argued the blanks were administrative, a formatting gap. The county's privacy office disagreed. Without an approval trail, the agency could not tell whether some of that data had been collected from residents who had opted out of secondary use. Under the county's own data sharing agreement, that made the whole vendor relationship an open question.

Why nobody caught it earlier#

Procurement teams review models against test sets. They check error rates on held-out data, red team results, and latency under load. Provenance is a paperwork category, handled late, often by whoever draws the short straw.

In this case, the vendor had answered the model questions thoroughly and the data questions in a separate attachment that arrived nine days before the evaluation deadline. Reviewers treated the attachment as supporting material. The blanks looked like formatting.

That sequencing is the lesson. When provenance sits at the end of the review, it gets read by tired people against a clock. When it sits at the front, it shapes which systems are worth testing at all.

The cost of the fix#

The county restarted the procurement. The vendor rebuilt its data inventory: roughly 1,100 hours of legal and data engineering work, according to the timeline it later shared with the county. Two subcontractor datasets were dropped because the original consent forms could not be located. The model's performance on one demographic subgroup dropped by a measurable margin after the data was removed, which meant retraining.

Total delay: six months. The agency ran its triage work on spreadsheet rules in the interim, which it described as adequate for the volume and cheaper than an emergency contract.

The smaller detail underneath#

The blank column was not the root issue. The root issue was that the vendor tracked data by project, not by record. When a subcontractor delivered a dataset, the vendor logged it under the contract name. No one logged which consent form covered which row.

That is a schema problem, and it is common. Teams building AI systems often inherit data pipelines designed for analytics, where provenance means "which folder did this come from." Auditors ask a different question: for this specific record, who said yes, and when.

What procurement teams are changing#

A few patterns are showing up in agency reviews this year, and they are small on purpose: , Requiring a per-dataset approval field before the model evaluation starts, not after. If the field is blank, the review does not open the test results., Asking vendors to name the system of record for consent, not just the process. A named database can be inspected. A described process cannot., Treating subcontractor data as a separate submission with its own approval trail, because responsibility rarely travels cleanly down the chain., Writing the provenance requirement into the data sharing agreement itself, so it binds before anyone builds anything.

None of this requires new law. Most of it fits in existing contract language.

The general point#

AI governance debates tend to run at the level of rules and risk frameworks. The failures that stop projects run at the level of a column header, a log entry, a filename convention.

A model that cannot answer "who approved this row" is not a model with a documentation gap. It is a model whose training data has an unknown legal status. Procurement officers are starting to read it that way, and vendors who treat provenance as a first-class deliverable, built record by record, will clear reviews that their faster competitors will not.

The county's next solicitation includes one new line in the mandatory requirements: a completed provenance workbook, submitted with the bid, no exceptions. It is two sentences long. It is the most important thing in the document.

More articles