The 3.7% Rule: Why One AI Policy Number Surprises Regulators
Archi3 min read
The 3.7% Rule: Why One AI Policy Number Surprises Regulators#
Policy analyst Maya Chen has spent two years reading AI audit reports filed under emerging state rules. She says one number keeps coming up: 3.7%. It is the share of audit submissions that actually include a usable model card, meaning one with version, training data description, and known limitations filled in. The rest are incomplete or missing. We spoke with her about what that number means for enforcement and for companies trying to comply.
Q: Why does 3.7% matter more than other compliance metrics?
A: Because it is a leading indicator, not a lagging one. If only 3.7% of submissions have a usable model card, then the audit regime is mostly generating paperwork, not evidence. Regulators can write rules all day, but if the underlying documentation is absent, enforcement becomes guesswork. That number tells you where the bottleneck actually sits.
Q: Is this a filing problem or a deeper issue?
A: Both. Some teams simply do not have a model card template that matches what auditors ask for. Others have one but skip it because the model was fine-tuned from a vendor checkpoint and nobody documented the delta. The deeper issue is that model cards were designed for research papers, not for procurement or compliance. They describe architecture and benchmarks but rarely say what the model cannot do, which is the part auditors care about most.
Q: What surprised you most in the submissions?
A: The gap between what companies claim and what they can show. Many submissions list a model card as an attachment, but the attachment is a slide with three bullet points. One filing included a screenshot of a wiki page with a red banner reading "outdated." That is not malice. It is a documentation process that was never built to survive an audit.
Q: How does this compare with other regulated documentation?
A: Financial audits have a similar problem with control narratives, but the difference is materiality. A missing control narrative can be fixed with a memo. A missing model card means you cannot reconstruct why a system behaved a certain way six months ago. That is a bigger gap because AI systems change through retraining and fine-tuning, and the card is often the only record of what changed.
Q: What should a company do first if its model card is incomplete?
A: Pick the model that touches the most customers and write a one-page card for it this week. Include version, date, training data sources at a category level, evaluation results, and a short list of known failure modes. Do not wait for a template from a regulator. The 3.7% figure exists because most teams waited.
Q: Does this number vary by sector?
A: Yes. Submissions from companies with existing software supply chain practices tend to do better because they already track dependencies and versions. Companies that built AI features quickly on top of a vendor API often have no internal record of the model version they shipped. That is the pattern behind most of the incomplete filings.
Q: What would move the number?
A: Procurement. If large buyers required a usable model card before signing, the number would rise faster than any rulemaking. Audits are downstream. Contracts are upstream, and they change behavior in weeks rather than years.
Q: What is the risk of ignoring this?
A: The risk is not a fine. It is that when something goes wrong, the company cannot show what it knew and when. That is a harder position than a penalty. The 3.7% is a warning about records, not about technology.
More articles