Using AI Coding Assistants in a Regulated Codebase

The question I get asked is whether AI coding assistants are allowed in a regulated codebase. That is the wrong question, and it has a boring answer: your quality system governs how a change is reviewed, verified, and traced. It does not specify who typed it.

The useful question is what these tools break when you scale them up.

Review capacity is the bottleneck

An assistant can propose more change in an afternoon than a team can carefully review in a week. The constraint was never typing speed - it was the attention required to be confident a change is correct.

If review throughput does not rise with proposal throughput, the gap fills with code that nobody examined closely. In a regulated codebase, that is exactly the material your design controls exist to catch, arriving faster than the controls can process it.

The mitigation is unglamorous: keep changes small enough to review properly, and treat review capacity as the thing that sets your pace.

Provenance is a real question

These tools are trained on public code. A generated implementation may closely resemble something that carries a license, and your SBOM and third-party software controls exist precisely to answer "what is in this product and under what terms."

For a device submission, that matters. You are asserting what your software comprises. Code of uncertain provenance sits awkwardly against that assertion.

Practically: keep humans accountable for what enters the repository, and treat generated code that looks like a recognizable library implementation as something to check rather than assume.

Verification does not get easier

A generated test that passes tells you the code does what the generator thought it should. If the same tool wrote the implementation and the test, a shared misunderstanding produces a green build and a wrong device.

Requirements-derived tests are the point of verification in a regulated context. Those still trace back to a requirement written by a person who understood the clinical or operational intent.

What a reviewer actually asks for

The claim that your quality system already governs this is easy to make and worth showing, because the evidence is specific and none of it has a field for which tool produced the code.

A reviewer asks for the requirement the change traces to, the design output it belongs to, the record that somebody competent reviewed it, the verification that exercises it, and the change control that authorized it to ship. That is the same list whether the first draft came from an engineer, a contractor, or a model. If you can produce it, the question of how the code was drafted never becomes interesting. If you cannot, the tool is not your problem.

Two places where the usual evidence does get strained, and both are worth getting ahead of:

The review record has to mean something. An approval recorded by somebody who skimmed a large generated diff is weaker evidence than the same approval on a change small enough to have been read. Nobody can see the difference in the record, which is exactly why volume is the risk. If a reviewer ever does look closely, large machine-generated changes approved in minutes are visible in the timestamps.

Traceability has to be real in both directions. Generated code tends to be helpful in ways nobody asked for: an extra endpoint, a convenience path, a fallback. Anything in the product that traces to no requirement is either an undocumented feature or dead code, and both are findings. The discipline is the same one that catches it in hand-written code; there is simply more of it to catch.

The data question nobody asks first

The governance conversation about coding assistants is usually about output quality. The one that arrives later, from a customer or a regulator, is about input.

Your source code is proprietary, and in a regulated product it frequently describes how safety controls are implemented. Sending it to a third party is a disclosure, governed by whatever agreement you hold with that vendor and by whatever your contracts with customers say about confidentiality. Worth verifying rather than assuming: what the terms say about using your code for model training, which product tier the agreement actually covers, and whether a consumer-tier account somebody signed up for personally is covered at all.

The usual shortcut applies here too. Reaching a model through a cloud provider you already hold an agreement with is often faster than negotiating a new one, and a large share of blocked AI projects are blocked on paper that already exists somewhere in the building. The full version of that argument is in deploying AI in regulated environments.

Where they genuinely help

I use them heavily, including on regulated work, for the tasks where output is easy to verify: reading unfamiliar code, drafting documentation from a diff, mechanical refactors, exploring an approach before committing to it. Anywhere the cost of checking the answer is low, the leverage is real.

The trap is the opposite case - complex logic where the output looks plausible and checking it properly takes longer than writing it would have. Deciding which tasks fall on which side of that line, and writing it down, is AI governance rather than a tooling preference.