AI in Regulated Industries

Deploying AI in Regulated Environments Without Breaking Compliance

Bill Barksdale, CISSP

In short

The compliance risk is rarely the pilot that stalls in review. It is that everything else proceeded without governance - staff using whatever assistant they like, on whatever data, under no acceptable use policy. Publish the policy, route model access through a vendor you already hold an agreement with, and design human review into the product.

The story everyone expects about AI in a regulated industry is the pilot that dies in review. Legal will not sign the agreement, security sends a ninety-question assessment, the clinical governance committee wants a regulatory classification, and the thing quietly stops.

That happens. It is not the failure I see most, and it is not the one that should worry you. The one that should worry you is the opposite: AI adoption that succeeded completely, with no governance around it whatsoever.

The pilot is not where it goes wrong

Walk into most companies now and staff are already using AI. Not through a procurement process. They opened a browser tab. Some of them use Claude, some use ChatGPT, some use whatever is embedded in a tool they already had.

In an environment I am working in right now, that is exactly the situation. People are using assistants daily and getting real value from it. There is no standard, no acceptable use policy, and nothing that says which data can go into which tool. The AI is a success. The governance is missing entirely.

That combination is worse than a stalled pilot, because a stalled pilot at least has someone assigned to it. Here, nobody has an inventory of which tools are in use, and nobody would be able to answer a customer or a regulator who asked what happens to their data. The answer exists. It is just that nobody in the building knows it.

Fix that before you build anything. It is not expensive. It is a short policy naming which tools are approved, what categories of data may go into them, what may not, and who to ask when it is unclear. What makes it hard is that it belongs to nobody by default, which is the same ownership gap that leaves everything else undone.

The agreement question has a boring answer

The vendor negotiation people dread is often avoidable.

If you are handling protected health information, any AI vendor processing it on your behalf is a business associate and you need an agreement in place before data moves. The instinct is to go negotiate one with the model provider, which means legal review, a new vendor, and a security assessment of a company nobody in your organization has dealt with before.

On one healthcare build, the decision we landed on was Amazon Bedrock, for the unglamorous reason that the organization already held a business associate agreement with Amazon. That is the whole reasoning. Managed model services from the major clouds let you reach the same class of models under paper you have already signed, already had reviewed, and already have a relationship behind.

It is not always the right answer. If you need a specific model that a cloud does not host, you go negotiate. But check what you already hold before you start a new negotiation, because a large share of AI projects that appear to be blocked on legal are blocked on a contract that already exists somewhere in the building.

Two things worth verifying rather than assuming, whichever route you take: what the terms say about using your data for model training, and which product tiers and features the agreement actually covers. Vendor coverage is specific, and the consumer tier of a product is generally not the tier under contract.

Human in the loop is a feature, not a brake

Human review gets discussed as a compliance tax - the thing you bolt on because a regulator will want it. Designed properly it is a better product, and the compliance benefit is a side effect.

On a hiring platform I worked on, resumes were parsed by AI. The next screen showed the candidate everything the parser had extracted: their skills, degrees, and work history, laid out as fields. They could change any of it before it was saved.

That design does three things at once. The candidate corrects the parser, so your data quality goes up rather than down as volume grows. The person the data is about has seen it and approved it, which is a far stronger position than having inferred it about them silently. And you have a record of human review at the point of entry, which is what a reviewer will eventually ask you to demonstrate.

None of that required a governance framework. It required putting the review on the screen where the work was already happening, rather than in a separate approval queue nobody has time for.

Most of what gets asked for is not AI

The cheapest way to not break compliance is to not send the data anywhere at all.

I have watched a company buy a natural language search product, watch it fail in production, and discover that what their users actually wanted was dropdown filters over well-structured data with the result count updating as they narrowed. We built that instead and it beat the AI product on every measure the users cared about.

That is worth its own argument, and I have made it at length in sometimes you don't need AI, you just need I. The compliance version is short. Every model in your architecture is a vendor agreement, a data flow somebody has to justify, an output somebody has to be accountable for, and an explanation you will eventually owe a customer or a regulator. All of that is manageable. None of it is free. A deterministic function has none of it, and the exposure you never create is the one you never have to govern.

So ask what the user is trying to accomplish before you accept the premise that it needs a model. In a regulated environment that question is not pedantry, it is scope control.

What to actually put in place

An acceptable use policy naming approved tools and the data classes permitted in each. An inventory of what is already in use, which will be longer than you expect. Model access routed through a vendor you already have paper with, where that is possible. Human review designed into the workflow at the point of entry rather than added as an approval queue. A named person who decides, because a policy nobody owns is not a control.

Then apply the same standard you already apply to everything else. Your quality and security obligations attach to the output and the process, not to which tool produced it, which is the same reason AI-assisted code is governable under an existing quality system. The tools are new. Almost none of the obligations are.

This is the work I do — FDA cybersecurity submissions, penetration test remediation, and security ownership for companies too small for a full-time CISO. bill@bato.com.

Common questions

Do we need a BAA with an AI vendor to use AI on PHI?

Yes, if the vendor processes protected health information on your behalf. The practical shortcut is that you may already have one. If you hold a business associate agreement with a major cloud provider, using their managed model service can put the workload under an agreement that already exists rather than starting a new negotiation with each model vendor.

What is the most common AI governance gap?

No acceptable use policy. Staff adopt consumer assistants on their own, the tools work, and nobody has ever written down which tools are approved, what data may go into them, or who decides. The deployment succeeds and the governance never happens, which is the inverse of the failure everyone expects.

Is human-in-the-loop a limitation?

It is usually a better product. Parse with the model, then show the person the extracted result on screen and let them correct it before it is saved. You get a defensible record of human review, and you get corrected data as a byproduct, which is worth more than the automation you gave up.

How do we know if we actually need AI for this?

Ask what the user is trying to accomplish rather than what they asked for. A great many requests for natural language AI are really requests for good filters over clean data with immediate feedback. That version is cheaper, faster, more accurate, and carries no regulatory exposure at all.

Sometimes You Don't Need AI. You Just Need I.Using AI Coding Assistants in a Regulated Codebase Rescuing a Stalled Platform Migration What Enterprise Security Questionnaires Are Actually Asking For

← More in AI in Regulated Industries