With most high-risk AI obligations under the EU AI Act set to apply from 2 August 2026, enterprises running agentic AI in healthcare, finance, and energy face a problem most have been deferring: the data infrastructure beneath their AI was never built to be audited. Across U.S. healthcare, financial services, and energy, enterprises running AI in production are confronting the same challenge.
Vishnu Bapathi is a lead AI engineer at Boston Consulting Group with more than 15 years of experience building data and AI platforms for Fortune 500 companies across financial services, healthcare, energy, and manufacturing, leading large engineering teams and architecting governed cloud platforms for enterprise clients. Most recently, he has been working on a governed data foundation for a major medical device company, where every reported metric can be traced from the dashboard back to the underlying Salesforce field.
Over the past five years, his work has focused on helping organizations move AI from pilot projects to production in regulated environments, where auditability and accountability are as important as model performance.
We spoke with him about what separates pilots from production, why agentic systems are forcing a rethink of data architecture, and what it takes to make compliance the default rather than the checkpoint.
Today, banks, insurance companies, and healthcare organizations are actively adopting GenAI and ML, but they do so more cautiously than other industries. How would you define the primary objective of AI in regulated environments?
The primary objective is decision support that can be defended after the fact. Not whether the model gave a good answer in the moment. Whether you can explain, six months later, what data the model used, what version of that data, and who is accountable if the answer turned out to be wrong.
Most teams optimize for the moment. They tune the model, test the prompt, ship the pilot. The harder problem starts on the day someone asks “how did we get to that recommendation?” and the system can’t answer.
I work in healthcare on a project right now where an analytics platform feeds clinical procedure attribution, revenue reporting, and patient routing decisions for a medical device launch. Every number on the executive dashboard has to trace back to a source field in Salesforce. Not because someone might audit it. Because someone will. Finance, compliance, the regulators, the field team that is operationally dependent on the number. If a metric shifts by 15% between two reports and we can’t explain why, the program loses trust faster than any model accuracy issue would cause it to.
The cautious pace in regulated industries is not resistant to AI. It is the lived experience of what happens when fast and defensible diverge.
What happens to the organizations that treat governance as something to retrofit later?
They spend the second half of every project undoing decisions made in the first half. I have seen this on multiple engagements. A team builds a model that performs well in dev, hits production, and then finds out the data it was trained on had three different definitions of “customer” or “patient” depending on which source system it came from. The model is doing what it was told. The training data was the problem. By the time you discover that, the team has shipped reports, made staffing decisions, briefed executives. Unwinding it is expensive.
Retrofitting governance is roughly two to three times more expensive than building it in. That number comes from comparing the cost of platform redesigns I have run mid-engagement against the cost of doing it right the first time. And in some cases it cannot be cleanly retrofitted at all because the audit trail for past decisions no longer exists.
When we talk about “safe AI,” what does that mean from an engineering perspective: data security, model predictability, or something else?
Safe AI in an enterprise context comes down to three things working together: bounded inputs, predictable outputs, and observability over time. Most organizations focus on one of the three and miss the others.
Bounded inputs means the model only ever sees data it is authorized to see, enforced at the platform layer instead of inside the application. Predictable outputs means the same input produces the same output, or the variation is measured and tolerated within known bounds. Observability means you know what the model did yesterday, what it is doing today, and what changed.
The lever that matters most is the data layer. On a commodity trading platform I architected at BCG, we used RAG to ground LLM responses in curated trading data. The safety did not come from the LLM. It came from the governed data underneath. The LLM was the interface to the platform. The platform was where access, freshness, and lineage were enforced. The model could only reason about what we gave it, and what we gave it was traceable.
The same pattern is showing up in the healthcare work I am leading now. We are designing a conversational AI layer on top of a governed Snowflake data foundation. The conversational layer is the easy part. The hard part, the part that makes it trustworthy, is the data architecture underneath. The semantic layer, the lineage, the access controls, the freshness metadata. That is the safety boundary.
That framing – the data platform as the safety boundary – shifts a lot of responsibility away from the model itself.
Intentionally. The model is a consumer of your data platform. It will do what you give it permission to do, with the data you give it access to. If the platform has weak access controls, the model inherits that weakness. If the data it retrieves is stale or ungoverned, the model will produce confident answers based on stale, ungoverned data. The safety work happens upstream. That’s where the leverage is.
Which elements of data governance do you consider critical for scaling AI safely in an enterprise environment?
Four elements: lineage, access control, data quality, and definition consistency.
Lineage tells you where every piece of data came from and how it was transformed. Without it, you cannot defend an AI decision in an audit, and you cannot trace a wrong answer back to its root cause. On the healthcare work I am leading now, we built end-to-end lineage from dashboard widget back through the reporting view, through the curated layer, back to the raw Salesforce field. Five layers. Every metric on every dashboard can be traced. That is not an optional feature. It is what makes the model defensible at handoff and at audit.
Access control determines who and what can read the data, enforced through role-based controls at the platform layer rather than inside individual applications.
Data quality has to operate at three levels. Structural quality means the pipes are running and the schemas match. Semantic quality means the data means what people think it means. Contextual quality means it is appropriate for the specific use case the model is being asked to support. Most governance programs stop at structural. AI demands all three. Stale data is structurally valid and contextually useless.
Definition consistency is the one people consistently underestimate. On a recent engagement, the same metric, implant count was being computed three different ways by three different teams using two different source systems. The model trained on one definition. The business reported against another. The reconciliation report flagged a 15% variance and the program nearly stalled trying to explain it. The data was not wrong. The definitions had diverged over years of independent evolution. A semantic layer with one enforced definition for each entity that matters is the foundation that makes everything downstream governable.
With the emergence of RAG, enterprise LLMs, and agentic AI systems, companies now have the ability to work directly with internal data through AI. How should such systems be designed to maintain control, auditability, and data quality – especially when AI agents are making decisions autonomously?
RAG is often described as a way to ground LLMs in enterprise data. The more accurate framing is that RAG moves the risk surface from the model to the retrieval layer. The model is no longer the black box. The retrieval layer is. If it pulls stale, ungoverned, or incorrectly classified data, the LLM will produce a confident wrong answer and the user will not know.
Three design principles matter for RAG systems. First, every chunk in the vector store should be tagged with source, owner, freshness, and access classification, treated as first-class metadata, not optional. Second, retrievals must be logged and replayable. If a user gets an answer today, you should be able to reproduce six months later exactly which chunks were retrieved and which version of the source data they came from. Third, the system has to surface what it does not know rather than fill the gap with plausible-sounding generation.
For agentic systems, the bar moves higher. An agent does not just answer a question. It takes action. It moves data, triggers decisions, modifies records. Every action needs an audit trail showing what data the agent had access to, what it inferred, and what it did. The teams I see succeeding here are building an agent registry, every agent in production, its capabilities, its permissions, its data scope alongside the model registry. The teams that have not started that work are accumulating risk faster than they realize.
The piece most teams miss is replayability. They invest heavily in making inference fast. They invest almost nothing in making it auditable. In a regulated context, a fast decision you cannot defend is worse than a slower decision you can.
What’s the piece most teams are missing when they get this wrong?
Replayability. The ability to reproduce, after the fact, exactly what data an agent or RAG system used to make a decision – which chunks were retrieved, which version of the source, what the system knew and didn’t know at that moment. Teams invest heavily in making inferences fast. They invest almost nothing in making it auditable. In a regulated context, a fast decision you cannot defend is worse than a slower decision you can.
In your experience, what marks the transition point when an AI solution stops being an experiment and becomes a full-fledged enterprise system? Can you walk through a specific example?
The transition happens when the AI solution starts being used to make decisions that have consequences if it fails. Not when it goes to production. Not when the team calls it live. When a business process depends on it, and the alternative is a manual rollback that costs real money or real time.
Three signals indicate the transition is genuine. Someone owns it operationally, with a runbook and an on-call rotation. The system is monitored beyond uptime data quality, drift, latency, and cost have alerts. The model has a defined retraining cadence and a rollback path if the new version performs worse than the old one.
A concrete example. On a propane delivery optimization platform I led at BCG, the transition to enterprise-grade happened the day we built data quality guardrails for the sensor data. Before, the model ran on whatever data arrived including sensors that had flatlined and were reporting full tanks. The model optimized routes based on that, trucks did not show up, and customers ran out of propane. The model was doing exactly what it was supposed to do with the data it was given. The data was wrong. After we built the guardrails, the model rejected inputs that failed quality checks and the operations team got an alert instead of a customer complaint. We projected that change alone would eliminate 50-60% of data-driven incidents.
The model did not change. The system around it became serious. That is the transition. Not when the model improves. When the operating model around it does.
Scaling AI in regulated environments isn’t just an architecture problem – it’s an adoption problem. How do you approach getting engineering teams to actually use AI in their daily workflows, not just in pilots?
The framing that works is starting from the decision, not the data. Most AI projects start with “what data do we have, what can we build with it?” That leads to demos that look impressive and never get used.
The better question is “what decision are people making right now, manually or painfully, that this could support?” On an energy engagement, instead of starting from the data, we spent time with the supply planners and mapped their actual morning workflow. Where did they spend time hunting for information? Where were they making judgment calls that could be better informed? That shaped the architecture and made adoption a natural outcome rather than a change management exercise.
Knowledge transfer is the other half. Architecture gets you to day one. Adoption decides whether there is a day one hundred. On every engagement where I have seen AI actually land, the team built runbooks, on-call routines, and enough hands-on practice for the client team to run, question, and improve the system after the engagement ended. The teams that skip that step end up with platforms that get used for three months and then quietly stop running because no one inside the organization knows how to operate them.
If a company in a regulated industry is just beginning its AI journey today, which two or three engineering or architectural decisions would you consider critical at the start to avoid issues at scale?
Three decisions, in priority order.
First, invest in the data foundation before the model. The teams I have seen succeed at AI in regulated industries built a governed data layer first lineage, access control, definitions, quality. The teams that struggled started with a model and tried to retrofit governance later. Retrofitting is two to three times more expensive than building it in, and sometimes impossible because the audit trail for past decisions no longer exists.
Second, decide on a semantic layer early and enforce it. One definition of customer, claim, patient, revenue, whatever the key entities are for that business. This is the single most consequential governance decision an organization will make for AI. Without it, every new use case re-litigates definitions and every output is subject to challenge. The healthcare engagement I am leading right now has 82 business metrics mapped to a single source-to-target specification, which becomes the contract between data engineering, business reporting, AI consumers, and the auditors. That mapping is the most valuable artifact we will produce.
Third, build a pilot environment that mirrors production controls. The failure mode I see constantly is pilots that succeed in dev because they bypassed governance, then fail in production because the controls reveal the data was never actually ready. Test under the same conditions you will deploy under.
There’s a pattern you’d specifically warn against?
The use-case-first approach. Picking three AI use cases in parallel and trying to build them before the foundation exists. It produces three half-built systems, none of which scale, and a governance debt that compounds with each new one. Every new pilot multiplies the inconsistency, because each one made different shortcut decisions about definitions, access, and quality.
The foundation is not a delay before getting started. The foundation is what getting started actually looks like in a regulated environment. The teams that understand this ship slower in the first quarter and dramatically faster after that. The teams that skip it look fast for a few months and then spend the next two years untangling.
In your view, what will a “mature” AI infrastructure in regulated industries look like in the next few years? Will it be an evolution of data platforms, or will a distinct layer of AI governance and orchestration emerge?
Both, but the foundation stays the data platform. Lineage, access control, quality, and definitions become more automated and more granular, but the fundamental shape raw, curated, serving, with governance enforced at the boundaries does not change. What changes is what sits on top.
A distinct AI governance and orchestration layer is going to emerge, with three parts. A model registry that tracks every version, its training data, its validation results, its production performance. An agent orchestration layer where data contracts, formal agreements about quality, freshness, and behavior are enforced in CI/CD pipelines rather than committee documents that nobody reads. And a continuous evaluation layer that checks AI outputs the way data quality frameworks check data, running constantly, surfacing drift before users notice it.
The pattern that wins is governance at the speed of operations. Manual audits and quarterly reviews cannot keep up with systems that make thousands of decisions an hour. The leading organizations are building governance into the operating environment itself, automated checks, real-time monitoring, escalation paths, and evidence trails running alongside the models rather than in a separate compliance workflow.
The data engineers and platform leaders who understand this shift are in the best position to lead it. AI is not a layer on top of the data platform. It is the next consumer of the data platform, and it has the highest demands of any consumer the platform has ever served. The teams that build for that consumer succeed. The teams that bolt AI onto whatever platform they already have spend the rest of the decade reconciling.
