There's a conversation happening in healthcare and finance boardrooms right now that goes something like this:
"We need to use AI. Our competitors are using AI. But we can't send patient data to OpenAI. So what do we do?"
Most companies are still waiting for an answer. A few have found it. And the ones who found it didn't solve a technology problem — they solved an operations and compliance problem that happened to have a technology solution.
The public cloud AI problem in regulated industries
When most people think about using AI in their business, they're thinking about OpenAI's API, Claude, Gemini — frontier models available via subscription. These are powerful tools. For most businesses, they're the right starting point.
But for healthcare, finance, legal, and government contracting, there's a fundamental constraint: these services require sending your data to someone else's infrastructure.
That might be fine for marketing copy. It is not fine for:
- Patient records, diagnoses, and treatment plans
- Financial transactions and account information
- Legal documents and privileged communications
- Government contract data with classification requirements
HIPAA, SOC 2, and related frameworks aren't just legal requirements — they represent a real risk. A breach isn't just a fine. It's patients who trusted you with sensitive information having that information exposed. It's the end of every client relationship whose data was involved.
But compliance is only the start of the problem. The deeper issue is control.
When your operational workflows depend on a third-party AI service, you don't control the model's behavior, update schedule, data retention policy, or what happens when the vendor changes their terms. You've built load-bearing infrastructure on someone else's foundation.
What I learned running healthcare infrastructure at scale
At Centene Corporation — one of the largest Medicaid managed care organizations in the country — I ran infrastructure for systems that touched millions of patient records. The operational discipline required to handle that data wasn't optional. Every integration had to be audited. Every data path had to be documented. Every system that touched PHI had to be controlled end-to-end.
That experience shapes how I think about AI deployment in regulated industries. The question isn't just "can we use AI?" It's "can we control every part of how that AI operates, what data it sees, and what it does with that data?" If the answer is no, you don't have an AI strategy — you have a liability.
The private AI option most companies don't know exists
The alternative to public cloud AI isn't "don't use AI." It's private AI deployment.
Over the last two years, the open-source model ecosystem has matured significantly. Capable language models can now be deployed on infrastructure you own or control — in your data center, in a private cloud, on-premise — with zero data leaving your environment.
This isn't a compromise. For the use cases that matter most in healthcare and finance — document processing, patient intake, prior authorization, financial analysis, report generation — private models perform at a level that's entirely sufficient.
The operational advantages go beyond compliance:
Speed. Data doesn't travel to an external API and back. Local inference is faster than cloud API calls for many workloads.
Cost. Once deployed, the marginal cost per query is essentially electricity. No per-token fees that compound at scale.
Control. You decide when the model gets updated. You decide what data it sees. You control every aspect of how the system behaves.
Institutional knowledge. Private AI can be tuned to your specific documents, policies, and workflows — making it far more useful for your context than a general-purpose model.
What a private AI deployment actually looks like operationally
The infrastructure is more accessible than most companies think, but the operational work is real.
For a healthcare group or financial services firm, a private deployment typically involves:
- Dedicated server capacity with appropriate GPU resources, in your environment
- An open-source base model configured for your specific use cases
- A retrieval pipeline that connects the model to your documents, policies, and data sources
- An integration layer connecting the AI to your existing systems — EHR, CRM, document management
- Audit logging for every query and response, supporting compliance review
The technology piece is solvable. The harder work is mapping your existing workflows, identifying which processes actually benefit from AI augmentation, and building the integrations that connect the AI to the systems your people already use. That's not an IT project — it's an operations project.
Done right, a private AI deployment for a mid-size healthcare or finance company can be operational in 8–14 weeks. What separates the successful deployments from the failed ones isn't the model selection. It's whether the operational groundwork was done first.
The question to ask before any AI vendor signs a contract
Before you sign any contract with an AI vendor, ask them this: where does our data go when we use your product?
If the answer is "to our cloud" or "to our third-party AI provider" — and you're in a regulated industry — that's a problem to solve before you move forward.
The companies building AI on infrastructure they control now will have a durable advantage. Their competitors who went the easy route will spend the next several years trying to unwind cloud dependencies they should never have created.