Why is health plan data privacy now an AI question?
For years, the privacy question for an employer health plan was narrow: who is allowed to see a claim. The newer question is what a vendor’s AI system learned from thousands of claims, and whether that knowledge lives on inside a model after the service agreement ends. A filing cabinet can be emptied. A trained model does not forget the same way.
Insurers and their vendors have moved fast. A National Association of Insurance Commissioners survey of 93 health insurers in 16 states, released May 20, 2025, found that 84% already use AI or machine learning in some capacity. Every one of those tools was trained on data, and a growing share of that data started as someone’s health plan claim.
Most benefits leaders can explain their deductible and their stop-loss attachment point. Far fewer can say what their third party administrator, pharmacy benefit manager or wellness vendor does with member claims data inside an AI pipeline, or whether anyone agreed to it in writing.
None of this makes AI the problem. Used well, AI can speed up claims turnaround and catch billing errors a human reviewer would miss. Our overview of AI in employee benefits covers where it helps. The problem is agreeing to how a vendor’s AI touches plan data without asking a single question first.
Is your health plan a HIPAA covered entity?
Yes, in most cases. Under 45 CFR 160.103, a health plan is a covered entity, and a group health plan, insured or self-insured, is a health plan if it has 50 or more participants or is administered by someone other than the employer.
That means the privacy and security rules that apply to a hospital also apply to the plan your company sponsors. When a vendor asks to use claims data beyond the task the plan hired it for, the plan holds the compliance risk, not just the vendor.
Penalties are real. The inflation-adjusted HIPAA penalty table in 45 CFR 102.3 sets a calendar-year cap of $2,190,294 for violations of an identical provision, in its 2025 adjustment.
Ethos Benefits is itself a HIPAA business associate of the plans it advises, so we read these agreements from both sides of the table.
What does HIPAA allow when AI is involved?
HIPAA has no special carve-out for artificial intelligence. The same framework applies:
- Covered entities and business associates may use protected health information for treatment, payment and health care operations.
- Uses outside those purposes generally need the member’s authorization, or the data has to be de-identified first.
- The minimum necessary standard applies throughout.
Three parts of the rule matter most when a vendor wants to train or run AI on your plan’s data.
Is a business associate agreement a blanket yes?
No. Under 45 CFR 164.504(e), a business associate contract must establish the permitted and required uses and disclosures of protected health information, and it may not authorize the vendor to do anything the plan itself could not do.
Training a general-purpose AI model is a different use from processing a claim for payment. If the agreement does not name model training, fine-tuning or evaluation, a vendor that trains on your plan’s data anyway is operating outside the agreement you signed, whatever its sales team implied.
The same rule requires the vendor, at termination, to return or destroy the protected health information it still holds where feasible. Most agreements say nothing about a model that was trained on that information. That gap is worth closing in writing.
What makes training data de-identified?
HIPAA offers two paths, both in 45 CFR 164.514(b), and explained in HHS’s de-identification guidance:
| Method | How it works | What to ask the vendor |
|---|---|---|
| Safe Harbor | Remove 18 listed identifiers, from names and addresses to device numbers and biometric data, with no actual knowledge that the rest could identify someone | Show the identifier list you apply and how you test it |
| Expert Determination | A qualified expert finds the risk of re-identification is very small and documents the methods and results | Who the expert was, when, and the written determination |
Once data genuinely meets one of these standards, it is no longer protected health information. That is exactly why vendors lean on the word “de-identified.” Ask for the documentation, not the adjective.
Does minimum necessary still apply to AI features?
Yes. Under 45 CFR 164.502(b), covered entities and business associates must make reasonable efforts to limit protected health information to the minimum necessary for the purpose.
A chatbot that answers benefit questions does not need five years of a member’s full claims history. A model trained broadly across an entire book of claims for convenience, instead of scoped to a defined administrative task, is likely using more data than the task requires.
How are regulators treating AI and plan data?
On January 6, 2025, HHS published a proposed update to the HIPAA Security Rule. In it, HHS states that ePHI “in AI training data, prediction models, and algorithm data” maintained by a regulated entity for covered functions is protected by the HIPAA Rules. HHS also warns that training AI models on patient data could result in impermissible uses and disclosures. The proposal had not been finalized when this article was updated, but it answers the argument that AI training sits outside the rule.
State insurance regulators are moving too. In 2023, the NAIC adopted a model bulletin on insurers’ use of AI, and states have been adopting it since. Plan sponsors now have to track this at the federal and state level, not only on the day a contract is signed. Tracking those changes is part of ongoing health plan compliance.
Why is this also a fiduciary question?
A perfectly HIPAA-compliant arrangement can still be an ERISA problem. HIPAA asks whether a use of data is permitted. ERISA asks whether a prudent fiduciary, acting solely in the interest of participants, would have agreed to it.
The Department of Labor puts the vendor side plainly in Meeting Your Fiduciary Responsibilities: “Hiring a service provider in and of itself is a fiduciary function.” It also says an employer should establish and follow a formal review process at reasonable intervals. That duty does not pause because the vendor uses AI.
A signed BAA answers the HIPAA question. It does not answer whether a prudent fiduciary would have agreed to those data terms in the first place.
The vendor-monitoring theory is not new. It is the theory behind the wave of ERISA health plan lawsuits over pharmacy benefit contracts. Those cases are about drug pricing, not AI, but the principle carries over: signing a vendor agreement without scrutinizing its terms, and never revisiting it, is hard to defend.
What should you ask before you sign?
These five questions need no technical background, only a willingness to ask and to insist on a written answer:
- Does the BAA name model training as a permitted use? If training, fine-tuning and evaluation are not spelled out, assume they are not covered.
- How is the data de-identified before it reaches a training set? Ask which method applies, Safe Harbor or Expert Determination, and ask for the documentation.
- Is the vendor barred from using your plan’s data to train models for other clients? Without that clause, your claims experience can improve someone else’s product.
- What happens to the data, and to any resulting model, at termination? Return or destruction terms that cover raw data often say nothing about the model.
- Who on your benefits committee reviews this, and how often? A one-time review at signing is not ongoing monitoring.
Add these to every vendor scorecard. Vendor management is where they belong, next to service levels and performance guarantees, and the ERISA compliance checklist covers the plan documents they connect to.
What does this look like at renewal?
Picture a mid-size employer heading into its annual TPA renewal. Buried deep in the renewal packet is an updated “service enhancement” clause granting the TPA rights to use claims data to improve its AI-driven utilization review tools, for this plan and others. Nobody flags it, because nobody on the committee was looking for it.
A fiduciary-first review catches that clause before signing. The committee asks the TPA to name the permitted AI uses in the BAA, confirm which de-identification method applies, and strike the language that lets the plan’s data train models for other clients.
The renewal still closes on time. The difference is a written record the committee can point to if anyone asks how the decision was made. For a self-funded plan, the plan’s data is also the plan’s money; see how one self-funded employer runs its plan on its own claims data in the Hendry Marine case study.
This article is general information, not legal advice. Plan sponsors should consult ERISA counsel and a HIPAA privacy professional before making decisions about vendor contracts.