What Must Be in Place Before Using AI with Customer or Patient Data

Before AI processes customer or patient data subject to HIPAA, GDPR, CCPA or contractual restrictions, the organization must establish governance that defines permitted uses, assign clear accountability for privacy risk decisions, document its regulatory position in writing, ensure vendor contracts allocate liability appropriately, and implement technical controls that enforce those decisions. The gap most organizations face is not technical capability but executive ownership: no single leader is accountable for the outcome, and progress cannot be measured without a defined standard.

Why This Matters to the Business

When AI processes protected data without adequate preparation, the organization accepts regulatory exposure, contractual liability and reputational risk that leadership cannot quantify. HIPAA does not prohibit AI use with protected health information, but it requires covered entities and business associates to ensure that safeguards, access controls and breach notification procedures apply. GDPR requires lawful basis for processing, data protection by design, and in many cases data processing agreements with processors. CCPA grants California residents rights over their personal information and imposes obligations on businesses that sell or share it. Contracts with enterprise customers often impose stricter requirements than any statute.

The consequence of proceeding without clarity is not that AI cannot be used. It is that the organization cannot demonstrate compliance when asked, cannot measure whether controls are effective, and cannot assign accountability when something fails. Boards and executives are increasingly held personally accountable for privacy and security outcomes. A posture of good-faith effort is not a defense when the organization lacks documentation, assigned ownership or a method of verification.

What Must Be Documented and Decided

The NIST Privacy Framework provides a voluntary structure for identifying and managing privacy risk through enterprise risk management. It is not a checklist, but it describes the categories of decision and control that any organization processing personal data must address. The framework organizes privacy risk management into functions: Identify, Govern, Control, Communicate and Protect. Before AI is introduced, the organization must answer specific questions in each area.

Identify: Inventory and Classification

Leadership must know what categories of data will be processed, where it resides, and what legal or contractual restrictions apply. This requires a data inventory that distinguishes protected health information under HIPAA, personal data under GDPR, personal information under CCPA, and data subject to customer contracts. The inventory must identify data flows: where data originates, where it moves, who processes it, and where it is stored. Many organizations discover at this stage that they do not have a complete picture.

Govern: Policy and Accountability

The organization must document its position on permissible AI uses, who approves exceptions, and how risk decisions are escalated. This includes defining acceptable purposes for AI processing under the organization's legal basis, establishing approval workflows, and assigning accountability for ongoing compliance. Many organizations lack a written AI use policy, leaving each business unit to make its own risk decisions. That approach works until it fails visibly.

The NIST Cybersecurity Framework, now in version 2.0, emphasizes governance as a foundational element of cybersecurity risk management. While the Cybersecurity Framework addresses information security broadly and the Privacy Framework addresses privacy specifically, both emphasize that governance precedes implementation. An organization cannot implement controls it has not defined, and it cannot measure progress against standards it has not documented.

Control: Technical and Procedural Safeguards

Controls must enforce policy. HIPAA requires administrative, physical and technical safeguards that are reasonable and appropriate to the risk. GDPR requires security appropriate to the risk, including pseudonymization and encryption where appropriate. CCPA requires reasonable security procedures and practices. The FTC has repeatedly taken enforcement action under Section 5 of the FTC Act when companies fail to implement reasonable data security, particularly when they have made privacy promises in their policies.

For AI systems, this means access controls that limit who can submit data for processing, logging that records what data was processed and by whom, data minimization that restricts inputs to what is necessary, and contractual or technical measures that prevent the AI vendor from retaining or training on protected data. Many AI vendors' standard terms permit retention and model training. The organization must negotiate restrictions, verify them technically where possible, and document the residual risk where verification is not feasible.

Communicate: Transparency and Rights

GDPR requires that individuals be informed when their data is processed, including by automated means. CCPA grants California residents the right to know what personal information is collected and how it is used. HIPAA requires covered entities to provide a notice of privacy practices. If AI processing changes how data is used, privacy notices may need updating. If the AI system makes decisions that affect individuals, additional transparency or human review may be required.

Protect: Breach Response and Incident Management

The FTC's Health Breach Notification Rule requires certain entities not covered by HIPAA to notify individuals and the FTC following a breach of unsecured health information. HIPAA requires breach notification within specific timeframes. GDPR requires notification to supervisory authorities within 72 hours of becoming aware of certain breaches. If an AI vendor is processing protected data and experiences a breach, the organization must know within timeframes that allow it to meet its own notification obligations. Contracts must require prompt notice, and the organization must have a defined process for evaluation and response.

Who Owns This and What Adequate Ownership Looks Like

Accountability for privacy risk in AI use typically falls across legal, compliance, information security, privacy, IT and business units. In practice, this diffusion of responsibility means no one is accountable for the complete outcome. Legal may review contracts but lack technical context to assess controls. IT may implement the integration but lack authority to define acceptable use. Privacy may issue policy but lack visibility into what is actually deployed. Business units may adopt tools independently.

Adequate ownership requires a single executive point of accountability with the authority to define standards, approve or reject use cases, and require evidence of compliance. In organizations with a chief information security officer, chief privacy officer or dedicated data protection officer, that role may own the outcome. In many regulated organizations, however, these roles either do not exist or lack the authority and capacity to govern AI use across the business. [Virtual CISO (vCISO) leadership](/vciso/) provides executive accountability for cybersecurity and privacy strategy, governance and risk decisions, filling this gap without requiring a permanent executive hire.

Ownership includes defining the standard, assigning responsibility for each component, establishing reporting cadence, and making risk acceptance decisions when controls cannot fully eliminate exposure. It does not mean the accountable executive performs every task. It means they ensure the work is done, conflicts are resolved, and the board or CEO receives accurate reporting on the organization's position.

Practical Next Steps for Leadership

Organizations should begin with an inventory of AI use and planned use involving protected data. This includes both sanctioned tools and shadow IT. For each use case, document the data categories involved, the legal or contractual framework that applies, and the current state of controls. Identify gaps between current practice and documented policy, and between policy and regulatory requirements.

Assign clear accountability for the AI governance outcome. If no single leader has the authority and capacity to own this, leadership must either delegate that authority, create the role, or engage external executive support through [vCISO advisory](/vciso/). Draft or update AI use policy to define permissible purposes, approval requirements and prohibited uses. Review vendor contracts for data processing terms, liability allocation, breach notification obligations and audit rights. Where contracts are deficient, negotiate amendments or accept documented risk.

Implement baseline technical controls: access restrictions, logging, data minimization at input, and contractual or technical prohibitions on vendor retention and training. Establish a process for evaluating new AI use cases before deployment, including legal review, privacy impact assessment where required, and technical validation of controls. Verify that incident response and breach notification procedures account for AI vendor incidents.

Report progress to the board or executive leadership using a defined standard. The NIST Privacy Framework and NIST Cybersecurity Framework provide structure for this reporting. Progress should be measured against documented outcomes, not activity. The question is not how many policies were written or meetings held, but whether the organization can demonstrate, with evidence, that it knows what data AI processes, has authorized those uses, has enforced controls, and can detect and respond to failure.

If your organization is introducing or expanding AI use with customer or patient data and you need clarity on accountability, standards and governance, Heights Consulting Group offers confidential consultations with executives facing this question. These are strategy conversations, not sales calls, and there is no obligation. Contact us to discuss your specific situation.