What Happens When Artificial Intelligence Learns a Client’s Secret

Machine learning has become a silent partner in modern law offices, turning vast archives into searchable insight. Yet as firms rely on these tools for drafting, research, and analysis, a deeper concern surfaces. Confidential material now moves through systems designed to retain patterns, raising questions about how well traditional privilege can survive inside a memory that never fades.

The Accidental Apprentice

Learning by accident” occurs when a system retains client information, when inputs are shared with third parties, or when data from one matter is used to train or refine a model used for others. Many commercial AI tools reserve rights to retain or analyze user prompts, while enterprise systems promise stronger controls.

OpenAI’s enterprise platform specifies that user data entered through enterprise and API products is excluded from model training. Microsoft’s documentation for Microsoft 365 Copilot makes the same assurance, stating that prompts and responses are not used to train foundation models. While these safeguards are encouraging, lawyers must verify them directly. Ethical obligations under Model Rule 1.6 require independent confirmation of vendor practices and oversight of how those systems are configured within the firm.

The Legal Consequences of Contamination

If privileged material is disclosed to an AI vendor or retained in a training corpus, waiver arguments may follow. The American Bar Association’s Formal Opinion 512 states that lawyers using generative AI must maintain competence, protect confidentiality under Rule 1.6, supervise vendors under Rule 5.3, and communicate with clients about potential risks.

State bars echo these expectations. The Florida Bar’s Ethics Opinion 24-1 instructs lawyers to confirm how AI tools store and process data before using them in matters involving confidential information. The North Carolina State Bar warns that entering client data into systems without adequate safeguards may violate confidentiality duties. The Texas Center for Legal Ethics similarly urges due diligence regarding vendor terms, data security, and staff training.

How Privilege Can Be Lost

Privilege law has long punished carelessness. Under Federal Rule of Evidence 502(b), privilege is preserved only if the holder took reasonable steps to prevent disclosure and acted promptly to correct errors. In Victor Stanley Inc. v. Creative Pipe Inc., 250 F.R.D. 251 (D. Md. 2008), and Harleysville Insurance Co. v. Holding Funeral Home Inc., No. 1:2015cv00057 (W.D. Va. 2017), courts found that inadvertent technical disclosures can still waive privilege when reasonable precautions are lacking. Uploading client data to an AI platform that retains or analyzes inputs could be treated similarly if adequate controls were not in place.

The ABA Formal Opinion 06-442 on metadata in electronic discovery reached a parallel conclusion nearly two decades ago: lawyers must prevent inadvertent disclosure through technology. The same reasoning now applies to generative models that can retain or reconstruct information.

Understanding State Variations in Privilege Law

While the Model Rules provide a framework for confidentiality and privilege, state law governs privilege determinations in litigation. Lawyers working across jurisdictions must understand that attorney-client privilege protections vary by state, particularly regarding inadvertent disclosure and the standards for waiver.

Some states apply strict liability approaches where any disclosure waives privilege, while others use intermediate balancing tests that weigh the reasonableness of precautions taken. Federal courts sitting in diversity apply the privilege law of the forum state, making venue selection and multi-jurisdictional practice additional considerations when using AI tools that may expose privileged information.

Client Communication and Consent

Lawyers have a continuing duty to keep clients informed when adopting technology that affects confidentiality. The American Bar Association’s Formal Opinion 512 advises that lawyers evaluate risks, explain material limitations, and secure informed client direction when using artificial intelligence tools. These obligations connect to Model Rules 1.1, 1.4, 1.6, and 5.3, which together require more than a general disclosure. Firms must explain how input data is handled, who has access, and what controls protect privileged content.

Parallel guidance appears at the state level. The Florida Bar’s Ethics Opinion 24-1 instructs lawyers to determine how a platform stores, retains, and trains on information before entering confidential material, and to obtain consent when exposure cannot be avoided. The North Carolina State Bar’s 2024 Formal Ethics Opinion 1 warns that transmitting client data to systems lacking adequate safeguards may violate Rule 1.6. These opinions turn general duties into specific engagement-letter discussions and internal checklists.

Vendor Due Diligence and Rule 5.3

Rule 5.3 requires lawyers to ensure that nonlawyer assistants and vendors act consistently with professional duties. In the AI context, that means reviewing each provider’s data retention policies, subcontracting practices, breach procedures, and audit rights. The Law Society of Alberta’s Generative AI Playbook recommends pre-adoption reviews of confidentiality, access controls, and data isolation. The Solicitors Regulation Authority in the United Kingdom similarly urges firms to evaluate training data, accuracy rates, and contract terms before deployment.

Effective supervision also requires internal alignment. Firms should maintain registers of approved AI tools, assign compliance officers to oversee configuration, and document vendor assurances. Contract clauses should provide audit rights, deletion obligations, encryption standards, and clear breach-notification triggers.

Questions to Ask Your AI Vendor About Privilege Protection

Before implementing any AI tool that processes client information, firms should obtain clear answers to these essential questions:

• Does the system train on user inputs, and if so, can enterprise customers opt out completely?
• Where is data processed and stored geographically, and does it remain within specified jurisdictional boundaries?
• What encryption standards apply to data at rest and in transit?
• Who within the vendor organization can access customer data, and under what circumstances?
• What subprocessors or third parties have access to input data?
• How long is data retained, and can customers request deletion of specific inputs?
• What audit rights do customers have to verify compliance with data-handling commitments?
• What breach notification procedures are in place, and what service-level agreements govern response times?
• Does the vendor maintain relevant security certifications such as SOC 2 Type II or ISO 27001?
• What happens to customer data if the vendor is acquired, goes out of business, or terminates the service?

The Economics of Containment

Data-governance failures now carry substantial financial implications. Industry guidance shows that professional liability insurers now ask law firms about their AI governance, tool-use policies, supervision mechanisms, and risk-management frameworks before renewal or quote. Firms without documented AI-use policies may face coverage gaps or heightened scrutiny from underwriters.

Regulators share that view. The National Association of Insurance Commissioners issued an AI Model Bulletin in 2023 recommending governance frameworks, transparency, and human oversight for automated systems. Although designed for insurers, its principles mirror those now expected of law firms. Prevention is proving less costly than remediation, particularly as clients begin requesting proof of AI controls in engagement letters.

Containment and Remediation

Once sensitive data enters model parameters, removal is uncertain. Research from IBM Research and the Association for Computational Linguistics 2024 conference shows that machine unlearning methods remain experimental and incomplete. The practical safeguard is prevention supported by auditable process control.

The NIST AI Risk Management Framework and its Generative AI Profile emphasize lifecycle documentation, provenance tracking, and access management. The ISO/IEC 42001 standard adds requirements for governance policies and continual review. Firms can align by restricting who can upload client data, using retrieval systems that do not train on inputs, and logging all AI interactions. Incident response should mirror cyber protocols: isolate, record, notify, and remediate.

Auditing, Logging, and Proof of Control

Verification is now a core element of AI governance. The NIST AI Risk Management Framework and its Generative AI Profile emphasize traceability through documented purpose statements, data-provenance logs, and continuous monitoring. The ISO/IEC 42001 management-system standard adds structured roles, written policies, and audit evidence demonstrating that controls function in practice.

Enterprise providers now offer configurations that support these expectations. OpenAI’s enterprise privacy documentation and Microsoft 365 Copilot privacy materials both state that enterprise inputs are excluded from training. Firms should verify these settings, restrict upload permissions, and review audit logs on a recurring schedule. Maintaining such records demonstrates Rule 5.3 supervision and satisfies insurer or client requests for evidence of data-handling compliance.

Regulatory and Enforcement Landscape

U.S. regulators now view AI data handling through the lens of consumer protection and privacy law. The Federal Trade Commission warns that misrepresenting how AI systems manage data or failing to secure sensitive information can constitute an unfair practice under Section 5 of the FTC Act. In 2024, the agency broadened its inquiries to include the use of private or legally protected data in model training.

States are moving as well. Colorado’s SB 24-205 requires companies deploying high-risk AI systems to maintain documented risk-management programs and disclose model limitations. California’s SB 1047 proposed similar obligations for developers. Together these frameworks expand liability beyond professional-ethics boards to include statutory enforcement.

At the federal level, the White House Executive Order on Safe, Secure, and Trustworthy AI directs agencies to develop safety and privacy standards. For law firms, this alignment between ethics rules and regulatory expectations means confidentiality breaches could invite both disciplinary and governmental action.

The International Perspective

The European Union’s AI Act 2024 establishes obligations for providers and users of high-risk systems, including documented risk assessments and human oversight. The European Data Protection Board has reinforced that generative-model use involving personal data must comply with GDPR’s data-minimization and purpose-limitation principles.

Canada’s Law Society of Alberta encourages structured vendor governance and informed client consent. Australia’s Legal Services Council has opened consultations on AI in legal practice, while Singapore’s AI Verify Framework sets voluntary transparency benchmarks relevant to multinational discovery work.

Practical Implementation Checklist

To preserve confidentiality and demonstrate compliance, firms can implement the following controls drawn from bar and regulatory guidance:

• Use enterprise AI tools that guarantee no training on user data, and confirm compliance through contract review and technical testing.
• Prohibit staff from pasting client content into consumer AI platforms. Provide secure internal sandboxes for experimentation.
• Add AI-specific clauses to vendor agreements covering data retention, training, subcontracting, audit access, and deletion rights.
• Integrate AI incidents into cybersecurity playbooks and follow ABA Formal Opinion 483 when breaches involve client data.
• Deliver annual training on AI risk and document attendance for insurer and compliance verification.
• Establish firm-wide AI governance policies that identify permissible tools, define supervision responsibilities, and require verification before client use.
• Maintain detailed logs of AI tool usage, including what data was processed, by whom, and for what purpose.
• Conduct periodic audits of vendor compliance with contractual data-protection commitments.
• Review engagement letters to ensure clients are informed about AI use where substantive work is delegated to AI tools.
• Designate an AI governance officer responsible for tool evaluation, policy updates, and incident response coordination.

From Confidentiality to AI Hygiene

Confidentiality has shifted from a static rule to an operational system. Firms that record their AI lifecycle—procurement, configuration, use, and review—demonstrate competence under Model Rule 1.1 and accountability under Model Rule 5.3. The same rigor once applied to trust accounting now applies to digital governance.

This culture of AI hygiene includes access restrictions, labeling of sensitive material, and continuous monitoring. Such practices maintain privilege, protect clients, and align with both bar ethics and emerging data-protection regimes. The duty is unchanged, yet its execution has become technical as well as ethical.

Sources

• American Bar Association: Formal Opinion 483 (Data Breaches and Lawyers’ Duties) (2018)
• American Bar Association: Formal Opinion 512 on Generative AI Tools (July 29, 2024)
• Association for Computational Linguistics: “Machine Unlearning: A Survey” (ACL 2024)
• AI Verify Foundation (Singapore): AI Verify Framework (2023)
• California SB 1047 (Safe and Secure Innovation for Frontier Artificial Intelligence Models Act) (2024)
• Colorado SB 24-205 (Artificial Intelligence Act) (2024)
• European Data Protection Board: Opinion 28/2024 on Certain Data Protection Aspects Related to the Processing of Personal Data in the Context of AI Models (2024)
• European Union: Artificial Intelligence Act (Regulation 2024/1689) (2024)
• Federal Rules of Evidence: Rule 502 (Attorney-Client Privilege and Work Product; Limitations on Waiver)
• Florida Bar: Ethics Opinion 24-1 on Generative AI (January 19, 2024)
• Harleysville Insurance Co. v. Holding Funeral Home Inc., No. 1:2015cv00057 (W.D. Va. Feb. 9, 2017)
• IBM Research: “Machine Unlearning for LLMs” (October 2024)
• ISO/IEC 42001:2023 Artificial Intelligence Management System Standard
• Law Society of Alberta: “Generative AI Playbook” (2024)
• Legal Services Council (Australia): Consultation on Generative AI in Legal Practice (2024)
• Lockton UK: “Law Firm AI Insurance Considerations” (2024)
• Microsoft 365 Copilot: “Data, Privacy, and Security for Microsoft 365 Copilot” (2025)
• National Association of Insurance Commissioners: AI Model Bulletin (December 2023)
• NIST: AI Risk Management Framework 1.0 (2023)
• NIST: Generative AI Profile (NIST AI 600-1) (2024)
• North Carolina State Bar: 2024 Formal Ethics Opinion 1 on Artificial Intelligence
• OpenAI: “Enterprise Privacy and Data Controls” (2025)
• Solicitors Regulation Authority (UK): “Artificial Intelligence in the Legal Market” (2023)
• Texas Center for Legal Ethics: Opinion 705 on Generative AI (2025)
• White House: Executive Order on Safe, Secure, and Trustworthy Artificial Intelligence (October 30, 2023)

This article was prepared for educational and informational purposes only. It does not constitute legal advice and should not be relied upon as such. All cases, statutes, bar opinions, and sources cited are publicly available through official publications and reputable outlets. Readers should consult professional counsel for specific legal or compliance questions related to AI use.

See also: Navigating The Transparency Paradox in AI Regulation