OpenAI and Anthropic beef up customer data confidentiality for frontier models, but details lacking
It looks like I’m not isolated in thinking a lot about ways in which AI services can be provided not only securely, but also confidentially—even vis-à-vis the service provider (see my AI privilege and ChatGPT encryption, Trustworthy privacy for AI: Apple’s and Meta’s TEEs, and Meta AI’s Incognito Chat: a new bar for confidential AI). One thing you may have noticed in my commentary on confidential AI solutions is that I didn’t discuss the two frontier AI labs.
This is because, until now, I was only aware of third-party, indirect ways of accessing models like Claude or GPT with confidential computing guarantees (e.g. via Amazon Bedrock Mantle “Zero Operator Access”). What the AI labs offer directly is Zero Data Retention (ZDR), which is a promise that they do not retain customer content such as prompts and model outputs. The difference is that confidential computing can mean no service provider access to data being processed, while ZDR means only that the service provider does not keep the data after they’re done processing it. However, and notably, Anthropic seemed to move in the opposite direction at least temporarily—when releasing its Mythos and Fable models, it disabled the ZDR option for those models, imposing 30-day data retention for customer data.
This may be changing as both OpenAI and Anthropic just announced beefed-up versions of their customer data confidentiality options for enterprise customers using even the frontier models. To be clear, neither company provided much in the way of technical detail of how they’re going to implement this. But I hope this will involve properly configured confidential computing solutions with Trusted Execution Environment (TEE) hardware. If not, the confidentiality guarantees—though real—will not robustly cover data while it is processed for inference. And this should make customers worried about not only old risks like compelled government access, but also new ones and especially, the AI-model-as-insider threat that is no longer theoretical (following OpenAI’s and other AI security incidents).
Distinguishing confidentiality guarantees: “no personnel access” vs no service provider access
Perhaps I’m too enthusiastic about confidential computing and I’m reading too much into carefully worded announcements from the AI labs. So let me explain what I think is the key threat model issue not addressed explicitly in the announcements.
I think that a serious AI service confidentiality guarantee should cover the risk of highly skilled insider threats targeting not just data at rest but also data being processed during inference (including internal AI model-originated threats).
OpenAI’s announcement says that there will be no “OpenAI personnel access.” Regarding technical measures, it talks about customer-held keys to data stored with OpenAI and alternatively data being stored on customer infrastructure. But nothing about the security of the data while being processed (prompts, model outputs, metadata) and no mention of “OpenAI cannot access it.” In other words, it looks like OpenAI includes their automated processing stack within the trust boundary (what customers need to trust), but only excludes humans from it (how such a distinction can be maintained will, I’m sure, be subject to healthy debate).
Anthropic stresses that it will not have custody of customer data, while safety monitoring will be done by automated systems provided by Anthropic. Again, nothing explicitly about the security of data being processed (including the processing for safety monitoring: who will guarantee that Anthropic’s monitoring software is safe and does only what is advertised?).
I want to be clear: those measures could address some important classes of threats. Neither a compromised internal AI system nor a court subpoena directed at the lab should mean retrospective access to stored data merely by targeting the labs directly (of course, customers could separately be hacked for their stored data and whether the guarantee holds regarding the providers’ systems depends on many implementation details).
But if we take seriously a risk of someone (a human insider, an AI, a government agency) establishing a live and prospective interception at the inference servers or at the API gateway, then the guarantees about stored data are not sufficient.
What do we know?
OpenAI
OpenAI calls its new offering “Private Safety Processing” and describes it as a preview, which it talks about publicly now because clients have been telling them “they need predictability about how their content will be protected as AI systems become more capable.” As we can see from the diagram it published, the key promise is “no OpenAI personnel access.”
OpenAI describes this in the following way:
Private Safety Processing utilizes customer content regardless of where it is stored—whether in infrastructure customers control (ZDR deployments) or in storage provided by OpenAI. With OpenAI-provided storage, customer content is encrypted using keys controlled by the customer. OpenAI personnel do not have a copy of those keys, so they cannot access the underlying content.
When a risk is identified, OpenAI receives a narrowly defined signal indicating the type of activity involved, similar to our existing safety systems today. That signal can be used to determine whether enforcement is necessary. OpenAI personnel do not receive access to the customer content even when it is flagged.
This differs slightly from Meta’s no-escalation design which I covered in May, but also likely from an approach OpenAI apparently contemplated before (at least implying escalation that transfers content). As I said then:
OpenAI’s recent statement about its planned client-side encryption made this stack explicit: “fully automated systems to detect safety issues,” with human review reserved for “serious misuse and critical risks — such as threats to someone’s life, plans to harm others, or cybersecurity threats.” That escalation step is the part Meta does not appear to be reproducing inside Incognito Chat. Doing so would require an exit channel from the TEE — and the no-access promise would no longer hold.
I think Meta’s call here is the right one. Yes, model-level guardrails are imperfect, and motivated bad actors can sometimes work around them — just as they can work around classifier stacks. But the honesty of the model-level approach is its strength: a refusal that happens inside the TEE creates nothing that leaves the TEE. The moment a flagging mechanism is bolted onto this architecture, the user faces a pair of questions they cannot answer for themselves: which kinds of content trigger a flag, and what happens to that content once it crosses the boundary of the confidential environment?
Looking carefully at what OpenAI is saying now, its new approach to this problem is to use a mechanism (presumably an LLM) to only flag types of risks, but without sending any content. My first reaction is that this sounds like an elegant solution, but I would like to know more about the consequences that OpenAI plans to attach to flagging (we do know from the announcement about the possibility of “appeal” - see below). Notably, there is an exception to the confidentiality guarantee:
Like other frontier model providers, OpenAI is required by law to report apparent child sexual abuse material (CSAM). Images flagged for potential CSAM will continue to be retained for manual review and reporting purposes, even in Zero Data Retention deployments, as they are today.
This may sound reasonable and innocuous, but it is a serious weakening of the guarantee. And it does bring us back to the two questions I posed in my May text: how can the clients know in advance what kind of content will be flagged (let’s be honest: the CSAM detection tools are not, and probably cannot be, perfect) and what consequences will attach to that. I can imagine that some enterprise customers will be concerned about the very existence of the mechanism for transfer of user content.
Anthropic
As to Anthropic, we know even less. The most significant thing is probably that Anthropic will offer an option to at least self-custody data during the compulsory 30-day data retention it imposed for its state-of-the-art models (Mythos and Fable). According to Anthropic’s Sholto Douglas writing on X:
- Your data sits in infrastructure you own and control, and safeguards/monitoring is done via automated systems we provide to you
- We’ve been working on this for months with >100 customers
- Recent events have shown frontier models are capable of executing sophisticated cyber attacks in coordinated agent swarms.
- Both us and OAI believe that the responsible way to provide models which have this level of capability and introduce this risk into the world is to be able to monitor the models at more than a single request basis, because anomalous activity is much easier to detect over hours or days of behaviour. It looks like we’ve arrived at both the same conclusion - and the same solution.
I read this as suggesting that enterprise customers will keep their agent session history/memory, and maybe even full agent execution environments on their own infrastructure (which may include cloud services), while inference will be done by Anthropic (or partners like Amazon Bedrock), hopefully with confidential computing guarantees but possibly not for now. Anthropic (together with the company now known as Irregular) has independently researched a confidential-inference architecture, although there is no evidence yet that the forthcoming system implements it.
Safeguards and monitoring - how will that work (both for Anthropic and OpenAI)
When it comes to the “safeguards/monitoring is done via automated systems we provide to you” part, perhaps this will be a contractual requirement for those customer-operated agent execution environments. Or, perhaps, Anthropic will adopt a technical guarantee: requiring customers to run agents in TEEs of their own, providing Anthropic with attestations that customers run Anthropic-accepted “safeguards/monitoring” code (and gating model access via attestations to systems running the code).
Given that Douglas says that OpenAI and Anthropic arrived at the same high-level solution, both options may also apply to OpenAI. We know from OpenAI’s announcement that, in its framework, customers can investigate alerts and choose to share relevant information if they want to appeal or clarify enforcement decisions. This implies that at least OpenAI envisages “enforcement decisions” that may be subject to “appeal.”
I’d be very keen to learn more. OpenAI promises a technical white paper in September and I guess we’ll soon hear more from Anthropic as well.
Practical difficulties with confidential computing
I don’t want to oversell the current state of confidential computing. Even the state-of-the-art hardware deployed in setups we now consider the best may itself eventually be breached. But this is not a reason not to use it, only to think carefully about what data we allow to be processed by others even with such technical guarantees (as always, defense in depth).
I also understand that it could be difficult for the AI labs to provide large-scale inference with confidential computing guarantees. For instance, Blackwell-generation (B200/B300) NVIDIA GPUs are documented to allow eight-GPU nodes to be passed through to one confidential VM, but with still-common older Hopper GPUs (H100/H200) the NVLink/NVSwitch interconnects are not encrypted weakening any confidential computing guarantee. It’s also a separate issue whether Google or Amazon could provide sufficient confidential computing infrastructure for the needs of AI labs on their own chips.
Realistically, the AI labs may not be able to provide confidential computing-enabled AI data processing for all frontier model customers who may want that now, but perhaps we could at least learn about a roadmap towards that.