Protecting Model Weights

TL;DR: Weights are a file, and the attacker needs one copy. Attack paths include stolen credentials and over-broad cloud roles, insiders, supply chain compromise of training or serving software, exposed storage, and the serving infrastructure where weights sit decrypted in memory. Defend by minimising copies and access (few systems, few people, just-in-time and multi-party access), encrypting with keys whose use is controlled and logged, restricting and monitoring egress so large transfers are blocked or noticed, hardening serving, and running an insider-risk programme. RAND's 2024 report "Securing AI Model Weights" maps defences to attacker capability, from opportunistic criminals to state actors.

Why weights are a distinct asset

For a model provider, the weights are the product: the result of large compute and data investment, reproducible by nobody else cheaply. For the most capable models, providers also treat theft as a safety risk, because stolen weights can be run without safeguards. Frontier labs publish security commitments tied to model capability; Anthropic, for example, activated its AI Safety Level 3 security standard for a model in May 2025, which includes measures against weight theft (checked November 2026).

Weights are also unusually easy to steal relative to their value: a single large file, copied once, is gone. There is no partial breach.

Where weights exist

Map every place weights live, because each is an exposure:

Attack paths

• External attackers with stolen credentials, phished engineers, or exploited internet-facing services leading to cloud access (IAM escalation).
• Insiders with legitimate access who copy weights, or are coerced or recruited.
• Supply chain: compromised dependencies in training or serving images that read and exfiltrate weights.
• Serving infrastructure: vulnerabilities in model servers, exposed management dashboards, or side channels in shared hardware.
• Model extraction: approximating a model's behaviour through its API, a different and slower threat covered under adversarial ML.

RAND's 2024 study identified dozens of attack vectors and defined five security levels (SL1 to SL5) by attacker capability; the higher levels assume nation-state operations and require measures well beyond typical enterprise security.

RAND's levels, with the numbers

The RAND report (Nevo and colleagues, 2024, Securing AI Model Weights) is the reference interviewers expect. It catalogued about 38 attack vectors in nine categories, roughly a dozen of them feasible only for state actors, and defined attacker capability in five operational capacity levels, each matched by a security level that can likely defeat it.

RAND's urgent recommendations read like a checklist: centralise copies, reduce who has access, harden the interfaces to weights, run an insider-threat programme, layer defences, red-team the protections and use confidential computing where it fits. In an interview, the useful move is to name the capacity level you are defending against, because that determines which controls are proportionate.

What a frontier lab's controls look like

When Anthropic activated its AI Safety Level 3 protections in May 2025, it described more than 100 security controls, including two-party authorisation for access to model weights, allowlisting of the binaries that may run, and preliminary controls on egress bandwidth from environments holding weights. It was explicit about scope: the threat model did not include nation-state attackers or sophisticated insiders. For systemic-risk general-purpose models in the EU, the GPAI Code of Practice (final in July 2025) includes a security commitment covering non-state external threats and insiders.

A real path to fine-tuned weights

In the ModeLeak research on Vertex AI (Unit 42, November 2024), a malicious model deployed in a project could reach other fine-tuned models and adapter layers stored in the project's buckets, even though exporting models through the console was disabled. Adapters are small and easy to overlook, but they carry the proprietary behaviour a company paid to create. They belong in the weights inventory.

Defences

Minimise copies and access. Fewer locations, fewer people. Use a single authoritative store, no persistent copies on developer machines, and access granted just in time with approval and expiry. For the most sensitive weights, require multi-party approval for bulk reads.

Control keys, not just encryption. Encrypt with keys in a KMS or HSM whose policy permits decryption only by specific serving and training identities, and log every use. Alert on decryption by anything else.

Egress controls and monitoring. Training and serving environments should not have open internet egress. Rate-limit and monitor outbound transfers; a multi-hundred-gigabyte upload from a serving node is an incident. Data-loss detection tuned for large binary transfers helps here.

Harden serving. No interactive shell access to inference nodes by default; immutable images; minimal software; patched model servers; management interfaces never exposed to the internet. See the inference stack lesson.

Confidential computing. Trusted execution environments with GPU support can keep weights encrypted in use and attest which code is running, reducing exposure to cloud operators and some host compromises. It narrows the attack surface; it does not remove software vulnerabilities inside the enclave.

Insider-risk programme. Background checks for privileged roles, separation of duties, monitoring of anomalous access, and clear policies. Many of the controls above (just-in-time access, multi-party approval, logging) serve insider risk too.

Confidential computing, precisely

Confidential computing for GPUs is often described loosely. On NVIDIA's H100, the details are:

• The GPU establishes an authenticated session with a CPU trusted execution environment (AMD SEV-SNP or Intel TDX), and data crossing PCIe passes through encrypted bounce buffers.
• The GPU's own high-bandwidth memory is not encrypted; protection there relies on the GPU's isolation and attestation.
• Confidential mode disables performance counters, and encryption limits CPU-to-GPU bandwidth to roughly 4 GB/s.
• Azure made confidential GPU VMs with H100s generally available in September 2024.

Design guidance has moved beyond the hardware. Anthropic and Pattern Labs' 2025 paper on confidential inference keeps weights encrypted except inside a small trusted component that loads the model and runs inference, and warns that not every accelerator fully supports confidential computing and that an insecure backup can defeat the whole design. Apple's Private Cloud Compute (2024) set five requirements: stateless computation, enforceable guarantees, no privileged runtime access (no remote shell even during an outage), non-targetability, and verifiable transparency through published software images. Apple extended it in 2026 to run some workloads on Google Cloud with NVIDIA confidential computing and Intel TDX under the same requirements.

What confidential computing does not stop: a compromised inference process inside the protected environment, and over-permissioned storage where weights sit at rest. It protects weights in use from the host and operator; identity and storage controls still protect them everywhere else.

Worked example: shrinking the exposure of one checkpoint

All figures are illustrative. A 140 GB checkpoint exists on three training clusters, in one registry, in caches on 12 serving nodes and in one backup: 17 copies.

Misconceptions interviewers listen for

• "Encrypt at rest and we are done." Weights are decrypted to be used, and backups are often the weakest copy.
• "Confidential computing encrypts the whole GPU." On H100, GPU memory is not encrypted; the protection covers the PCIe path and attestation.
• "SL5 is a product we can buy." RAND says it is not achievable today for internet-connected systems with off-the-shelf solutions.
• "ASL-3 covers insiders." Anthropic's stated ASL-3 threat model excludes sophisticated insiders.
• "Adapters are low value." They carry the fine-tuned behaviour; ModeLeak reached them specifically.

How this shows up in interviews

"Design controls that protect model weights from external attackers and insiders" is an expert-level question at labs and large providers. Strong answers begin by mapping where weights exist, separate attacker types, and layer controls: access minimisation, key control, egress, serving hardening, insider measures, and detection. Referencing RAND's security levels and being honest that state-level adversaries need extraordinary measures shows calibration.

Say it in 30 seconds: "Weights are one file, so the attacker needs one copy. I start by mapping every copy: training checkpoints, registry, serving nodes where they're decrypted, dev environments, backups. Then I minimise copies and access with just-in-time and multi-party approval, encrypt with KMS keys only serving and training identities can use, block and monitor egress, harden serving with no interactive access, and run insider-risk controls. RAND's security levels help match defences to the attacker."