Machine Learning Model Security

C H A P T E R 0 2 Build an inventory that survives a rollback The security unit is a release bundle. It includes weights, tokenizer, configuration, preprocessing, postprocessing, dependencies and evaluation evidence. A weight digest alone does not identify the behavior that ran. Different preprocessing can change the effective input; a changed label map can reverse the meaning of the same numeric prediction. Begin the inventory with the production endpoint and work backward. Record which release is active, which deployment identity selected it and which artifact files it loads. Record whether dependencies are resolved during startup. A worker that fetches “latest” is creating a new release without the release process, even if the original container was reviewed. Bind inputs to an output SLSA provenance distinguishes the produced artifact from the build information supporting it. Use that distinction for model releases: an attestation should identify the output digest and enough build inputs to reproduce or investigate it. [S05] The organization must still decide which builder identities, source repositories and approval policies are acceptable. For an ML release, maintain the dataset snapshot, training code revision, dependency image digest, training configuration, base-model revision and evaluation manifest. The record should identify sensitive inputs without copying those inputs into a broadly readable metadata store. A protected dataset identifier can be preferable to a raw storage URL that reveals a customer name. Machine Learning Model Security SECURITY FIELD GUIDES aisecurityinterviews.com / field guides 6

C H A P T E R 0 8 Evaluate robustness against a stated threat Robustness is always relative to a threat model. A classifier that tolerates one bounded image perturbation has not been shown safe against altered metadata, distribution shift or poisoned training data. The test report should identify the input domain, attacker knowledge, allowed changes, query budget, success criterion and the exact candidate bundle. Distinguish a naturally occurring error from a deliberately selected input. Both may harm users, but the evaluation methods and controls differ. If an attacker can repeatedly observe outcomes and adapt, a single fixed benchmark may underestimate the risk. If the real attacker cannot change a field, allowing that field to vary can overstate practical risk. FIGURE 06 / CONTROL MODEL A robustness number needs its surrounding contract Threat set Which changes are permitted? Adversary view What can be observed or queried? Success rule Which application outcome counts? ↓ Keep the setup fixed while comparing candidates ↓ Candidate Model + preprocessing + runtime Evaluation Independent methods and held-out cases Decision Utility, harm and residual uncertainty A score without a threat set cannot establish a deployment guarantee. The same model may be robust under one constraint and brittle under another. Report the boundary of the claim. Why one test can mislead Athalye and colleagues showed that difficult or misleading gradients can make a defense appear more robust than it is. [S10] The broader review lesson is that a failed evaluator may be evidence about the evaluator rather than the defended system. A preprocessing step that disrupts one test method still needs independent scrutiny. Machine Learning Model Security SECURITY FIELD GUIDES aisecurityinterviews.com / field guides 18

C H A P T E R 1 5 Match hardware isolation to the adversary Hardware controls answer specific questions about co-tenants, host operators, memory and device access. They do not produce one universal “secure GPU” state. Start by naming the party from whom weights or inputs must be protected, then inspect the actual deployment boundary. A dedicated GPU can reduce sharing but still runs under a host administrator. A partitioning feature may provide defined resource or fault isolation without protecting plaintext from privileged host software. Confidential computing can change that trust boundary when the supported hardware, firmware, software and attestation policy are correctly deployed. [S13] Attestation must influence a decision An attestation report is evidence to a verifier. A useful design checks expected measurements, security configuration, freshness and relevant platform status before releasing a decryption key or admitting a workload. Merely logging “attestation succeeded” without identifying the policy or key- release consequence is weak assurance. Keep the verifier and key-release policy outside the workload being evaluated. Otherwise a compromised workload can approve its own environment. Define how policy changes, revocations and verifier outages affect running and newly starting jobs. Availability choices should be explicit: refusing a new key release may be safer than silently bypassing verification. Machine Learning Model Security SECURITY FIELD GUIDES aisecurityinterviews.com / field guides 32