TL;DR: Treat models, datasets and their companion files as untrusted dependencies. Route every one through a single intake point (an internal registry or proxy): verify provenance (publisher, pinned revision, hashes, signatures), enforce safe formats and no unreviewed remote code, scan, then evaluate in isolation for behaviour and backdoor indicators. Store approved artifacts immutably by hash with an AI-BOM entry. Deploy only registry artifacts whose hash and provenance match policy, sign internally trained models, and monitor in production. Plan the incident path: lineage to find every system affected by a bad model.
How to approach it
Lay out the stages a model passes through (discovery, download, evaluation, fine-tuning, packaging, deployment, monitoring) and put controls at each, noting which are deterministic and which are probabilistic. Name the threats first so each control has a reason.
A strong answer
Public hub
models, datasets
Intake registry
single entry point
Verify
publisher, hashes, sigs
Formats + scan
no remote code
Deploy pinned
by digest
Ongoing
Inventory and re-evaluation
Threats. Malicious model files (pickle code execution, remote code), tampered artifacts on mirrors or hubs, typosquatted or impersonated model repositories, backdoored or poisoned weights, poisoned fine-tuning data, compromised training or build environments, and stale models with known vulnerabilities in their serving stack.
Controls by stage.
| Stage | Control | Type |
|---|---|---|
| Discovery | Allowlist of approved publishers; require organisation-verified repositories | Deterministic |
| Download | Only through an internal proxy or registry; pin by commit and hash | Deterministic |
| Format | Safetensors (or other data-only formats); weights_only=True; no trust_remote_code without review and pinning | Deterministic |
| Verification | Check publisher signatures (model signing with Sigstore) and hashes | Deterministic |
| Scanning | Pickle opcode scanning, unexpected files, licence checks | Heuristic |
| Evaluation | Isolated sandbox with no credentials or egress; behavioural tests, safety and security evals, trigger probing | Probabilistic |
| Storage | Immutable registry keyed by hash; AI-BOM with lineage | Deterministic |
| Fine-tuning | Data provenance and review; training jobs in isolated, least-privilege environments; provenance for outputs | Mixed |
| Packaging and signing | Internal models signed with build provenance from the training platform | Deterministic |
| Deployment | Admission policy: only registry hashes with valid signatures and provenance | Deterministic |
| Runtime | Monitoring for behaviour drift and abuse; ability to roll back by hash | Detection |
Organisational pieces. Ownership for each approved model; a review path for new models that is fast enough that teams use it; and a block on direct hub downloads from production and training clusters so the intake point cannot be bypassed.
Incident path. When a public model or dataset is found to be malicious or backdoored, query the AI-BOM lineage for every fine-tune and deployment derived from it, roll back by hash, and review outputs produced during exposure.
Honest limits. Backdoor detection is weak; a capable attacker can hide behaviour from evaluation. Provenance and publisher trust carry most of the weight for that threat.
Direct pulls
Pipeline fetches by name
latest revision
Malicious pickle or renamed repo
runs on load
Code execution in training
Outcome, unsafe: Backdoored model
No inventory
Controlled intake
Internal registry
provenance, safe formats
Pinned by hash
scanned, AI-BOM
Only vetted artefacts
Outcome, safe: Changes visible
Traceable use
What interviewers probe next
- "A research team needs a new model today." A fast lane: isolated sandbox use immediately, production use after intake review.
- "How do you handle LoRA adapters from third parties?" Same intake: they change behaviour like weights, can be backdoored, and need provenance and evaluation.
- "What about models served by a vendor API?" The supply chain shifts to vendor assurance: contracts, model version pinning, change notifications, and your own evaluations on updates.
What if?
What if · interview follow-ups
Hold your answer as the constraints change
A team pulls models and datasets from public hubs directly in its training and serving pipelines, using repository names and the latest revision.
Start here
What is the main risk in this setup?
What decides which bytes you download?
Every run trusts whoever controls those names at download time. A compromised account, renamed organisation or malicious revision changes what you train on or serve without any review, and pickle files or remote code can execute in the pipeline.
Carry it forwardFetching by name trusts the current owner of the name.
Weak answerTreating popular repositories as safe by default.
Constraint changes
What does a single intake point give you?
Where can you enforce checks once for everyone?
An internal registry or proxy through which every model and dataset enters. It verifies provenance (publisher, pinned revision, hashes, signatures), enforces safe formats and no unreviewed remote code, scans files and stores approved artefacts immutably by hash.
Carry it forwardRoute all ML artefacts through one controlled intake.
Weak answerLetting each pipeline apply its own checks.
Constraint changes
What happens after the file checks?
Does a safe file mean a well-behaved model?
Evaluate the model in isolation for behaviour and backdoor indicators, record it with an AI-BOM entry (base model, data, licences, evaluations), and only then approve it for use. File safety and behavioural trust are separate questions.
Carry it forwardEvaluate behaviour after verifying files.
Weak answerApproving on scan results alone.
Constraint changes
How do you enforce it at deployment?
What stops a pipeline pulling from the hub directly?
Deploy only registry artefacts whose hash and provenance match policy, sign internally trained models, and block direct downloads from public hubs in production. Admission checks at deploy time stop pipelines from bypassing the registry.
Carry it forwardAdmission control enforces the registry path.
Weak answerRelying on documentation to keep pipelines compliant.
Constraint changes
What if a model you use is later found to be malicious?
Which systems inherited from the bad model?
Lineage tells you every system affected: which fine-tunes derived from it, which services serve it and which datasets it touched. Revoke it in the registry, roll back affected deployments and retrain derived models where needed.
Carry it forwardLineage makes ML supply-chain incidents tractable.
Weak answerSearching code by hand for model names.
Now defend it
How would you secure the team's ML supply chain?
A single intake registry that verifies publisher, pinned revision, hashes and signatures, enforces safe formats and no unreviewed remote code, scans and evaluates in isolation; immutable storage by hash with AI-BOM entries; signing of internal models; deploy-time admission against registry policy; production monitoring; and lineage for incident response.
How it's scored
Weak: "Scan models and use safetensors."
Hire: Central intake with provenance, safe formats, scanning, isolated evaluation and deployment checks.
Strong Hire: Full lifecycle with deterministic versus probabilistic controls, fine-tuning and adapters, AI-BOM lineage for incident response, and candid limits on backdoor detection.
Common mistakes
- Allowing direct downloads from hubs into production or training clusters.
- Referencing models by mutable names instead of hashes.
- Evaluating new models on machines with credentials and network access.