inference
AI security interview questions tagged inference, across every topic.
3 questions · 0 unlocked for you
Concepts behind "inference"
Explanations of the mechanisms these questions rely on.
Core
Self-Hosted Inference Servers: A Secure Reference DeploymentTeams increasingly run their own model servers, such as Ollama, vLLM, Triton, Ray Serve and llama.cpp. Most were designed for trusted networks: some have no authentication by default, expose model management next to inference, and rely on internal channels that deserialise data. Vulnerabilities and exposed servers followed. A secure deployment binds servers to loopback or a private segment behind an authenticating proxy, separates admin endpoints from inference, loads models only from an internal registry by digest, gives nodes no unnecessary cloud identity or egress and patches on a schedule. Infrastructure SecuritySign in
Advanced
The Inference Stack as Attack SurfaceAn LLM inference service is a stack of components that all handle untrusted input: an API gateway, authentication and quotas, request and prompt processing, tokenizers, the model server, GPU drivers, caches, logging, and the management plane. Each layer has classic risks such as parsing bugs, exposed dashboards and broken tenancy, plus AI-specific ones: cost exhaustion, model extraction through the API, prompt leakage through shared caches, and data retained in logs. Threat-modeling the stack layer by layer turns 'secure the model' into concrete controls. Infrastructure Security🔒 Premium