Skip to content
AI Security

Interview question · Cloud, Container & AI Infrastructure Security

Teams are running Ollama and vLLM in your cloud. What makes self-hosted inference servers risky, and how do you secure them?

Inference servers are built to run on a trusted machine, so most have no authentication and some trust their own network completely. Internet scans keep finding them anyway.

mediumNewPreparation tags: NVIDIA, Wiz, CiscoUpdated Nov 2026

Inference servers are built to run on a trusted machine, so most have no authentication and some trust their own network completely. Internet scans keep finding them anyway.

Make your first attempt

What would you say before reading the answer?

  1. 01 State the assumptions your answer depends on.
  2. 02 Explain the mechanism and choose a control.
  3. 03 Name a condition that would change your decision.

The worked answer includes an approach, scoring criteria and follow-up questions. Compare the reasoning with your own.

10 answers per topic instead of 5, plus saved progress and bookmarks · costs nothingor unlock all 529 remaining answers · $25

Every worked answer in all 17 topics, every lesson in every course, the full concept curriculum, the complete practice-test bank, and the downloadable security field guides.

6 months · One payment · No auto-renewal

What if · the follow-ups in this answer

The answer works through 6 follow-ups like an interviewer would

A data science team set up an Ollama server on a cloud GPU instance so colleagues could try open models. To make it reachable they changed it to listen on all interfaces.

  1. What can anyone who reaches that port do?
  2. What if the server is patched and only reachable inside the VPC?
  3. What if the team moves to multi-node vLLM for bigger models?
  4. What if users ask to pull any model from public registries?
  5. What if security did not know this server existed until a cloud bill spike?
  6. What standard would you set for self-hosted inference?

Each rung has a nudge, a model answer, the principle that carries over and the weak answer to avoid. Premium opens every answer and ladder.

UP NEXT ON YOUR JOURNEY
Next in this trackHow does agentless cloud security scanning (CNAPP) work, and what does it miss compared with an agent on the workload?Next in this trackDesign a data security toolchain: discovery and classification, DSPM, DLP and de-identification, including data flowing into AI systems.Next in this trackBuild runtime detection for Linux hosts and Kubernetes with open-source tools: auditd, osquery, Falco and Tetragon. What does each do best?
DISCUSSION · 0

No comments yet. Share a correction or a different way to answer it.