What separates kernel space from user space, and what actually happens when a program makes a system call?

TL;DR: The CPU runs ordinary programs in an unprivileged mode that cannot touch hardware, page tables or other processes' memory; only the kernel runs privileged. A program asks for anything privileged through a system call: a controlled entry point where the CPU switches mode and jumps to kernel code chosen by the kernel, which must then validate every argument the program supplied, especially pointers. Because the kernel enforces isolation between all processes, a single kernel vulnerability can break all of it at once, which is why containers sharing a kernel are a weaker boundary than separate virtual machines.

How to approach it

Answer in three layers: what the hardware enforces, how a request crosses the boundary, and what the kernel must check on the far side. Then give the security consequence: what an attacker gains by crossing it and why the boundary matters to sandboxes and containers. Using "ring 0 and ring 3" is fine on x86, but say that the property is privilege mode, not the number.

A strong answer

What the hardware enforces. The processor has at least two privilege levels. In user mode, privileged instructions (changing page tables, disabling interrupts, talking to devices directly) fault instead of running. Memory is reached through virtual addresses that the memory management unit translates using page tables the kernel controls, so a user process can only see the pages the kernel mapped for it. Kernel memory is mapped either not at all or as supervisor-only, so a user-mode read of it faults.

How a request crosses. A program that wants to open a file, send a packet or allocate memory places a system call number and arguments in registers and executes a dedicated instruction (syscall on x86-64, svc on ARM64). The CPU switches to kernel mode and jumps to an entry address the kernel registered at boot, not one the program chose. The kernel dispatches on the number, does the work with its own privileges, and returns to user mode with a result. Interrupts and faults enter the kernel the same controlled way.

What the kernel must distrust. Everything the caller passed is attacker-controlled input:

• Pointers. A program can pass the address of kernel memory, or of memory belonging to a different mapping, and ask the kernel to write there. The kernel copies through helpers (copy_from_user, copy_to_user in Linux) that check the address range and handle faults. Hardware features such as SMAP and SMEP stop the kernel from accidentally reading user data or executing user pages while in kernel mode, which closes a classic class of exploits.
• Time. User memory can change while the kernel is reading it. Reading a length, checking it, then reading it again is a double-fetch race; the kernel must copy once and use the copy.
• Objects and permissions. A file descriptor number must belong to this process, and the operation must be allowed for this process's credentials. Access checks happen here, at the moment of use.

Why it matters for security.

Every container on a host shares one kernel, so the kernel's system call surface is the container boundary. That is why sandboxes shrink it: seccomp filters block system calls a workload never needs, and gVisor puts a user-space kernel in front of the host so the real kernel sees a much smaller surface. A virtual machine adds a hypervisor boundary: a guest kernel compromise still leaves the attacker inside the guest.

One correction interviewers like to hear: running a program as administrator or root changes its permissions, not its mode. Root's code still runs in user mode and still asks the kernel; root is simply allowed to ask for more, including loading kernel modules, which is how root becomes kernel.

What interviewers probe next

• "Why must the kernel validate a user pointer even after checking that it points into user space?" Because the memory can be unmapped or changed by another thread between check and use, and because the check must happen with the right fault handling. Copy once, then validate the copy.
• "How does a hypervisor change this picture?" It adds a second privilege boundary below the guest kernel, so a guest kernel exploit needs a further hypervisor escape to reach the host or other guests.
• "What does seccomp actually reduce?" The set of kernel entry points reachable from the sandboxed process, and with it the kernel code an exploit can target. It does not change what allowed calls are permitted to do.
• "Is a system call expensive, and does that matter for security?" A mode switch costs real time, which is why some data is exposed through shared pages (the vDSO for time on Linux). Anything mapped that way is readable by every process, so it must contain nothing sensitive.

What if?

How it's scored

Weak: "The kernel has more privileges than user programs." No mention of the system call path or of what the kernel must check.

Hire: Explains hardware-enforced modes, virtual memory, the system call instruction and the kernel's duty to validate arguments, and links kernel bugs to container escape.

Strong Hire: Adds pointer validation and double-fetch races, SMAP/SMEP, the distinction between root and kernel mode, and uses the boundary to justify seccomp, gVisor and VMs for untrusted code.

Common mistakes

• Saying root runs "in the kernel".
• Treating the boundary as a software convention rather than something the CPU enforces.
• Forgetting that arguments, especially pointers and lengths, are hostile input.
• Calling containers a kernel-level isolation boundary without noting the shared kernel.