Prompt and scope
This is a system-design question for platform security engineers. Kubernetes v1.36 introduces manifest-based admission control as an alpha feature. Policies are loaded from an API-server-local directory before requests are served, so the design must work before etcd is available and must include a file-based recovery path.
What the interviewer evaluates
- Separation between a bootstrap policy and API-managed policies.
- Atomic validation, rollback, and fail-fast startup behavior.
- Protection of policy and webhook configuration from privileged deletion.
- Configuration distribution and drift detection across API-server instances.
- Operator escape hatches, auditability, and failure-mode testing.
Which resources are protected?
Clarify whether the baseline protects only ValidatingAdmissionPolicy objects or also bindings, webhooks, and mutating configurations. The match rules and blast radius change with that answer.
What is the recovery authority?
If the API is unavailable or the policy blocks its own repair, the authoritative path must be the API-server host or its immutable configuration pipeline. Define who can change that path and how changes are reviewed.
How many API servers run?
Each instance reads its own files. A fleet needs content-addressed artifacts, rollout sequencing, and a configuration hash metric; a single server can use a simpler local watcher but still needs atomic replacement.
30-second answer framework
“I would install a small, reviewed manifest policy before enabling any API-managed policies. It denies updates or deletes for resources marked as protected, and its name uses the reserved .static.k8s.io suffix. Every API server receives the same versioned directory and exposes its configuration hash. File changes validate and swap atomically; invalid updates keep the last good version, while invalid startup fails fast. Operators recover through the host configuration path, never through the blocked API.”
Step-by-step design
Bootstrap the trust boundary
Enable ManifestBasedAdmissionControlConfig on every API server and configure staticManifestsDir through the existing admission configuration file. Store manifests in a read-only, integrity-checked artifact. Require every static object name to end in .static.k8s.io so metrics and audit records distinguish file-backed objects from API objects.
apiVersion: apiserver.config.k8s.io/v1
kind: AdmissionConfiguration
plugins:
- name: ValidatingAdmissionPolicy
configuration:
staticManifestsDir: /etc/kubernetes/admission/staticWrite the protection rule
The bootstrap policy matches UPDATE and DELETE operations on admission policies, bindings, and webhook configurations. It denies changes only when the old object has a protection label such as platform.example.com/protected=true. This keeps ordinary experimentation possible while shielding the baseline.
Make updates transactional
The API server validates a changed file set and swaps it atomically. If validation fails at runtime, retain the previous good configuration and log the error. At startup, fail before serving requests when any manifest is invalid; silently starting without the baseline would create the exact bootstrap gap the feature is meant to close.
Operate a multi-server fleet
Render the same content-addressed bundle to every API server and roll out one instance at a time. Compare the configuration-hash label and admission decision metrics. A hash mismatch is drift, not a harmless version difference; stop rollout and restore the known bundle before changing policy semantics.
Preserve an escape hatch
The static policy must not depend on a Service, paramKind, or another API object. Those references are unavailable before cluster state exists. Keep an audited host-level path to replace the files, and test that a malformed or overbroad policy can be reverted without an API call.
High-quality sample answer
“I would treat the static directory as a root-of-trust, distribute a signed versioned bundle to every API server, and use a static policy to deny modifications to labeled admission resources. Names ending in .static.k8s.io make provenance visible. Runtime edits validate and swap atomically; startup rejects any invalid bundle. Each server exports a configuration hash, so rollout stops on drift. Recovery is an audited host change, not an API request, and canary tests cover bootstrap, deletion attempts, malformed updates, server restarts, and mixed bundles.”
Common mistakes
- Protecting policies with another API policy → The API cannot guard its own configuration → anchor protection in static files.
- Letting one bad edit replace the active set → A syntax error can remove all protection → validate the whole set and retain the last good version.
- Starting with invalid manifests → The server runs without the intended baseline → fail fast before serving requests.
- Assuming API servers share files → One instance can enforce a different policy → distribute bundles and compare hashes.
- Removing every operator escape hatch → A policy bug becomes an outage → keep a privileged, audited host recovery path.
Scoring rubric and self-check
Score bootstrap safety, policy matching, atomic updates, fleet consistency, recovery, and tests. A strong answer states why static configuration can protect API-managed resources without circular admission, and where the design deliberately gives operators a non-API recovery channel.
Follow-ups and extensions
What if an API server restarts during rollout?
Keep the old bundle available, gate readiness on successful static admission load, and compare the reported hash before adding the server back to traffic.
Can static policies reference a Service webhook?
No. The feature is self-contained before cluster state exists; use a URL-only webhook or a CEL policy with no API resource dependency, then document the availability trade-off.
How do you test a policy that blocks its own deletion?
Create a protected test object, attempt update and delete through the API, verify denial and audit records, then replace the static bundle through the recovery path and confirm the object becomes manageable.
What is the rollout invariant?
Every serving API server must enforce the same approved bundle hash, and at least one audited recovery path must remain available even when the API rejects configuration changes.