Prompt and scope
Design a multi-tenant Certificate Transparency (CT) log monitor. A user submits one or more domains; the service continuously checks public CT logs, alerts on matching certificates or precertificates, and proves that log data was not silently rewritten. It must also detect stale logs, proof failures, and maximum merge delay violations.
This is a strong system-design question for platform security and certificate-infrastructure roles. The important part is a verifiable data pipeline and explicit failure boundaries, not a list of fashionable services.
What the interviewer evaluates
- Whether you distinguish a monitor from browser policy and CA issuance.
- Whether you understand signed tree heads (STHs), Merkle inclusion proofs, consistency proofs, and append-only semantics.
- Whether maximum merge delay (MMD) becomes a measurable scheduling and alerting condition.
- Whether you cover multiple logs, duplicate reads, split views, outages, and tenant isolation.
- Whether storage, idempotency, alert noise control, and capacity assumptions are concrete.
Clarifications to ask first
Confirm four boundaries:
- Should matching cover an exact name, a registrable domain, wildcards, or names in SAN?
- Is near-real-time detection required, or is minute-level latency acceptable? Which channels and on-call escalation are needed?
- Must the monitor inspect every public log, or only a selected trusted set? Should raw certificates and proofs be retained?
- What are the tenant count, retention period, privacy requirements, and budget?
If no answer is supplied, assume 100 logs polled once per minute, a five-minute end-to-end discovery target, and retained audit evidence.
Thirty-second answer framework
I would split the system into log collection, cryptographic verification, certificate matching, alerting, and audit storage. Each log keeps a verified tree size and latest STH. Collectors verify the STH signature, fetch entries incrementally, and use inclusion or consistency proofs to check append-only behavior. The matching layer normalizes SANs, wildcards, and registrable domains; the alert layer deduplicates and escalates per tenant policy; the audit layer stores raw entries, STHs, proofs, and hashes. Key metrics are STH freshness, log lag, proof-failure rate, alert latency, and false-positive rate. A split view or MMD violation freezes that log's trusted state and escalates it.
Step-by-step deep dive
1. Collection and state machine
Store each log's identity, public key, trusted tree size, last verified STH, last poll time, and state. A scheduler assigns work by state: normal logs use incremental reads, transient failures use exponential backoff, and repeated failures enter quarantine. Use log identity plus target tree size as the task key so retries are idempotent.
2. STH and Merkle verification
First verify the STH signature and timestamp with the log's public key, then require a monotonically nondecreasing tree size. An initial sync can establish a full snapshot; later syncs use a consistency proof to show that the new tree includes the old tree. For each matching entry, use an inclusion proof to show that it belongs to that tree. A failed proof, rollback, or bad signature must not overwrite the old state; retain the evidence and mark the log suspicious.
3. Incremental reads and integrity
Request the range after the last confirmed tree size and write raw certificates, precertificates, log positions, and receive times. Deduplicate by entry identity without discarding the fact that one certificate appeared in several logs. If no verifiable new STH arrives within MMD, record log lag and raise a platform alert. “No matching certificate” must not be inferred from “no new entries were observed.”
4. Certificate matching and alerts
Parse SANs, wildcards, issuer, notBefore, notAfter, and certificate fingerprints. Prefer explicit-name and registrable-domain rules over substring matching. An alert should include tenant, name, log, fingerprint, first-observed time, and evidence links. Deduplicate the same fingerprint within a short window; a new issuer, unusually short validity, or a production domain can raise severity.
5. Storage, tenancy, and recovery
Keep hot state in a relational or key-value store; put raw entries and proofs in object storage indexed by log and tree size. Write events to an immutable audit stream for replay. Tenant queries return only authorized names. Apply per-log rate limits and a global concurrency cap so the monitor does not overload public logs. If local state is lost, recover from the last trusted STH and catch up with consistency proofs instead of trusting an unverified cursor.
6. Observability and failure handling
Measure STH age, MMD lag, confirmed tree size, fetch throughput, proof failures, log availability, match rate, alert latency, and duplicates. During an outage, retain the last trusted state and retry. On a split view or consistency failure, stop using that log's matches, use other logs, and create a security event. If notifications fail, persist events in a durable queue and deliver by event ID after recovery.
High-quality sample answer
I would define trusted progress first: a log cursor advances only after a correctly signed, monotonically sized STH passes a consistency proof. The collector stores that cursor and the requested range with a retry token. The first run establishes a baseline; later runs read by tree-size range. Every entry retains its fingerprint, SAN, log position, raw response, and verification proof.
The matcher normalizes SANs before comparing them with tenant rules for exact names, registrable domains, and explicit wildcards. An event keyed by tenant, fingerprint, and log enters a queue; notification workers deduplicate, escalate, and record delivery. Audit storage retains the STH, proof, entry hash, and verifier version so security engineers can replay the decision independently.
I treat log anomalies as data-trust failures: a bad signature, rollback, failed consistency proof, or MMD timeout creates a high-priority platform event. That log is quarantined and its old trusted state is preserved. Sharding by log, rate limiting, batch reads, and object storage control cost; authorized queries and tenant quotas enforce isolation. I would accept the design using STH freshness, proof failures, discovery latency, alert false-positive rate, and replay success rate.
Common mistakes
- Polling one log and ignoring that certificates can appear in several logs.
- Reading certificate text without validating STH signatures and Merkle proofs.
- Advancing a cursor because the latest request succeeded, even though data was not verified.
- Treating MMD as certificate validity, or having no log-freshness alert at all.
- Matching domains with substring search, creating false positives for similar names and unrelated SANs.
- Globally deduplicating a fingerprint and losing its locations across logs.
- Continuing to issue low-confidence “not found” conclusions while a log is anomalous.
- Keeping only final alerts and no raw entries, STHs, or proofs, making audits impossible.
Follow-up questions and responses
What if a log returns a smaller tree?
Do not advance the cursor. Preserve both STHs and the response, quarantine the log, and raise a consistency alert. Resume only after human review or a newly established trusted baseline.
How do you reduce near-real-time polling cost?
Shard by log, batch new ranges, tune polling frequency dynamically, and place cold evidence in object storage. Do not trade away verification to meet a latency target.
How do you verify domain ownership?
Require DNS, HTTP, or organizational authorization when a monitor is created. Only an authorized principal may change matching rules, and every change is audited.
How is a monitor different from browser CT policy?
A monitor discovers and proves entries in public logs. Browser policy decides whether a certificate satisfies connection requirements. Their failure handling and trust boundaries are different.
How would you test it?
Use a controllable log or recorded responses to inject bad signatures, invalid proofs, rollbacks, MMD timeouts, duplicate entries, and notification retries. Verify that cursors never advance incorrectly, events remain idempotent, and audits replay successfully.