Representative interview topic

Backend interview: How does ACME ARI prevent a certificate renewal storm?

BackendHard
Offer.cc Editorial TeamPublished Updated

Question

You operate 100,000 ACME-managed certificates and previously saturated a CA because they renewed at the same time. Explain RFC 9773 ARI and how clients, the CA, and operations should cooperate to avoid renewal storms.

Prompt and context

A platform team operates 100,000 certificates. Older clients renew on a fixed cron or a percentage of lifetime, clustering requests in one hour. The candidate must explain RFC 9773 RenewalInfo, suggested windows, and retry semantics, then design CA, client, cache, alerting, and fallback behavior. A strong answer separates protocol advice, client decisions, and final certificate configuration.

What the interviewer is testing

  • Whether the candidate understands ARI as an ACME renewal-information extension, not a replacement for the ACME order flow.
  • Whether they explain suggestedWindow, Retry-After, and random selection accurately.
  • Whether they know how replaces links a predecessor certificate, account, and conflicting order.
  • Whether they consider clients without ARI, clock skew, caching, rate limits, and CA failure.
  • Whether they use renewal success, window distribution, and remaining lifetime to prove the design.

Clarifying questions to ask first

  1. Which CA issues the certificates, and can clients upgrade to an ARI-capable version?
  2. How quickly must the window react to emergency revocation or mass replacement?
  3. What are the targets for retry, expiry protection, and human takeover?
  4. Can the CA expose RenewalInfo through anonymous GET, CDN caching, and IP limits?
  5. For clients without ARI, can cron schedules be sharded or given stable jitter?

A 30-second answer

“RFC 9773 lets an ACME server provide a suggested renewal window through RenewalInfo. A client reads suggestedWindow, chooses a uniform random time inside it, and follows Retry-After, exponential backoff, and its existing retry policy. An order uses replaces to link the predecessor certificate, allowing the server to track replacement, prioritize it, or reject a duplicate. I would make RenewalInfo cacheable and rate-limited, monitor request distribution and expiry headroom, and keep a jittered fallback schedule for clients that do not implement ARI.”

Step-by-step deep answer

1. State the problem ARI solves

Fixed intervals, expiry-date offsets, and lifetime percentages all cluster requests, so a CA cannot move load dynamically. ARI lets a CA suggest a new window based on load, an upcoming revocation, or a certificate-lifecycle change. It does not create an order for the client or bypass ACME account, authorization, or validation steps.

2. Fetch the RenewalInfo resource

An ARI-capable directory advertises a renewalInfo URL. The client constructs a resource path from the certificate Authority Key Identifier keyIdentifier and serial number, then sends an unauthenticated GET. The response contains RFC3339 timestamps and an optional explanation link.

http
GET /renewal-info/<aki>.<serial> HTTP/1.1
Host: acme.example.com
Accept: application/json

HTTP/1.1 200 OK
Retry-After: 21600
Content-Type: application/json

{
  "suggestedWindow": {
    "start": "2025-01-02T04:00:00Z",
    "end": "2025-01-03T04:00:00Z"
  },
  "explanationURL": "https://acme.example.com/docs/ari"
}

In ARI, Retry-After expresses the desired interval before checking again; it should not be read only as a minimum wait for the current HTTP request.

3. Choose a randomized renewal time

The client should choose uniformly inside the window, renewing promptly if the window is already in the past. A cron client that cannot sleep precisely compares the target with its next wake-up. Every attempt still obeys account limits, network backoff, and recorded order failures.

4. Handle retries and invalid windows

Connection timeouts, request timeouts, and 5xx responses are temporary errors that can use capped exponential backoff. Missing or invalid Retry-After, an invalid window, DNS failure, and non-5xx errors are long-term errors; the client should retry after a local default interval and use a fallback schedule. An end timestamp at or before the start is not a valid normal window.

5. Preserve replacement with replaces

The client includes replaces in a new order when a clear predecessor exists. The server checks the predecessor’s account and identifiers and rejects a certificate already replaced by another valid order. This supports emergency replacement tracking, priority policy, and post-revocation cleanup without treating concurrent renewals as independent new certificates.

6. Build server and operations protection

RenewalInfo is a non-confidential anonymous GET resource. Use a normalized cache key, CDN or edge caching, and IP rate limits so the endpoint cannot amplify denial-of-service traffic. The server chooses a sensible Retry-After based on client population and watches window QPS, cache hits, 5xx responses, renewal success, expiry headroom, and the share of clients without ARI.

High-quality sample answer

“I would split the design into CA advice, client scheduling, and order linkage. RFC 9773 advertises renewalInfo; the client queries by certificate AKI and serial, chooses a uniform point in suggestedWindow, and uses Retry-After to control checks while retaining capped exponential backoff. The order carries replaces; the server validates account and identifiers and prevents duplicate replacement. Because RenewalInfo is anonymous, I would protect it with CDN caching, a normalized cache key, and rate limits. I would monitor window distribution, expiry headroom, and fallback clients. A client without ARI keeps a jittered legacy schedule, while emergency CA events retain a human acceleration path.”

Common mistakes

  • Treating ARI as an automatic renewal service → it only supplies advice → the client still creates and completes the order.
  • Ignoring Retry-After semantics → polling can create a new spike → follow the server interval with sensible bounds.
  • Making every client renew at the same second → a micro-storm remains → choose uniform random points and keep backoff.
  • Treating replaces as an arbitrary certificate ID → it can link another account → validate account, identifiers, and replacement state.
  • Watching only issuance success → expiry headroom and ARI coverage can degrade → track distribution, fallback share, and remaining lifetime.

Follow-ups and responses

Why can RenewalInfo be an anonymous GET?

It only conveys a suggested time window and is not considered confidential. Anonymous GET also enables caching to reduce client and CA load. The server still needs normalized cache keys, rate limiting, and denial-of-service controls.

What if the ARI window is already in the past?

Renew promptly while obeying error backoff and account limits. A server placing the window in the past generally signals an urgent replacement, so the client should not wait for its next normal cycle.

How do clients without ARI migrate smoothly?

First add stable jitter and sharding to the fixed schedule, then enable ARI by client version. Monitor fallback share, renewal failures, and expiry headroom; insufficient coverage means the CA cannot assume that all traffic is distributed.

How would you test 100,000 certificates?

Load-test RenewalInfo, caches, and orders with synthetic certificates sharing an expiry date. Compare per-minute distribution, peak, P95 renewal latency, retry volume, and expiry headroom before and after ARI. Inject 5xx errors, invalid windows, clock skew, and CA limits to verify that clients do not synchronize retries.

Public sources

Related questions