Prompt and context
Teams use different logging libraries and agents, with levels ranging from TRACE and DEBUG to CRITICAL. Explain how to map them to OpenTelemetry SeverityNumber and SeverityText, preserve source information, and handle exceptions, sampling, queries, and version migration.
What the interviewer is testing
- Distinguishing standardized SeverityNumber, source SeverityText, and the log body.
- Understanding that numeric ranges express severity and same-named levels need not have identical meaning across systems.
- Combining exception events, resource attributes, trace/span correlation, and collector transformations.
- Accounting for unknown levels, sampling bias, query compatibility, alert thresholds, and replay validation.
Clarifying questions to ask
- What does each source's level mean, what numeric range does it use, and does it define custom levels?
- Must the original level string, logging library, and version be retained? Which fields do downstream queries use?
- Is an exception a separate event or ordinary body text, and should it link to a trace, span, or request ID?
- Will mapping change alerts, sampling, storage cost, or retention compliance? How will historical logs be replayed?
A 30-second answer
I would build a semantic dictionary for every source, then map meanings to an OpenTelemetry SeverityNumber range while preserving the original SeverityText and source metadata. Exceptions use the standard exception semantics and trace/span correlation; a stack trace should not be stuffed into the level field. The collector transforms and validates records, while unknown levels follow an observable fallback path. Before rollout I would replay representative and failure samples to check alert thresholds, sampling, queries, and cost rather than silently redefining all history.
Step-by-step deep dive
1. Define the standard fields
The OpenTelemetry Logs Data Model separates SeverityNumber and SeverityText. Number supports comparisons within the model; Text preserves the producer's original or display name. Body, attributes, resources, and timestamps carry other semantics. The mapping table should record the source, original level, target range, and rationale instead of hiding everything in conversion code.
2. Map sources to semantic ranges
WARN, ERROR, or FATAL can trigger different operational actions in different frameworks. Map by meaning to a range, retain an unconfirmed level as undefined or low confidence, and place the original value in SeverityText or a controlled attribute. Do not compare raw numbers across systems until each source's documentation and samples are verified.
3. Handle exceptions and correlation
OpenTelemetry exception semantics recommend fields for exception type, message, and stack trace, with severity chosen according to whether the exception causes application failure. Logs should carry trace ID, span ID, service, and deployment version so queries can distinguish records from one request. Stack traces are diagnostic data and require the same sensitive-data and retention controls as other logs.
4. Validate ingestion, alerts, and migration
Perform mapping, field validation, and compatibility conversion in the Collector or an edge agent, recording invalid values and drop reasons. Replay historical samples and synthetic failures to verify alert thresholds, sampling, query results, and storage cost. During upgrades, keep a mapping version and source fields so consumers can interpret history by version instead of silently changing alert meaning.
Model answer
I would create documented semantic mappings for Java, Python, Nginx, and other sources, targeting OpenTelemetry SeverityNumber while retaining original names in SeverityText and controlled attributes. Numbers are compared only within the same model; unknown or custom levels follow a low-confidence path with an alert. Exceptions use OpenTelemetry exception fields and link to trace/span and service version. The Collector handles transformation, validation, and metrics. Historical and failure samples are replayed to test alerts, sampling, queries, and cost. Mapping versions and source fields remain available for auditability.
Common mistakes
- Mapping every source's ERROR, WARN, or FATAL one-to-one without checking semantics.
- Keeping only SeverityNumber and dropping the original level and source version.
- Putting exception data in the body and making stack traces hard to query or unsafe to retain.
- Omitting trace/span correlation and losing request-level context.
- Silently dropping failed mappings or forcing them to INFO without metrics or alerts.
- Recomputing historical alerts after a mapping change without a version or replay record.
Follow-up questions and responses
What number should an unknown level use?
Keep the original text and mark it unknown or low confidence instead of forcing ERROR. If business ordering is required, define a documented default range, record the mapping version, monitor the unknown ratio, and fix the source semantics.
Can sampling change the severity distribution?
Yes. Prioritize severe records and exception events, record the sampling decision and input denominator, and compare pre- and post-sampling distributions. Sampled counts should not be presented as the true error rate without that context.
How do you keep old queries compatible?
Keep source fields and old aliases in the conversion layer temporarily, and expose versioned views or query functions. During migration, dual-write or replay alerts and reports until differences are understood, then retire the old fields.