Stale Debug Log in Production

Problem

A production service has been emitting the same DEBUG-level log lines for weeks. That’s usually the residue of an old investigation: someone raised the log level to chase an incident, the incident ended, and the setting stayed. The detail was for a developer who has long since stopped looking.

Insights fingerprints debug messages per service and flags ones that keep appearing in production for more than 14 days.

Impact

• You pay ingest and storage for detail nobody reads.
• Debug lines crowd the view during real incidents, when log volume matters most.
• Debug output is written for developers’ eyes and often carries internal detail that doesn’t belong in long-term storage.

Example

Left behind:

DEBUG cache lookup key=cart:8842 hit=true ttl=291s   ← every request, since August

Cleaned up:

# log level back to INFO in production;
# debug available on demand via a runtime toggle

How to fix

1. Find where the log level is set for the flagged service: an environment variable, config file, or feature flag.
2. Return production to INFO (or WARN) as the default.
3. If you regularly need debug detail in production, make it a runtime toggle with an expiry instead of a permanent setting.

Expected gains

Log volume drops immediately, incident-time logs get readable again, and debug detail stops accumulating in long-term storage.

See the Insights catalog for everything Insights can detect.