Managing Sensitive Data in Logs

In this lesson, you will learn how to implement proactive masking strategies to ensure Personally Identifiable Information (PII) never reaches your diagnostic logs in plaintext.

When building high-throughput systems, logging is essential for observability, but dumping raw objects—such as User or CreditCard records—often results in sensitive data being persisted to disk or external log aggregators. As an advanced developer, you must move beyond simple "don't log it" policies and implement automated masking at the diagnostic pipeline level.

Implementing a Masking Enricher

Instead of manually stripping fields in every LogInformation call, the most efficient approach is to implement an ILogEventEnricher. By intercepting the log event before it is serialized, you can inspect the properties and redact sensitive values.

Consider a scenario where you want to mask email addresses:

public class PiiMaskingEnricher : ILogEventEnricher
{
    public void Enrich(LogEvent logEvent, ILogEventPropertyFactory propertyFactory)
    {
        foreach (var property in logEvent.Properties)
        {
            if (property.Key.Contains("Email", StringComparison.OrdinalIgnoreCase))
            {
                var value = property.Value.ToString();
                var masked = MaskEmail(value);
                logEvent.AddOrUpdateProperty(propertyFactory.CreateProperty(property.Key, masked));
            }
        }
    }

    private string MaskEmail(string email) => 
        $"{email[0]}***{email.Substring(email.IndexOf('@'))}";
}

Strategic Performance Considerations

While masking protects data, performing complex Regex operations inside an enricher can introduce significant latency into your logging path—which we know from our previous work on Analyzing Log Throughput Bottlenecks is a critical performance risk.

To maintain high performance:

1. Prefer String Manipulation: Use simple string slicing or Span<T> over heavy Regex patterns inside your enricher logic.
2. Targeted Redaction: Apply masking only to specific properties identified via custom attributes (e.g., [SensitiveData]) rather than reflecting over entire objects, which minimizes allocation overhead.
3. Fail-Safe: Ensure your masking logic is wrapped in a try-catch block; a crash in an enricher should never bring down the main application thread or block your request pipeline.

Key Takeaway: Automate sensitive data redaction via log enrichers to prevent PII leaks while minimizing performance degradation through targeted, allocation-efficient string processing.