Course contentsShow
← Python
Lesson 401 of 1,0606. Error Handling and File I/OFree lesson

Formatting Log Records for Machine Readability

Learners will design custom logging formatters to include timestamps, severity levels, and process identifiers for effective downstream log analysis.

In this lesson, you will learn how to design custom logging formatters to ensure your logs are structured perfectly for automated analysis tools.

When you send logs to a database or a monitoring service, human-readable text (like "The system crashed at noon") is often difficult to parse programmatically. To make your logs "machine-readable," you need a consistent structure—most commonly JSON—that separates your data points into distinct fields like timestamps, severity levels, and process IDs.

While the standard logging module provides a Formatter class, default outputs are often plain strings. To optimize for machines, we define a formatter that bundles our metadata into a structured dictionary and then serializes it using the json module, which we explored in earlier lessons.

Consider this custom formatter that organizes log data:

import logging
import json
import os

class JsonFormatter(logging.Formatter):
    def format(self, record):
        log_record = {
            "timestamp": self.formatTime(record, self.datefmt),
            "level": record.levelname,
            "pid": os.getpid(),
            "message": record.getMessage()
        }
        return json.dumps(log_record)

# Applying the formatter
logger = logging.getLogger("MachineLogger")
handler = logging.StreamHandler()
handler.setFormatter(JsonFormatter())
logger.addHandler(handler)

logger.warning("Connection timeout detected")

When this runs, the output isn't just a string; it is a valid JSON object: {"timestamp": "2023-10-27 10:00:00", "level": "WARNING", "pid": 1234, "message": "Connection timeout detected"}.

By including the pid (process identifier), you allow downstream tools to correlate logs across multiple instances of your application. Including the timestamp in a standard format (like ISO 8601) ensures that your analysis software can sort events chronologically with ease. When you structure logs this way, you turn raw data into a reliable stream of information that any ingestion tool can index and query without complex regex patterns.

Key Takeaway: Formatting your logs as structured JSON with consistent metadata fields makes them immediately ready for automated processing and high-performance analysis.