Logging

Logging is an essential part of any application, either to monitor or debug issues. Expanse put its own twist on it to make it intuitive and simple to use.

Logging channels

Expanse uses the concept of logging channels to manage different logging configurations. Each channel can have its own handlers, formatters, and log levels. This allows you to have different logging configurations for different parts of your application.

Configuring logging channels

Channels are configured through environment variables prefixed with LOG_. Examples for different channels are provided in the example .env.example file at the root of you application directory. You can edit and/or remove any value as you see fit.

You can configure as many logging channels as you want by following the following pattern:

LOG_CHANNELS__CHANNEL_NAME__DRIVER=stream

CHANNEL_NAME is the name of the channel you want to configure and DRIVER is the logging driver you want to use for that channel. The supported drivers are:

• stream: Logs will be written to a dedicated stream (stdout or stderr, defaults to stderr).
• file: Logs will be written to a file. The file path can be specified via the PATH parameter.
• console: Logs will be written to the console with colors and pretty formatting. This driver is only meant to be used in development mode.
• daily: Logs will be written to a file that is rotated daily. The file path can be specified via the PATH parameter.
• group: Logs will be grouped together and sent to multiple channels. The channels to send the logs to can be specified via the CHANNELS parameter as a comma-separated list of channel names.

However, if you need support for more logging drivers, you can easily extend the logging manager.

Specifying levels and formats

You can also specify the log level and format for each channel. The log level can be specified via the LEVEL parameter, while the log format can be specified via the FORMAT parameter. If not specified, the default log level is INFO, and the default log format is %(asctime)s - %(name)s - %(levelname)s - %(message)s.

LOG_CHANNELS__FILE__DRIVER=file
LOG_CHANNELS__FILE__PATH=storage/log/app.log
LOG_CHANNELS__FILE__LEVEL=INFO
LOG_CHANNELS__FILE__FORMAT=%(asctime)s - %(name)s - %(levelname)s - %(message)s

The LEVEL parameter accepts the following values: DEBUG, INFO, WARNING, ERROR, and CRITICAL. The log format can be any valid format string supported by the logging module of Python.

Configuring the logging mode

You can also configure the logging mode of your application via the LOG_MODE environment variable. The logging mode determines how log messages are processed and routed to the configured channels.

LOG_MODE=async

The supported logging modes are:

• async: This is the default logging mode. In this mode, log messages are processed via a non-blocking queue and routed to the configured channels based on the logger name and the log level. This mode is suitable for production environments, as it allows for efficient logging without blocking the main application flow.
• sync: This mode processes log messages synchronously, meaning that log messages are processed and routed to the configured channels immediately when they are logged. This mode is suitable for development environments, as it allows for easier debugging and immediate feedback on log messages.

Writing log messages

Writing log messages is done through the standard logging module of Python.

import logging

from expanse.logging.logger import Logger
from expanse.http.response import Response
from expanse.routing.helpers import get

logger = logging.getLogger(__name__)


@get("/test")
def test() -> Response:
    logger.info("This is an info message")
    logger.error("This is an error message with data: %s", "foo")

    return Response()

Routing logs to channels

For log messages to go through configured channels, you need to route loggers to specific channels. This is done via the LOG_ROUTING environment variable, which should be a multi-line variable where each line specifies a logger and the channels it should be routed to. The logger and channels:

LOG_ROUTING="
app:channel1,channel2
app.submodule:channel3
"

In the above example, all log messages from the app logger (and sub-loggers) will be routed to channel1 and channel2, while log messages from the app.submodule logger (and its sub-loggers) will be routed to channel3.

Adding contextual data

You can also add contextual data to your log messages using the extra parameter of the logging methods. This allows you to include additional information in your log messages, such as the user ID, request ID, or any other relevant data. This data will automatically be included in the log messages if your log format includes the corresponding placeholders.

import logging

logger = logging.getLogger(__name__)

logger.info("User logged in", extra={"user_id": 123})

Any data added to the global context will also be added to the log messages as contextual data, and will be available in the log format as well under the context name.

Structured log messages

By default, log messages are formatted as plain text. However, you can also configure your channels to use structured log messages in JSON format. This is done by setting the STRUCTURED parameter to true for the desired channel.

LOG_CHANNELS__FILE__DRIVER=file
LOG_CHANNELS__FILE__PATH=storage/log/app.log
LOG_CHANNELS__FILE__STRUCTURED=true

When using structured log messages, the log messages will be formatted as JSON objects with the following structure:

{
    "asctime": "2024-01-01 14:48:09,777",
    "name": "app",
    "levelname": "INFO",
    "message": "This is an info message",
    "extra1": "value1",
    "extra2": "value2",
     ...
    "context": {
        "user_id": 123
    }
}

Any additional data added to the log message via the extra parameter will be included as additional fields in the JSON object. The global context will also be included in a context field in the JSON object.