Request ID Logging

When things go wrong, it’s important to be able to identify all relevant log messages for a particular request. This is commonly done by generating a unique ID for each request and then adding that ID to every log entry.

If you wish to trace each request throughout your application, including from within components that are deeply nested or otherwise live outside of the normal request context, you can use a thread-local context object to store the request ID:

# context.py

import threading

class _Context:
    def __init__(self):
        self._thread_local = threading.local()

    @property
    def request_id(self):
        return getattr(self._thread_local, 'request_id', None)

    @request_id.setter
    def request_id(self, value):
        self._thread_local.request_id = value

ctx = _Context()

Then, you can create a middleware class to generate a unique ID for each request, persisting it in the thread local:

# middleware.py

from uuid import uuid4
from context import ctx

class RequestIDMiddleware:
    def process_request(self, req, resp):
        ctx.request_id = str(uuid4())

    # It may also be helpful to include the ID in the response
    def process_response(self, req, resp, resource, req_succeeded):
        resp.set_header('X-Request-ID', ctx.request_id)

Alternatively, if all of your application logic has access to the request, you can simply use the context object to store the ID:

# middleware.py

from uuid import uuid4

# Optional logging package (pip install structlog)
import structlog

class RequestIDMiddleware:
    def process_request(self, req, resp):
        request_id = str(uuid4())

        # Using Falcon 2.0 syntax
        req.context.request_id = request_id

        # Or if your logger has built-in support for contexts
        req.context.log = structlog.get_logger(request_id=request_id)

    # It may also be helpful to include the ID in the response
    def process_response(self, req, resp, resource, req_succeeded):
        resp.set_header('X-Request-ID', req.context.request_id)

Note

If your app is deployed behind a reverse proxy that injects a request ID header, you can easily adapt this recipe to use the upstream ID rather than generating a new one. By doing so, you can provide traceability across the entire request path.

With this in mind, you may also wish to include this ID in any requests to downstream services.

Once you have access to a request ID, you can include it in your logs by subclassing logging.Formatter and overriding the format() method, or by using a third-party logging library such as structlog as demonstrated above.

In a pinch, you can also output the request ID directly:

# some_other_module.py

import logging

from context import ctx

def create_widget_object(name: str) -> Any:
    request_id = 'request_id={0}'.format(ctx.request_id)
    logging.debug('%s going to create widget: %s', request_id, name)

    try:
        # create the widget
    except:
        logging.exception('%s something went wrong', request_id)

    logging.debug('%s created widget: %s', request_id, name)