Request ID Logging#
When things go wrong, it’s important to be able to identify all relevant log messages for a particular request. This is commonly done by generating a unique ID for each request and then adding that ID to every log entry.
If you wish to trace each request throughout your application, including
from within components that are deeply nested or otherwise live outside of the
normal request context, you can use a ContextVar
object
to store the request ID:
import contextvars
class _Context:
def __init__(self):
self._request_id_var = contextvars.ContextVar('request_id', default=None)
@property
def request_id(self):
return self._request_id_var.get()
@request_id.setter
def request_id(self, value):
self._request_id_var.set(value)
ctx = _Context()
Then, you can create a middleware class to generate a unique ID for each request, persisting it in the contextvars object:
from uuid import uuid4
# Import the above context.py
from my_app.context import ctx
class RequestIDMiddleware:
def process_request(self, req, resp):
request_id = str(uuid4())
ctx.request_id = request_id
# It may also be helpful to include the ID in the response
def process_response(self, req, resp, resource, req_succeeded):
resp.set_header('X-Request-ID', ctx.request_id)
Alternatively, if all of your application logic has access to the
request, you can simply use the
req.context
object to store the ID:
from uuid import uuid4
# Optional logging package (pip install structlog)
import structlog
class RequestIDMiddleware:
def process_request(self, req, resp):
request_id = str(uuid4())
# Using Falcon 2.0+ context style
req.context.request_id = request_id
# Or if your logger has built-in support for contexts
req.context.log = structlog.get_logger(request_id=request_id)
# It may also be helpful to include the ID in the response
def process_response(self, req, resp, resource, req_succeeded):
resp.set_header('X-Request-ID', req.context.request_id)
Note
If your app is deployed behind a reverse proxy that injects a request ID header, you can easily adapt this recipe to use the upstream ID rather than generating a new one. By doing so, you can provide traceability across the entire request path.
With this in mind, you may also wish to include this ID in any requests to downstream services.
Once you have access to a request ID, you can include it in your logs by
subclassing logging.Formatter
and overriding the format()
method,
or by using a third-party logging library such as
structlog as demonstrated above.
In a pinch, you can also output the request ID directly:
import logging
# Import the above context.py
from my_app.context import ctx
def create_widget_object(name: str):
request_id = 'request_id={0}'.format(ctx.request_id)
logging.debug('%s going to create widget: %s', request_id, name)
try:
# create the widget
pass
except Exception:
logging.exception('%s something went wrong', request_id)
logging.debug('%s created widget: %s', request_id, name)