How To Override Gunicorn's Logging Config To Use A Custom Formatter
Solution 1:
I have had very good luck by specifying my own custom logging class. You can do this by making a class that inherits from Gunicorn's gunicorn.glogging.Logger class, and then override the setup(self, cfg) method of this class.
For example:
import logging
from gunicorn import glogging
classCustomLogger(glogging.Logger):
"""Custom logger for Gunicorn log messages."""defsetup(self, cfg):
"""Configure Gunicorn application logging configuration."""super().setup(cfg)
# Override Gunicorn's `error_log` configuration.
self._set_handler(
self.error_log, cfg.errorlog, logging.Formatter(
fmt=('timestamp=%(asctime)s pid=%(process)d ''loglevel=%(levelname)s msg=%(message)s')))
Now, depending upon how you launch Gunicorn, you can either provide the full path to this class; IE: program.app.CustomLogger via the --logger-class option, or you can directly pass it in via your own customer Gunicorn application class like so:
from gunicorn.app import base
from program.app import app, CustomLogger
classWebServer(base.BaseApplication):
"""Gunicorn WSGI Web Server."""def__init__(self, app, options):
"""Initialize server object."""
self.options = options or {}
self.application = app
super().__init__()
defload():
"""Return WSGI application."""return self.application
defload_config():
"""Load configuration into Gunicorn."""
self.cfg.set('logger_class', CustomLogger)
if __name__ == '__main__':
WebServer(app, {}).run()
This should accomplish your desired goal without any yucky INI files being required and it is a completely supported configuration style.
Solution 2:
Gunicorn uses a custom logger called glogger glogger code which internally has a Logger class, you could change the format your messages with attributes that are already available to the logger.
The gunicorn_config file with custom format may look like this (the attributes are self explanatory)
from gunicorn import glogging
glogging.Logger.error_fmt = '{"AppName": "%(name)s", "logLevel": "%(levelname)s", "Timestamp": "%(created)f", "Class_Name":"%(module)s", "Method_name": "%(funcName)s", "process_id":%(process)d, "message": "%(message)s"}'
glogging.Logger.datefmt =""
glogging.Logger.access_fmt = '{"AppName": "%(name)s", "logLevel": "%(levelname)s", "Timestamp": "%(created)f","Class_Name":"%(module)s", "Method_name": "%(funcName)s", "process_id":%(process)d, "message": "%(message)s"}'
glogging.Logger.syslog_fmt = '{"AppName": "%(name)s", "logLevel": "%(levelname)s", "Timestamp": "%(created)f","Class_Name":"%(module)s", "Method_name": "%(funcName)s", "process_id":%(process)d, "message": "%(message)s"}'
I hope this helps, in my opinion it is one of the clean way of overriding the logger format.
but if you want add custom attributes in the logs you may have to create a new instance of the logger class as this logger class doesn't support filter. and assigning a logger instance wont help either.
Solution 3:
I recently run into this myself, with gunicorn, as well as some other python packages that do not honor existing logging configuration when they are imported.
The following function resets all existing logging configuration and loads the new configuration from a dictionary. See logging.config.dictConfig for more information on the configuration dictionary schema.
Just call reset_logging_configuration() after you have imported all offending modules.
import logging
import logging.config
from my_logging_config import CONFIG # see `logging.config.dictConfig`defreset_logging_configuration():
logging._acquireLock()
for logger in logging.Logger.manager.loggerDict.values():
ifnotisinstance(logger, logging.Logger):
continue
logger.handlers.clear() # unset all logger handlers
logger.filters.clear() # unset all logger filters
logger.level = logging.NOTSET # unset logger level
logger.propagate = True# enable logger propagation
logging._releaseLock()
logging.config.dictConfig(CONFIG) # load actual configBeaware: this method messes with logging internals and is more of a work-around than an actual solution. However, many modules blindly overwrite existing logging configuration when they are imported, leaving you with no choice but to overwrite their configuration after they are loaded. Use with caution!
This code is based on a code-snippet found in logging/config.py.
Solution 4:
The problem with gunicorn's logging is that it normally runs in the same Python process before your code, configures logging on its own and logs some messages before any of your WSGI-related code runs and you have a chance to configure logging your way. Fortunately, as already mentioned its "logger" (gunicorn.glogging.Logger by default) is configurable.
$ gunicorn --help | grep logger
--logger-class STRING
The logger you want to use to log events in Gunicorn.
Note that incremental configuration (i.e. dictConfig called once by gunicorn then by you) is not recommended and likely not what you want:
[...] there is not a compelling case for arbitrarily altering the object graph of loggers, handlers, filters, formatters at run-time, once a configuration is set up; the verbosity of loggers and handlers can be controlled just by setting levels (and, in the case of loggers, propagation flags). Changing the object graph arbitrarily in a safe way is problematic in a multi-threaded environment; while not impossible, the benefits are not worth the complexity it adds to the implementation.
Hence, my suggests it to treat gunicorn as a regular library with regards to logging and completely disable its logging configuration (in favour of you application's).
This is a trivial application which has HTTP and console entry points that should have the same logging configuration for both:
logging_cfg = {
'version': 1,
'disable_existing_loggers': False,
'formatters': {
'KeyValueFormatter': {
'format': (
'timestamp=%(asctime)s pid=%(process)d ''loglevel=%(levelname)s msg=%(message)s'
)
},
},
'handlers': {
'console': {
'level': 'INFO',
'class': 'logging.StreamHandler',
'formatter': 'KeyValueFormatter',
}
},
'loggers': {
'gunicorn.access': {
'propagate': True,
},
'gunicorn.error': {
'propagate': True,
},
},
'root': {
'level': 'INFO',
'handlers': ['console'],
}
}
And here its implementation, app.py.
import logging.config
from gunicorn import glogging
logger = logging.getLogger(__name__)
defget_result():
logger.info('Calculating result')
returnb'Hello world!'defwsgi_app(environ, start_response):
result = get_result()
start_response('200 OK', [('Content-Type', 'text/html')])
return [result]
defconfigure_logging():
logging.config.dictConfig(logging_cfg)
classUniformLogger(glogging.Logger):
defsetup(self, cfg):
configure_logging()
if __name__ == '__main__':
configure_logging()
print(get_result())
If you run python app.py you should get:
timestamp=2021-07-25 12:09:04,488 pid=4599 loglevel=INFO msg=Calculating result
b'Hello world!'
And if you run gunicorn --logger-class=app.UniformLogger app:wsgi_app followed by curl localhost:8000:
timestamp=2021-07-25 12:16:56,892 pid=4874 loglevel=INFO msg=Starting gunicorn 20.1.0
timestamp=2021-07-25 12:16:56,892 pid=4874 loglevel=INFO msg=Listening at: http://127.0.0.1:8000 (4874)
timestamp=2021-07-25 12:16:56,893 pid=4874 loglevel=INFO msg=Using worker: sync
timestamp=2021-07-25 12:16:56,894 pid=4876 loglevel=INFO msg=Booting worker with pid: 4876
timestamp=2021-07-25 12:17:06,926 pid=4876 loglevel=INFO msg=Calculating result
Solution 5:
I think you only need to give the configuration file (.ini or .conf) with your own configuration. Make sure you override the default.
There is a examples/logging.conf in the examples directory.
Post a Comment for "How To Override Gunicorn's Logging Config To Use A Custom Formatter"