如何扩展python的logging组件支持json日志输出
程序员文章站
2022-05-03 22:30:32
...
这两天在优化公司一个python的项目,顺便研究了一下如何将python日志转成json格式,原来在Java的项目中搞过类似的事情,知道日志转成json之后有很多便利的之处,最常见的就是可以直接对接各种日志分析系统,如开源的ELK,将数据导入之后就能快速的进行查询和分析,方便做各种统计,监控或报警等。
python里面的logging组件,其实已经是组件化了,有Logger组件,Handler组件,Fomatter组件,如下图所示:
logger=>handler=>formatter分别是一对多的关系,日志的格式其实是由formatter决定的,所以想要扩展成你想要的各种格式,就重写定制formatter组件就可以了,它实际上和Java里面Log4j的LayOut组件类似。
核心代码如下:
```python REMOVE_ATTR = ["filename", "module", "exc_text", "stack_info", "created", "msecs", "relativeCreated", "exc_info", "msg"] class JSONFormatter(logging.Formatter): host_name, host_ip = HostIp.get_host_ip() def format(self, record): extra = self.build_record(record) self.set_format_time(extra) # set time self.set_host_ip(extra) # set host name and host ip extra['message'] = record.msg # set message if record.exc_info: extra['exc_info'] = self.formatException(record.exc_info) if self._fmt == 'pretty': return json.dumps(extra, indent=1, ensure_ascii=False) else: return json.dumps(extra, ensure_ascii=False) @classmethod def build_record(cls, record): return { attr_name: record.__dict__[attr_name] for attr_name in record.__dict__ if attr_name not in REMOVE_ATTR } @classmethod def set_format_time(cls, extra): now = datetime.datetime.utcnow() format_time = now.strftime("%Y-%m-%dT%H:%M:%S" + ".%03d" % (now.microsecond / 1000) + "Z") extra['@timestamp'] = format_time return format_time @classmethod def set_host_ip(cls, extra): extra['host_name'] = JSONFormatter.host_name extra['host_ip'] = JSONFormatter.host_ip ```
使用的时候,可以通过简单配置即可:
``` [loggers] keys=root [handlers] keys=console,file [formatters] keys=json,json_pretty [logger_root] level=DEBUG ;handlers=console,file,rotate handlers=console,file [handler_console] class=StreamHandler level=INFO formatter=json_pretty args=(sys.stderr,) [handler_file] class=FileHandler level=INFO formatter=json args=('log/base_conf.log','a','utf-8') [handler_rotate] class=handlers.TimedRotatingFileHandler level=INFO formatter=json args=('log/rotate.log', 'D',1,0,'utf-8') [formatter_json] class=format.json_formatter.JSONFormatter [formatter_json_pretty] format=pretty class=format.json_formatter.JSONFormatter ```
如下的一段异常代码:
``` fileConfig('log_conf.ini') log = logging.getLogger(__name__) try: a = 1 / 0 except Exception: log.exception(" occurred exception ") ```
输出结果如下:
``` { "name": "__main__", "args": [], "levelname": "ERROR", "levelno": 40, "pathname": "C:/texx.py", "lineno": 17, "funcName": "base_configuration", "thread": 10608, "threadName": "MainThread", "processName": "MainProcess", "process": 11916, "@timestamp": "2019-01-10T12:50:20.392Z", "host_name": "your-PC", "host_ip": "192.168.10.11", "message": " occurred exception ", "exc_info": "Traceback (most recent call last):\n File \"C:/txxx.py\", line 14, in base_configuration\n a = 1 / 0\nZeroDivisionError: division by zero" } ```
可以看到内容非常详细,并且组件还支持自定义字段的加入,在收集到日志系统上,可以非常的方便检索统计。
详细的解释和代码,可以fork我的github连接:
https://github.com/qindongliang/python_log_json
上一篇: 身体和灵魂,总有一个在路上
下一篇: 法棍吃法有哪些?花样吃法吃出花样浪漫