python - 使用正则表达式将自定义日志文件解析为字典

Question

示例日志文件

Jun 15 02:04:59 combo sshd(pam_unix)[20897]: authentication failure; logname= uid=0 euid=0 tty=NODEVssh ruser= rhost=220-135-151-1.hinet-ip.hinet.net  user=root\n'
Jun 15 02:04:59 combo sshd(pam_unix)[20898]: authentication failure; logname= uid=0 euid=0 tty=NODEVssh ruser= rhost=220-135-151-1.hinet-ip.hinet.net  user=root\n'
Jun 15 04:06:18 combo su(pam_unix)[21416]: session opened for user cyrus by (uid=0)\n'
Jun 15 04:06:19 combo su(pam_unix)[21416]: session closed for user cyrus\n'
Jun 15 04:06:20 combo logrotate: ALERT exited abnormally with [1]\n'
Jun 15 04:12:42 combo su(pam_unix)[22644]: session opened for user news by (uid=0)\n'
Jun 15 04:12:43 combo su(pam_unix)[22644]: session closed for user news\n'

我想将数据分成 4 列:日期、时间、PID 和消息。

示例输出为

Dict = {"Date": "Jun 15", "Time": "02:04:59", "PID": "20897", "Message": "authentication failure; logname= uid=0 euid=0 tty=NODEVssh ruser= rhost=220-135-151-1.hinet-ip.hinet.net  user=root\n'"}

之后我打算根据列将此信息保存到 CSV 文件中

我尝试查看其他示例，例如:

Parse a custom log file in python

How to parse this custom log file in Python

但我不知道如何创建捕获组来帮助我实现这一目标。

我当前的正则表达式是

“(\w{3}\d{2})”表示日期

“(\d{2}:\d{2}:\d{2})”表示时间

“(?<=[).+?(?=]:)”表示 PID

“((?<=:).*)”消息

但是当我将它们组合在一起时什么也没有发生

Answer 1

解决方案是迭代每一行。对于每一行，使用特定的正则表达式选择日期、时间、PID 和消息。

如果找到，则返回值。否则，返回None。

代码如下:

# Import module
import re

# Output list
out = []
# Read file
with open("data.txt", "r") as f:
    # Iterate over all lines
    for line in f.readlines():
        # Select the different fields
        date = re.search(r'^(\w{3}\s\d{2})', line)
        time = re.search(r'(\d{2}:\d{2}:\d{2})', line)
        PID = re.search(r'\[([0-9]+)\]:', line)
        message = re.search(r":\s(.*?)$", line)
        # Append them to the output using a dict
        # If field isn't found, None is return
        out.append({
            "Date": date.group(1) if date else None,
            "Time": time.group(1) if time else None,
            "PID": PID.group(1) if PID else None,
            "Message": message.group(1) if message else None
        })

输出:

# [
#     {'Date': 'Jun 15', 'Time': '02:04:59', 'PID': '20897', 'Message': "authentication failure; logname= uid=0 euid=0 tty=NODEVssh ruser= rhost=220-135-151-1.hinet-ip.hinet.net  user=root\\n'"},
#     {'Date': 'Jun 15', 'Time': '02:04:59', 'PID': '20898', 'Message': "authentication failure; logname= uid=0 euid=0 tty=NODEVssh ruser= rhost=220-135-151-1.hinet-ip.hinet.net  user=root\\n'"},
#     {'Date': 'Jun 15', 'Time': '04:06:18', 'PID': '21416', 'Message': "session opened for user cyrus by (uid=0)\\n'"},
#     {'Date': 'Jun 15', 'Time': '04:06:19', 'PID': '21416', 'Message': "session closed for user cyrus\\n'"},
#     {'Date': 'Jun 15', 'Time': '04:06:20', 'PID': None, 'Message': "ALERT exited abnormally with [1]\\n'"},
#     {'Date': 'Jun 15', 'Time': '04:12:42', 'PID': '22644', 'Message': "session opened for user news by (uid=0)\\n'"},
#     {'Date': 'Jun 15', 'Time': '04:12:43', 'PID': '22644', 'Message': 'session closed for user news\\n'}
# ]

希望有帮助!