java - 解析日志文件中的请求和响应时间

Question

我正在尝试解析日志文件。我需要帮助检查请求和响应之间的时间差。本质上是花费超过 10 秒的请求数量。以下是该文件的示例摘录:

16/08/29-03:20:39.538 << 1250 REQUEST COUNTER="1"
16/08/29-03:20:39.542 << 1250 REQUEST COUNTER="2"
16/08/29-03:20:39.656 >> 2250 RESPONSE COUNTER="1"
16/08/29-03:20:39.655 >> 2250 RESPONSE COUNTER="2"

此文件可以包含大约 300 万条记录。我正在考虑使用 BufferedReader 读取它，读取一行，然后检查该行中是否存在“COUNTER”，存储请求时间和计数器的值，然后在整个文件中搜索相同的计数器值以获得响应时间，然后检查差异。

我需要帮助来存储时间并检查差异。另外，有没有更好的方法？

    BufferedReader reader = new BufferedReader(new FileReader(new File(fileName)));
        String line;
        while((line = reader.readLine()) != null) {
            if(line.contains("COUNTER")) {
                String[] tokens = line.split("\\s+");
                Date date = new SimpleDateFormat("yy/mm/dd-HH:mm:ss.SSS").parse("16/08/29-12:42:48.167");
                //this is not storing the correct value                 
            }

Answer 1

我会采取一次读取日志文件并使用 HashMap 跟踪请求/响应的方法。映射键将是计数器，映射值将是请求的日期。为了节省内存，请在收到响应后从 map 中删除项目。使用Java的pattern matching确定它是请求还是响应并获取计数器 ID 和日期。

这是一个例子:

final static long EXPIRED_MILLIS = 1000 * 10; // ten seconds
final static Pattern REQUEST_PATTERN = Pattern.compile("(.* )(<< \\d{4} )(REQUEST COUNTER=\")(\\d)(\")");
final static Pattern RESPONSE_PATTERN = Pattern.compile("(.* )(<< \\d{4} )(RESPONSE COUNTER=\")(\\d)(\")");
final static SimpleDateFormat DATE_FORMAT = new SimpleDateFormat("yy/MM/dd-HH:mm:ss.SSS");

List<String> getExpiredCounterList(File file) throws Exception {
    Map<String,Long> requestMap = new HashMap<>();
    List<String> expiredCounterList = new ArrayList<>();

    try( FileReader fileReader = new FileReader(file);
         BufferedReader buffReader = new BufferedReader(fileReader) )
    {
        String line;
        while((line = buffReader.readLine()) != null) {
            Matcher requestMatcher = REQUEST_PATTERN.matcher(line);
            if(requestMatcher.matches()) {
                addRequestToMap(line, requestMatcher, requestMap);
                continue;
            }

            Matcher responseMatcher = RESPONSE_PATTERN.matcher(line);
            if(requestMatcher.matches()) {
                String expiredCounter = checkIfExpiredAndRemoveFromMap(line, responseMatcher, requestMap);
                if(expiredCounter != null)
                    expiredCounterList.add(expiredCounter);
            }
        }
    }
    return expiredCounterList;
}

void addRequestToMap(String request, Matcher requestMatcher, Map<String,Long> requestMap) {
    String counter = getCounter(request, requestMatcher, 4);
    long date = getDate(request, requestMatcher, 1);
    requestMap.put(counter, date);
}

String checkIfExpiredAndRemoveFromMap(String response, Matcher responseMatcher, Map<String,Long> requestMap) {
    String counter = getCounter(response, responseMatcher, 4);
    if( requestMap.containKey(counter) ) {
        long date = getDate(response, responseMatcher, 1);
        long elapsedMillis = date - requestMap.remove(counter);
        if(elapsedMillis > EXPIRED_MILLIS)
            return counter;
     }
     return null;
}

String getCounter(String line, Matcher matcher, int group) {
    return line.substring(matcher.start(group), matcher.end(group));
}

long getDate(String line, Matcher matcher, int group) throws ParseException {
    return DATE_FORMAT.parse(line.substring(matcher.start(group), matcher.end(group))).getTime();
}