php - 拖尾日志文件并将结果写入新文件

Question

我不确定如何表达，所以我会把它打出来，然后编辑并回答出现的任何问题..

目前在我的本地网络设备(基于 PHP4)上，我正在使用它来跟踪实时系统日志文件:http://commavee.com/2007/04/13/ajax-logfile-tailer-viewer/

这很好用，每 1 秒它加载一个执行 tail -n 100 logfile.log 的外部页面 (logfile.php) 该脚本不做任何缓冲，所以它显示的结果屏幕上是日志文件的最后 100 行。

logfile.php 包含:

<? // logtail.php $cmd = "tail -10 /path/to/your/logs/some.log"; exec("$cmd 2>&1", $output);
foreach($output as $outputline) {
 echo ("$outputline\n");
}
?>

这部分运行良好。

我已经调整了 logfile.php 页面以将 $outputline 写入一个新的文本文件，只需使用 fwrite($fp,$outputline."\n");

虽然这有效，但我在创建的新文件中遇到了重复问题。

很明显，每次运行 tail -n 100 都会产生结果，下次运行时可能会产生一些相同的行，因为这样重复我最终会在新文本文件中得到多行重复。

我无法直接将我即将写入的行与之前的行进行比较，因为可能存在相同的匹配项。

有什么方法可以将当前的 100 行 block 与前一个 block 进行比较，然后只写入不匹配的行。再次可能的问题是 block A 和 B 将包含所需的相同行...

是否可以更新 logfile.php 以记录它在我的日志文件中的最后位置，然后只从那里读取接下来的 100 行并将它们写入新文件？

日志文件可能高达 500MB，所以我不想每次都阅读它..

欢迎任何意见或建议..

谢谢

更新 @ 16:30

我已经使用 :

$file = "/logs/syst.log";
$handle = fopen($file, "r");

if(isset($_SESSION['ftell'])) {   
    clearstatcache();
    fseek($handle, $_SESSION['ftell']); 

    while ($buffer = fgets($handle)) { 
        echo $buffer."<br/>";
        @ob_flush(); @flush();
    }   

    fclose($handle);
    @$_SESSION['ftell'] = ftell($handle);        
} else {
    fseek($handle, -1024, SEEK_END);
    fclose($handle);
     @$_SESSION['ftell'] = ftell($handle);
}

这似乎可行，但它会先加载整个文件，然后才加载更新。

我如何从最后 50 行开始，然后只是更新？

谢谢:)

2013 年 4 月 6 日更新 虽然这很有效，但对于大文件来说速度很慢。

我试过这段代码，它看起来更快，但它不只是从它停止的地方读取。

function last_lines($path, $line_count, $block_size = 512){
    $lines = array();

    // we will always have a fragment of a non-complete line
    // keep this in here till we have our next entire line.
    $leftover = "";

    $fh = fopen($path, 'r');
    // go to the end of the file
    fseek($fh, 0, SEEK_END);
    do{
        // need to know whether we can actually go back
        // $block_size bytes
        $can_read = $block_size;
        if(ftell($fh) < $block_size){
            $can_read = ftell($fh);
        }

        // go back as many bytes as we can
        // read them to $data and then move the file pointer
        // back to where we were.
        fseek($fh, -$can_read, SEEK_CUR);
        $data = fread($fh, $can_read);
        $data .= $leftover;
        fseek($fh, -$can_read, SEEK_CUR);

        // split lines by \n. Then reverse them,
        // now the last line is most likely not a complete
        // line which is why we do not directly add it, but
        // append it to the data read the next time.
        $split_data = array_reverse(explode("\n", $data));
        $new_lines = array_slice($split_data, 0, -1);
        $lines = array_merge($lines, $new_lines);
        $leftover = $split_data[count($split_data) - 1];
    }
    while(count($lines) < $line_count && ftell($fh) != 0);
    if(ftell($fh) == 0){
        $lines[] = $leftover;
    }
    fclose($fh);
    // Usually, we will read too many lines, correct that here.
    return array_slice($lines, 0, $line_count);
}

可以通过任何方式对其进行修改，使其从最后已知的位置开始读取......？

谢谢

Answer 1

简介

您可以通过跟踪最后位置来跟踪文件；

示例

$file = __DIR__ . "/a.log";
$tail = new TailLog($file);
$data = $tail->tail(100) ;
// Save $data to new file 

TailLog 是我为此任务编写的一个简单类，这里是一个简单示例，用于显示它实际跟踪文件

简单测试

$file = __DIR__ . "/a.log";
$tail = new TailLog($file);

// Some Random Data
$data = array_chunk(range("a", "z"), 3);

// Write Log
file_put_contents($file, implode("\n", array_shift($data)));

// First Tail (2) Run
print_r($tail->tail(2));

// Run Tail (2) Again
print_r($tail->tail(2));

// Write Another data to Log
file_put_contents($file, "\n" . implode("\n", array_shift($data)), FILE_APPEND);

// Call Tail Again after writing Data
print_r($tail->tail(2));

// See the full content
print_r(file_get_contents($file));

输出

// First Tail (2) Run
Array
(
    [0] => c
    [1] => b
)

// Run Tail (2) Again
Array
(
)

// Call Tail Again after writing Data
Array
(
    [0] => f
    [1] => e
)

// See the full content
a
b
c
d
e
f

实时拖尾

while(true) {
    $data = $tail->tail(100);
    // write data to another file
    sleep(5);
}

Note: Tailing 100 lines does not mean it would always return 100 lines. It would return new lines added 100 is just the maximum number of lines to return. This might not be efficient where you have heavy logging of more than 100 line per sec is there is any

尾类

class TailLog {
    private $file;
    private $data;
    private $timeout = 5;
    private $lock;

    function __construct($file) {
        $this->file = $file;
        $this->lock = new TailLock($file);
    }

    public function tail($lines) {
        $pos = - 2;
        $t = $lines;
        $fp = fopen($this->file, "r");
        $break = false;
        $line = "";
        $text = array();

        while($t > 0) {
            $c = "";

            // Seach for End of line
            while($c != "\n" && $c != PHP_EOL) {
                if (fseek($fp, $pos, SEEK_END) == - 1) {
                    $break = true;
                    break;
                }
                if (ftell($fp) < $this->lock->getPosition()) {
                    break;
                }
                $c = fgetc($fp);
                $pos --;
            }
            if (ftell($fp) < $this->lock->getPosition()) {
                break;
            }
            $t --;
            $break && rewind($fp);
            $text[$lines - $t - 1] = fgets($fp);
            if ($break) {
                break;
            }
        }

        // Move to end
        fseek($fp, 0, SEEK_END);

        // Save Position
        $this->lock->save(ftell($fp));

        // Close File
        fclose($fp);
        return array_map("trim", $text);
    }
}

尾锁

class TailLock {
    private $file;
    private $lock;
    private $data;

    function __construct($file) {
        $this->file = $file;
        $this->lock = $file . ".tail";
        touch($this->lock);

        if (! is_file($this->lock))
            throw new Exception("can't Create Lock File");

        $this->data = json_decode(file_get_contents($this->lock));

        // Check if file is valida json
        // Check if Data in the original files as not be delete
        // You expect data to increate not decrease

        if (! $this->data || $this->data->size > filesize($this->file)) {
            $this->reset($file);
        }
    }

    function getPosition() {
        return $this->data->position;
    }

    function reset() {
        $this->data = new stdClass();
        $this->data->size = filesize($this->file);
        $this->data->modification = filemtime($this->file);
        $this->data->position = 0;
        $this->update();
    }

    function save($pos) {
        $this->data = new stdClass();
        $this->data->size = filesize($this->file);
        $this->data->modification = filemtime($this->file);
        $this->data->position = $pos;
        $this->update();
    }

    function update() {
        return file_put_contents($this->lock, json_encode($this->data, 128));
    }
}