php - 如何在php中使用桶系统写入百万级数据

Question

我试了一个小时，但找不到解决方案。我有 6000 万或更多记录(mysql)我想导出为 CSV 文件。请告诉我如何在这个过程中使用桶系统。如果可能的话，导出单独的文件对我来说很好。

<?php
// Fetch Record from Database
@ini_set('log_errors','On');
@ini_set('display_errors','On');
@ini_set('error_log','error.log'); // path to server-writable log file
@ini_set('max_execution_time', 0); //unlimited
@ini_set('memory_limit', '-1');
$output = "";
$connection= mysqli_connect('192.168.1.203','cnsdb','cnsdb','test_irmt');
$table = "tracking";
$sql = mysqli_query($connection,"select * from {$table} limit 0,2000");
$columns_total = mysqli_num_fields($sql);

// Get The Field Name
$query = "SHOW COLUMNS FROM {$table}";
$result_set = mysqli_query($connection,$query);
while ($result = mysqli_fetch_array($result_set)) {
    $heading = $result[0];
    $output .= trim($heading.',');
}

$output = substr($output,0,strlen($output)-1)."\r\n";

// Get Records from the table


// Download the file

$filename = "output".".csv";
header("Pragma: public", true);
header("Expires: 0"); // set expiration time
header("Cache-Control: must-revalidate, post-check=0, pre-check=0");
header("Content-Type: application/force-download");
header("Content-Type: application/octet-stream");
header("Content-Type: application/download");
header('Content-Disposition: attachment; filename='.$filename);
header("Content-Transfer-Encoding: binary");
header('Connection: Keep-Alive');

while ($row = mysqli_fetch_array($sql)) {
    for ($i = 0; $i < $columns_total; $i++) {
        $output .='"'.$row["$i"].'",';
    }
    $output = substr($output,0,strlen($output)-1)."\r\n";
}
echo $output;
exit;

?>

Answer 1

示例尝试使用这个

SELECT *
INTO OUTFILE 'file.csv' -- Change the filename here
FIELDS
  TERMINATED BY ','
  OPTIONALLY ENCLOSED BY '"'
  ESCAPED BY '\\'
  LINES TERMINATED BY '\n'
FROM table

WHERE id BETWEEN 0 AND 10000000

根据需要更改上/下值