memory - 单次调用 fgetcsv() 会耗尽 PHP 中的大内存限制

Question

我正在尝试从平均大小的 CSV (10 MB) 中读取两行，标题和第一行。我将 PHP 的内存限制设置为 64 MB。然而，在第二次调用 fgetcsv() 时，我收到内存耗尽错误。

我已经删除了脚本中所有不必要的部分进行调试，并逐步跟踪内存使用情况，但我仍然无法弄清楚。第一次调用后，内存使用量不到 1 MB，但在第二次调用时，发生内存耗尽错误。

代码如下:

<?php
function trace($msg) {
    echo "<br>" . date("Y-m-d H:i:s") . " - " . memory_get_usage() . " bytes - {$msg}";
}

ini_set('display_errors', 'on');
error_reporting(E_ALL);
ob_implicit_flush(true);

trace("CSV Memory Limit test");

$path = "../data/uploaded/1405712684_base_leitores_july2014.csv";
trace("path = {$path}");

$fp = fopen($path, "r");
trace("file pointer opened");

trace("getting header...");
$header = fgetcsv($fp, 1024);
trace("header fetched");
var_dump($header);
trace("header displayed");

trace("fetching another line");
$l1 = fgetcsv($fp, 1024);
trace("line fetched");
var_dump($header);
trace("line displayed");

这是输出:

2014-07-18 16:06:26 - 249544 bytes - CSV Memory Limit test
2014-07-18 16:06:26 - 249800 bytes - path = ../data/uploaded/1405712684_base_leitores_july2014.csv
2014-07-18 16:06:26 - 250840 bytes - file pointer opened
2014-07-18 16:06:26 - 250840 bytes - getting header...
2014-07-18 16:06:26 - 279280 bytes - header fetched
array (size=111)
  0 => string 'razao social' (length=12)
  1 => string 'Identificaao' (length=13)
  2 => string 'save' (length=4)
  3 => string 'id' (length=2)
  4 => string 'categoria' (length=9)
  5 => string 'nome fantasia' (length=13)
  6 => string 'email' (length=5)
  7 => string 'website' (length=7)
  8 => string 'logradouro' (length=10)
  9 => string 'endereco' (length=8)
  10 => string 'numero' (length=6)
  11 => string 'complemento' (length=11)
  12 => string 'bairro' (length=6)
  13 => string 'cidade' (length=6)
  14 => string 'estado' (length=6)
  15 => string 'cep' (length=3)
  16 => string 'localidade' (length=10)
  17 => string 'ddd' (length=3)
  18 => string 'telefone' (length=8)
  19 => string 'fax' (length=3)
  20 => string 'contato' (length=7)
  21 => string 'cargo' (length=5)
  22 => string 'departamento' (length=12)
  23 => string 'ramo de atividade' (length=17)
  24 => string 'data de nascimento' (length=18)
  25 => string 'especialidade' (length=13)
  26 => string 'linha de atuacao' (length=16)
  27 => string 'data de fundacao' (length=16)
  28 => string 'classificacao de leitor' (length=23)
  29 => string 'observacoes' (length=11)
  30 => string 'is_protocolada' (length=14)
  31 => string 'is_assinante' (length=12)
  32 => string 'qtd_reclamacoes' (length=15)
  33 => string 'error' (length=5)
  34 => string 'errorReport' (length=11)
  35 => string 'saved' (length=5)
  36 => string 'onTrash' (length=7)
  37 => string 'SugestAo de matEria' (length=19)
  38 => string 'Campo2
#redacted#' (length=31)
  39 => string '1' (length=1)
  40 => string '0' (length=1)
  41 => string '141657' (length=6)
  42 => string '2000000041' (length=10)
  43 => string '' (length=0)
  44 => string '' (length=0)
  45 => string '' (length=0)
  46 => string 'RUA' (length=3)
  47 => string '#redacted#' (length=15)
  48 => string '#redacted#' (length=3)
  49 => string '' (length=0)
  50 => string 'JABAQUARA' (length=9)
  51 => string 'PARI' (length=4)
  52 => string 'SP' (length=2)
  53 => string '#redacted#' (length=7)
  54 => string '' (length=0)
  55 => string '11' (length=2)
  56 => string '#redacted#' (length=9)
  57 => string '#redacted#' (length=9)
  58 => string '#redacted#' (length=5)
  59 => string '#redacted#' (length=12)
  60 => string '' (length=0)
  61 => string 'Centros Automotivos, Mec‰nicas e Oficinas - Leves' (length=49)
  62 => string '' (length=0)
  63 => string '' (length=0)
  64 => string '' (length=0)
  65 => string '' (length=0)
  66 => string '' (length=0)
  67 => string 'Centros Automotivos, MecAnicas e Oficinas - Leves ' (length=50)
  68 => string 'N' (length=1)
  69 => string '0' (length=1)
  70 => string '0' (length=1)
  71 => string '0' (length=1)
  72 => string '' (length=0)
  73 => string '1' (length=1)
  74 => string '0' (length=1)
  75 => string '' (length=0)
  76 => string '
#redacted#' (length=16)
  77 => string '2' (length=1)
  78 => string '0' (length=1)
  79 => string '228109' (length=6)
  80 => string '2000000041' (length=10)
  81 => string '0800 AUTOMOTIVE' (length=15)
  82 => string '#redacted#@hotmail.com' (length=32)
  83 => string '' (length=0)
  84 => string 'RUA' (length=3)
  85 => string '#redacted#' (length=7)
  86 => string '322' (length=3)
  87 => string '' (length=0)
  88 => string '' (length=0)
  89 => string 'SAO PAULO' (length=9)
  90 => string 'SP' (length=2)
  91 => string '#redacted#' (length=7)
  92 => string '' (length=0)
  93 => string '11' (length=2)
  94 => string '#redacted#' (length=9)
  95 => string '#redacted#' (length=9)
  96 => string '#redacted#' (length=40)
  97 => string 'PROPRIETARIO' (length=12)
  98 => string '' (length=0)
  99 => string 'Centros Automotivos, Mec‰nicas e Oficinas - Leves' (length=49)
  100 => string '' (length=0)
  101 => string '' (length=0)
  102 => string '' (length=0)
  103 => string '' (length=0)
  104 => string '' (length=0)
  105 => string 'Centros Automotivos, MecAnicas e Oficinas - Leves ' (length=50)
  106 => string '' (length=0)
  107 => string '0' (length=1)
  108 => string '0' (length=1)
  109 => string '0' (length=1)
  110 => string 'Registro criado' (length=15)

2014-07-18 16:06:26 - 287608 bytes - header displayed
2014-07-18 16:06:26 - 287608 bytes - fetching another line
( ! ) Fatal error: Allowed memory size of 67108864 bytes exhausted (tried to allocate 72 bytes) in /home/lqdi/quantum.pranaeditora.com.br/sandbox/test_csv_read.php on line 25
Call Stack
#   Time    Memory  Function    Location
1   0.0011  249048  {main}( )   ../test_csv_read.php:0
2   0.0093  287520  fgetcsv ( ) ../test_csv_read.php:25

我在 Ubuntu 14.04 LTS 上运行 php5-fpm (5.5.9) 和 nginx (1.4.6)。

Answer 1

终于，经过一番摸索，我找到了问题所在:行尾。

不知何故，Mac 的 Excel 导出的 CSV 文件的标题为 \r\n，其他行为 \n。标题会很好地出现，但 fgetcsv 理解第二行是整个文档。

修复方法是在 fopen 之前添加以下内容:

ini_set('auto_detect_line_endings',TRUE);

这将使 fgetcsv 在解析 CSV 文件时正确地检测行尾。