php - 在 php 或 Cakephp 中使用 fgetcsv 导入大型 CSV 文件

Question

我在导入超过 1,00,000 行的 CSV 文件时遇到问题。我正在使用 fgetcsv 从 CSV 文件中获取数据。

我的要求是先截断表格，从 CSV 文件中获取数据并将其存储在数据库中。但是每次它只获取 40000 到 50000 行。谁能告诉我这是什么问题？我的代码如下，

$destination    = realpath('../../admin/webroot/');
$originalPath   = $destination."/".$this->request->data['Importmodel']['importfile'];

# Get Table Fields
$tableFields = ClassRegistry::init('Systemoption')->getTableDetails($this->request->data['Importmodel']['modeloptions']);

ClassRegistry::init($this->request->data['Importmodel']['modeloptions'])->query("TRUNCATE TABLE " . $this->request->data['Importmodel']['modeloptions']);
$row = 1;
if(($handle = fopen($originalPath, "r")) !== false) {
    try {
        while (($data = fgetcsv($handle, 0, "|")) !== false) {
            $num = count($data);

            if($num != (count($tableFields) - 1)) { 
                unlink($originalPath);
                $this->Session->setFlash("Table \"".$this->request->data['Importmodel']['modeloptions']."\" fields and \"".$fileName."\" fields does not match", 'default', array('class'=>'flash_bad'));
                $this->redirect(array('action' => 'index'));
                exit;
            }

            $queryData = array();
            for ($c=0; $c < $num; $c++) {
                $queryData[$this->request->data['Importmodel']['modeloptions']][$tableFields[$c+1]] = $data[$c];
            }

            ClassRegistry::init($this->request->data['Importmodel']['modeloptions'])->create();
            ClassRegistry::init($this->request->data['Importmodel']['modeloptions'])->save($queryData);
            $row++;
        }
        $result = 1;
    } catch (Exception $e) {
        $result = 0;    
    }
}

if($result == 1) {
    unlink($originalPath);
    $this->Session->setFlash("Table \"" . $this->request->data['Importmodel']['modeloptions'] . "\" data populated successfully", 'default', array('class'=>'flash_good'));
    $this->redirect(array('action' => 'index'));
    exit;
} else {
    $this->Session->setFlash("Error : Table \"" . $this->request->data['Importmodel']['modeloptions'] . "\" data not fully populated", 'default', array('class'=>'flash_bad'));
    $this->redirect(array('action' => 'index'));
    exit;
}

Answer 1

您是从控制台执行此操作吗？如果不是，请尝试增加 php.ini 中的内存和时间限制