php - 将数据从php中的CSV文件插入mysql

Question

我正在开发需要通过 csv 文件上传数据然后将记录保存到 mysql 表中的应用程序。我在 php 中这样做。到现在为止我已经正确使用了这段代码。

 <?php
 //date_default_timezone_set("Asia/Karachi");
//echo "The time is " . date("Y-m-d h:i:sa");
$DateTime=date("Y-m-d h:i:sa");
 $FilePath=$_POST['img'];
 $ID=$_POST['ID'];
 $Query_String2="";
// echo $FilePath;
 $row = 1;
 if(isset($FilePath))
 {
    // echo "ID is =".$ID;
       $Query_String1 = "INSERT into ap_form_$ID VALUES";

        if (($handle = fopen($FilePath, "r")) !== FALSE)
         {
            while (($data = fgetcsv($handle, 1000, ",")) !== FALSE) 
            {
                if($row!=1)
                $Query_String2=$Query_String2.",";
                $Query_String2= $Query_String2."("."NULL".","."'".$DateTime."'".","."NULL".","."'127.0.0.1'".","."1".","."NULL";
                $num = count($data);
                $row++;
                $master = array();
                for ($c=0; $c < $num; $c++) 
                {

                    if($c==0)
                        $Query_String2=$Query_String2.","."'".$data[$c]."'";
                    else
                        $Query_String2=$Query_String2.","."'".$data[$c]."'";
                //$master[$c]= $data[$c];
                 }
                       $Query_String2=$Query_String2.")";


                  }

                  $Final_Query=$Query_String1.$Query_String2;
                  echo "Final Query =".$Final_Query;
                $connection = mysqli_connect("localhost", "root","","form_db");

                $result = mysqli_query($connection,$Final_Query) or mysql_error();

                if($result > 0)
                {
                    echo "
                    <script type=\"text/javascript\">
                     alert('Record has been inserted into the form');
                    </script>
                ";
                }
                        else
                {
                        echo "
                    <script type=\"text/javascript\">
                     alert('I am sorry there is some problem');
                    </script>
                ";

                }
    fclose($handle);

我在这段代码中所做的是，我从 CSV 文件中读取数据，然后生成一个长查询字符串，例如“插入表值 ("bla, bla ,bal),("bla, bla, bla)" 这段代码适用于小数据，就像我在 csv 文件中有高达 1000 的数据一样但是当数据超过 1000 时，过程卡住了，我无法在 mysql 中保存数据。在我的应用程序中，我需要上传至少50000。 解决此问题的任何想法或解决方案。

Answer 1

如果您在运行 MySQL 的同一台服务器上拥有 CSV 文件，您还可以使用 LOAD DATA INFILE 将 CSV 文件直接加载到数据库中，例如

LOAD DATA INFILE 'yourfile.csv' 
 INTO TABLE yourtable
 FIELDS TERMINATED BY ',';

参见 LOAD DATA INFILE 的 MySQL 引用手册了解更多详情。

编辑:

您可以启用LOAD DATA LOCAL 以允许将 CSV 文件从 Web 服务器加载到数据库中，但需要考虑安全因素。同样，reference manual更多详情:

There are two potential security issues with supporting the LOCAL version of LOAD DATA statements:

• The transfer of the file from the client host to the server host is initiated by the MySQL server. In theory, a patched server could be built that would tell the client program to transfer a file of the server's choosing rather than the file named by the client in the LOAD DATA statement. Such a server could access any file on the client host to which the client user has read access.
• In a Web environment where the clients are connecting from a Web server, a user could use LOAD DATA LOCAL to read any files that the Web server process has read access to (assuming that a user could run any command against the SQL server). In this environment, the client with respect to the MySQL server actually is the Web server, not the remote program being run by the user who connects to the Web server.

要在服务器端启用此功能，请使用:

SET GLOBAL local_infile=ON;

之后你可以验证这一点:

> SHOW GLOBAL VARIABLES like "%local%";
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| local_infile  | ON    |
+---------------+-------+
1 row in set (0.00 sec)

您还需要在 php.ini 中启用它:

[MySQL]
; Allow accessing, from PHP's perspective, local files with LOAD DATA statements
; http://php.net/mysql.allow_local_infile
mysql.allow_local_infile = On

---

启用本地加载数据后，您可以使用从 Web 服务器直接加载 CSV 文件到数据库中:

LOAD DATA LOCAL INFILE 'yourfile.csv' 
 INTO TABLE yourtable
 FIELDS TERMINATED BY ',';

如果需要在加载数据的同时修改数据，可以先将其存储在局部变量中，然后在插入数据库的同时进行修改，例如:

LOAD DATA LOCAL INFILE 'yourfile.csv' 
 INTO TABLE yourtable
 FIELDS TERMINATED BY ','
 (@c1,@c2)
 SET column1=NULL, column2=SUBSTRING(@c2,2,3), column3='127.0.0.1';