PHP MySqli 插入

Question

我有这个脚本:

<?php
ini_set('max_execution_time', 0);
ini_set('display_errors','1');
ini_set('default_charset','utf-8');
include("includes/mysqli.php");
$con->set_charset("utf8");


$sql = "INSERT INTO clans(id, clanid, name, badge, status, playercount, score, requiredtrophies, warswon, warslost, warstied, location,warfrequency, exp, level, description, playerjson, lastupdate)
            VALUES ('', ?, ?, ?, ?, ?, ?, ?, ?,?, ?, ?, ?, ?, ?, ?, ?, now())";
$stmt = $con->prepare($sql); //prepare update statement

$stmt->bind_param('ssisiiiiiissiiss',$clanid,$name,$badge,$status,$playercount,$score,$requiredtrophies,$warswon,$warslost,$warstied,$location,$warfrequency,$exp,$level,$description,$playerarray);

$stmts = $con->prepare("SELECT * FROM activeclans WHERE id > 137439657919 ORDER BY id ASC"); //Prepare select statement
$stmts->execute(); //execute select statement 
$result = $stmts->get_result(); //get select statement results

while ($row = $result->fetch_assoc()) {

    $clanid = $row['id'];
    $clanurl = "http://185.112.249.77:9999/Api/clan?clan=$clanid";
    $jsondata = file_get_contents($clanurl);
    $data = json_decode($jsondata,true);

    if($data['name'] != null){
        $name = $data['name'];
    }else{
        $name = "";
    }

    $badge = $data['badge'];

    if($data['status'] != null){
        $status = $data['status'];
    }else{
        $status = "";
    }

    $playercount = $data['playerCount'];
    $score = $data['score'];
    $requiredtrophies = $data['requiredTrophies'];
    $warswon = $data['warsWon'];
    $warslost = $data['warsLost'];
    $warstied = $data['warsTied'];

    if($data['clanLocation'] != null){
        $location = $data['clanLocation'];
    }else{
        $location = "";
    }

    if($data['warFrequency'] != null){
        $warfrequency = $data['warFrequency'];
    }else{
        $warfrequency = "";
    }

    $exp = $data['exp'];
    $level = $data['level'];
    $description = $data['description'];
    $playerarray = json_encode($data['players']);

    /* Execute update statement */

    $stmt->execute();

}

echo $stmt->affected_rows;

$stmt->close();
$stmts->close();
$con->close();

?>

它基本上插入了大约 157K (157 KHOUSAND) 行数据。而且数据还相当大!您无法检查 file_get_contents URL，因为该端口仅对 localhost 开放。

插入所有这些数据的最快方法是什么？现在已经运行了近 24 小时，完成了 65K。我确实尝试过使用交易，但效果不佳。它给了我 502 Bad Gateway，因此我在脚本上浪费了很多时间，因为它在添加 3000 行后回滚(但是速度很快!)

此外，脚本可能在某些时候失败，并将某些 varchar 字段保留为 null，因此我这样做了，以便它们最终成为空字符串，这样就不会出现任何 mySql 错误(我在使用 transactions 时抛出了这些异常)

这是我与交易内容一起使用的代码。我对准备好的陈述很陌生。我今天将此代码从标准查询转换为准备好的，然后尝试了事务。

<?php
ini_set('max_execution_time', 0);
ini_set('display_errors','1');
ini_set('default_charset','utf-8');
include("includes/mysqli.php");
$con->set_charset("utf8");


$sql = "INSERT INTO clans(id, clanid, name, badge, status, playercount, score, requiredtrophies, warswon, warslost, warstied, location,warfrequency, exp, level, description, playerjson, lastupdate)
            VALUES ('', ?, ?, ?, ?, ?, ?, ?, ?,?, ?, ?, ?, ?, ?, ?, ?, now())";
$stmt = $con->prepare($sql); //prepare update statement

$stmt->bind_param('ssisiiiiiissiiss',$clanid,$name,$badge,$status,$playercount,$score,$requiredtrophies,$warswon,$warslost,$warstied,$location,$warfrequency,$exp,$level,$description,$playerarray);

$stmts = $con->prepare("SELECT * FROM activeclans WHERE id > 137439657919 ORDER BY id ASC"); //Prepare select statement
$stmts->execute(); //execute select statement 
$result = $stmts->get_result(); //get select statement results

try{
    $con->autocommit(FALSE);
    while ($row = $result->fetch_assoc()) {

        $clanid = $row['id'];
        $clanurl = "http://185.112.249.77:9999/Api/clan?clan=$clanid";
        $jsondata = file_get_contents($clanurl);
        $data = json_decode($jsondata,true);

        if($data['name'] != null){
            $name = $data['name'];
        }else{
            $name = "";
        }

        $badge = $data['badge'];

        if($data['status'] != null){
            $status = $data['status'];
        }else{
            $status = "";
        }

        $playercount = $data['playerCount'];
        $score = $data['score'];
        $requiredtrophies = $data['requiredTrophies'];
        $warswon = $data['warsWon'];
        $warslost = $data['warsLost'];
        $warstied = $data['warsTied'];

        if($data['clanLocation'] != null){
            $location = $data['clanLocation'];
        }else{
            $location = "";
        }

        if($data['warFrequency'] != null){
            $warfrequency = $data['warFrequency'];
        }else{
            $warfrequency = "";
        }

        $exp = $data['exp'];
        $level = $data['level'];
        $description = $data['description'];
        $playerarray = json_encode($data['players']);

        /* Execute update statement */

        if(!$stmt->execute()){
            throw new Exception("Cannot insert record. Reason :".$stmt->error);
        }
    }
    $con->commit();

}catch (Exception $e) {
    echo 'Transaction failed: ' . $e->getMessage();
    $con->rollback();
}

echo $stmt->affected_rows;

$stmt->close();
$stmts->close();
$con->close();

?>

谢谢:)

Answer 1

        $sql="
            INSERT INTO
                ue_game_alliance_rank_rights
                (
                      rank
                    , `right`   
                )
            VALUES 
        ";

        $insertQuery    = array();
        $insertData     = array();
        foreach ($rights_status['add'] AS $row ) {
            $insertQuery[] = '(?,?)';
            $insertData[] = $rank;
            $insertData[] = $row['id'];
        }

        if (!empty($insertQuery)) {
            $sql .= implode(', ', $insertQuery);
            $stmt = $this->db->prepare($sql);
            $stmt->execute($insertData);
        }
    }

这是基本技术的一个示例，您需要将这些函数替换为其等效的 mysqli_* 函数。对于插入数据的每个字段，您需要:

$insertData[] = $row['id'];

$row 需要匹配您在 foreach 循环中使用的任何内容，并且 ['id'] 需要是您要插入的字段名称。

$insertQuery[] = '(?,?)';

您需要与要插入的字段一样多的占位符。

总的来说，它会创建批量插入，因此您需要将数据插入到数组中。考虑到您要插入的数据量，使用事务，您可能需要进行试验，看看在服务器提示之前一次可以批量插入多少行