PHP 使用 2 个 for 循环插入 15k 行

标签 php mysql database

我希望你能帮我一点点。我有一个包含 653*24 个对象的 JSON 文件,我想将它们添加到本地数据库(PHPMyAdmin、Mysql)中。 我写了两个 for 循环(第一个 1 - 654,第二个 0 - 24),在这些循环中我必须将信息插入 6 个不同的表中。我这样做了,但问题是循环不会插入所有 ~15k 行,只插入 ~200 行,然后浏览器上的加载标志中断,插入停止,没有错误消息。

我想,插入对于系统来说太多了,但我不知道所以希望你们能给我一个答案/解决方案来解决这个问题。

for($i = 1; $i < 654; $i++) {
    for($j = 0; $j < 24; $j++) {
         *do 6 inserts here*
    }
}

编辑:我又看了一遍代码,会不会是加载653 json链接有问题?因此,它可能会导致脚本结束而不是超时?

JSON

https://www.easports.com/fifa/ultimate-team/api/fut/item?jsonParamObject={%22page%22:1}

PHP

$db = $this->getServiceLocator()->get('db');
$k = 1;
$dataArray = [];

        for($i = 1; $i < 11; $i++) {
            for($j = 0; $j < 24; $j++) {

                $json = file_get_contents('https://www.easports.com/fifa/ultimate-team/api/fut/item?jsonParamObject={%22page%22:' . $i . '}');

                $obj = json_decode($json);

                $id = $k;

                //dbArray
                $playerArray = array("characteristics" => array(),
                    "club" => array(),
                    "nation" => array(),
                    "defending" => array(),
                    "dribbling" => array(),
                    "goalkeeper" => array(),
                    "league" => array(),
                    "pace" => array(),
                    "passing" => array(),
                    "physical" => array(),
                    "player" => array(),
                    "rating" => array(),
                    "shooting" => array());

                $traitArray = array();
                $specialitiesArray = array();
                $specialImageArray = array();
                $attributeArray = array();

                foreach($obj->items[$j]->traits as $value) {
                    array_push($traitArray, $value);
                }
                $traitArray = implode(", ", $traitArray);

                foreach($obj->items[$j]->specialities as $value) {
                    array_push($specialitiesArray, $value);
                }
                $specialitiesArray = implode(", ", $specialitiesArray);

                foreach($obj->items[$j]->specialImages as $value) {
                    array_push($specialImageArray, $value);
                }

                if($specialImageArray[0] != NULL) $obj->items[$j]->headshotImgUrl = $specialImageArray[0];

                foreach($obj->items[$j]->attributes as $key) {
                    array_push($attributeArray, $key->value);
                }

                array_push($playerArray['characteristics'], array(
                    "0" => $obj->items[$j]->firstName,
                    "1" => $obj->items[$j]->lastName,
                    "2" => $obj->items[$j]->commonName,
                    "3" => $obj->items[$j]->age,
                    "4" => $obj->items[$j]->height . " cm",
                    "5" => "",
                    "6" => "",
                    "7" => "",
                    "8" => $obj->items[$j]->headshotImgUrl,
                    "9" => $obj->items[$j]->position,
                    "10" => $obj->items[$j]->rating,
                    "11" => ucfirst($obj->items[$j]->quality),
                    "12" => ucfirst($obj->items[$j]->color),
                    "13" => $obj->items[$j]->skillMoves,
                    "14" => $obj->items[$j]->weakFoot,
                    "15" => $obj->items[$j]->foot,
                    "16" => $obj->items[$j]->atkWorkRate . " / " . $obj->items[$j]->defWorkRate,
                    "17" => $traitArray,
                    "18" => $specialitiesArray
                ));

                array_push($playerArray['league'], array(
                    "0" => $obj->items[$j]->league->name,
                    "1" => $obj->items[$j]->league->abbrName,
                    "2" => ""
                ));

                array_push($playerArray['nation'], array(
                    "0" => $obj->items[$j]->nation->name,
                    "1" => $obj->items[$j]->nation->imageUrls->large
                ));

                array_push($playerArray['club'], array(
                    "0" => $obj->items[$j]->club->name,
                    "1" => $obj->items[$j]->club->imageUrls->normal->large
                ));

                array_push($playerArray['defending'], array(
                    "0" => $attributeArray[4],
                    "1" => $obj->items[$j]->interceptions,
                    "2" => $obj->items[$j]->headingaccuracy,
                    "3" => $obj->items[$j]->marking,
                    "4" => $obj->items[$j]->standingtackle,
                    "5" => $obj->items[$j]->slidingtackle
                ));

                array_push($playerArray['dribbling'], array(
                    "0" => $attributeArray[3],
                    "1" => $obj->items[$j]->agility,
                    "2" => $obj->items[$j]->balance,
                    "3" => $obj->items[$j]->reactions,
                    "4" => $obj->items[$j]->ballcontrol,
                    "5" => $obj->items[$j]->dribbling
                ));

                array_push($playerArray['goalkeeper'], array(
                    "0" => $obj->items[$j]->gkdiving,
                    "1" => $obj->items[$j]->gkhandling,
                    "2" => $obj->items[$j]->gkkicking,
                    "3" => $obj->items[$j]->gkreflexes,
                    "4" => $obj->items[$j]->sprintspeed,
                    "5" => $obj->items[$j]->gkpositioning
                ));

                array_push($playerArray['pace'], array(
                    "0" => $attributeArray[0],
                    "1" => $obj->items[$j]->acceleration,
                    "2" => $obj->items[$j]->sprintspeed
                ));

                array_push($playerArray['passing'], array(
                    "0" => $attributeArray[3],
                    "1" => $obj->items[$j]->vision,
                    "2" => $obj->items[$j]->crossing,
                    "3" => $obj->items[$j]->freekickaccuracy,
                    "4" => $obj->items[$j]->shortpassing,
                    "5" => $obj->items[$j]->longpassing,
                    "6" => $obj->items[$j]->curve
                ));

                array_push($playerArray['physical'], array(
                    "0" => $attributeArray[5],
                    "1" => $obj->items[$j]->jumping,
                    "2" => $obj->items[$j]->stamina,
                    "3" => $obj->items[$j]->strength,
                    "4" => $obj->items[$j]->aggression
                ));

                array_push($playerArray['shooting'], array(
                    "0" => $attributeArray[1],
                    "1" => $obj->items[$j]->positioning,
                    "2" => $obj->items[$j]->finishing,
                    "3" => $obj->items[$j]->shotpower,
                    "4" => $obj->items[$j]->longshots,
                    "5" => $obj->items[$j]->volleys,
                    "6" => $obj->items[$j]->penalties
                ));

                array_push($playerArray['rating'], array(
                    "0" => "",
                    "1" => "",
                    "2" => "",
                    "3" => "",
                    "4" => "",
                    "5" => "",
                    "6" => ""
                ));

                array_push($playerArray['player'], array(
                    "0" => "",
                    "1" => "",
                    "2" => "",
                    "3" => "",
                    "4" => "",
                    "5" => "",
                    "6" => "",
                    "7" => $obj->items[$j]->isGK
                ));
$values = array(
                        ':id'               => $id,
                        ':charFirstname'    => $playerArray["characteristics"][0][0],
                        ':charLastname'     => $playerArray["characteristics"][0][1],
                        ':charCommonname'   => $playerArray["characteristics"][0][2],
                        ':charAge'          => $playerArray["characteristics"][0][3],
                        ':charHeight'       => $playerArray["characteristics"][0][4],
                        ':charNation'       => $playerArray["nation"][0][0],
                        ':charClub'         => $playerArray["club"][0][0],
                        ':charLeague'       => $playerArray["league"][0][0],
                        ':charPicture'      => $playerArray["characteristics"][0][8],
                        ':charPosition'     => $playerArray["characteristics"][0][9],
                        ':charRating'       => $playerArray["characteristics"][0][10],
                        ':charQuality'      => $playerArray["characteristics"][0][11],
                        ':charColor'        => $playerArray["characteristics"][0][12],
                        ':charSkill'        => $playerArray["characteristics"][0][13],
                        ':charWeak_foot'    => $playerArray["characteristics"][0][14],
                        ':charStrong_foot'  => $playerArray["characteristics"][0][15],
                        ':charWorkrates'    => $playerArray["characteristics"][0][16],
                        ':charTraits'       => $playerArray["characteristics"][0][17],
                        ':charSpecialities' => $playerArray["characteristics"][0][18]);
array_push($dataArray, $values);
$k++;
                }
            }
foreach($dataArray as $values) {

                $data[] = '("' . $values[':id'] . '", "' . $values[':charFirstname'] . '",
                            "' . $values[':charLastname'] . '", "' . $values[':charCommonname'] . '",
                            "' . $values[':charAge'] . '", "' . $values[':charHeight'] . '",
                            "' . $values[':charNation'] . '", "' . $values[':charClub'] . '",
                            "' . $values[':charLeague'] . '", "' . $values[':charPicture'] . '",
                            "' . $values[':charPosition'] . '", "' . $values[':charRating'] . '",
                            "' . $values[':charQuality'] . '", "' . $values[':charColor'] . '",
                            "' . $values[':charSkill'] . '", "' . $values[':charWeak_foot'] . '",
                            "' . $values[':charStrong_foot'] . '", "' . $values[':charWorkrates'] . '",
                            "' . $values[':charTraits'] . '", "' . $values[':charSpecialities'] . '")';
            }

            $query = 'INSERT INTO characteristics (id, firstname, lastname, commonname, age, height, nation, club, league, picture, position, rating, quality, color, skill, weak_foot, strong_foot, workrates, traits, specialities) VALUES' . implode(',', $data);

            $stmt = $db->createStatement($query);
            $stmt->prepare($query);
            $results = $stmt->execute(array(
                ':id'               => $id,
                ':charFirstname'    => $playerArray["characteristics"][0][0],
                ':charLastname'     => $playerArray["characteristics"][0][1],
                ':charCommonname'   => $playerArray["characteristics"][0][2],
                ':charAge'          => $playerArray["characteristics"][0][3],
                ':charHeight'       => $playerArray["characteristics"][0][4],
                ':charNation'       => $playerArray["nation"][0][0],
                ':charClub'         => $playerArray["club"][0][0],
                ':charLeague'       => $playerArray["league"][0][0],
                ':charPicture'      => $playerArray["characteristics"][0][8],
                ':charPosition'     => $playerArray["characteristics"][0][9],
                ':charRating'       => $playerArray["characteristics"][0][10],
                ':charQuality'      => $playerArray["characteristics"][0][11],
                ':charColor'        => $playerArray["characteristics"][0][12],
                ':charSkill'        => $playerArray["characteristics"][0][13],
                ':charWeak_foot'    => $playerArray["characteristics"][0][14],
                ':charStrong_foot'  => $playerArray["characteristics"][0][15],
                ':charWorkrates'    => $playerArray["characteristics"][0][16],
                ':charTraits'       => $playerArray["characteristics"][0][17],
                ':charSpecialities' => $playerArray["characteristics"][0][18]
            ));

最佳答案

这里有一些资源浪费问题。

  1. 您检索 JSON 文件的频率比您需要的频率高 24 倍。将对 file_get_contents() 的调用移出内部循环。
  2. 在使用准备好的语句时应牢记以下原则:准备一次,执行多次。将语句准备移到任何循环之外,然后在循环内执行。
  3. 您没有构建适当的准备好的语句。该语句应该包含占位符(?:label 格式),但您要将值插入到您的占位符中。您传递给 execute() 的所有这些参数都将被忽略。编辑:实际上,您在页面上插入每个值 24 次,即您为 11 页中的每一页插入 24*24 行。

建议更改摘要:

$query = "INSERT INTO characteristics (
        id, firstname, lastname, commonname, age, height, nation, club, league,
        picture, position, rating, quality, color, skill, weak_foot, strong_foot,
        workrates, traits, specialities
    ) VALUES (
        :id, :charFirstname, :charLastname, :charCommonname, :charAge, :charHeight,
        :charNation, :charClub, :charLeague, :charPicture, :charPosition, :charRating, 
        :charQuality, :charColor, :charSkill, :charWeak_foot, :charStrong_foot, 
        :charWorkrates, :charTraits, :charSpecialities
    )";
$stmt = $db->prepare($query);
for ($i = 1; $i < 11; $i++) {
    $json = file_get_contents('https://www.easports.com/fifa/ultimate-team/api/fut/item?jsonParamObject={%22page%22:' . $i . '}');
    $obj = json_decode($json);
    for ($j = 0; $j < 24; $j++) {
        //set up your player array
        $results = $stmt->execute(array(
            ':id'               => $id,
            ':charFirstname'    => $playerArray["characteristics"][0][0],
            ':charLastname'     => $playerArray["characteristics"][0][1],
            ':charCommonname'   => $playerArray["characteristics"][0][2],
            ':charAge'          => $playerArray["characteristics"][0][3],
            ':charHeight'       => $playerArray["characteristics"][0][4],
            ':charNation'       => $playerArray["nation"][0][0],
            ':charClub'         => $playerArray["club"][0][0],
            ':charLeague'       => $playerArray["league"][0][0],
            ':charPicture'      => $playerArray["characteristics"][0][8],
            ':charPosition'     => $playerArray["characteristics"][0][9],
            ':charRating'       => $playerArray["characteristics"][0][10],
            ':charQuality'      => $playerArray["characteristics"][0][11],
            ':charColor'        => $playerArray["characteristics"][0][12],
            ':charSkill'        => $playerArray["characteristics"][0][13],
            ':charWeak_foot'    => $playerArray["characteristics"][0][14],
            ':charStrong_foot'  => $playerArray["characteristics"][0][15],
            ':charWorkrates'    => $playerArray["characteristics"][0][16],
            ':charTraits'       => $playerArray["characteristics"][0][17],
            ':charSpecialities' => $playerArray["characteristics"][0][18]
        ));
    }
}

关于PHP 使用 2 个 for 循环插入 15k 行,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/35205009/

相关文章:

php 和 session 重定向错误

php - 自定义短代码始终出现在内容顶部

php - 从sql php获取所有数据

sql - 对同一列使用多个条件进行计数

php - 在php中将表记录保存在关联数组中

php - email@domain 是有效的电子邮件地址吗

php - 如何插入一个表并更新另一个表的计数

mysql - SSIS - 将数据从 SQL Server move 到 MySQL

MySQL 不使用索引排序

database - 使用 VS 2010 数据库管理示例数据