我有以下粗略的代码(完整代码146行,其中90行是字符串解析,如果需要可以添加):
ini_set('memory_limit', '7G');
$db = new PDO("mysql:host=".$dbhost.";dbname=".$dbname, $dbuser, $dbpass, array(PDO::ATTR_PERSISTENT => true));
$db->setAttribute(PDO::MYSQL_ATTR_USE_BUFFERED_QUERY, false);
$db_ub = new PDO("mysql:host=".$dbhost.";dbname=".$dbname, $dbuser, $dbpass, array(PDO::ATTR_PERSISTENT => true));
$db_ub->setAttribute(PDO::MYSQL_ATTR_USE_BUFFERED_QUERY, true);
$stmt = $db->prepare('select columns from stats where timestamp between ? and ?');
$stmt->execute(array('2020-04-25', '2020-05-25'));
while($row = $stmt->fetch(PDO::FETCH_ASSOC)) {
echo memory_get_usage() .PHP_EOL;
echo $row['id'] . PHP_EOL;
$stmt2 = $db_ub->prepare('select somedata from users limit 1');
$stmt2->execute();
$row2 = $stmt2->fetch(PDO::FETCH_ASSOC);
$type = !empty($row2['somedate']) ? 5 : 4;
$result = $db_ub->prepare('insert ignore into newtable (old, type) values (?, ?)');
$result->execute(array($row['id'], $type));
}
期间
$stmt->execute(array('2020-04-25', '2020-05-25'));
我的内存消耗为 .34GB
(使用 ps aux | grep 'php ' | awk '{$5=int(100 * $5/1024/1024)/100"GB";}{ print;}'
监控 select
和 show full processlist
SQL 端的消耗以进行验证)。一旦脚本进入 while
它跳到+5 GB。测试
setattribute
var_dump($db->setAttribute(PDO::MYSQL_ATTR_USE_BUFFERED_QUERY, false));
好像已经生效了:
bool(true)
但是当我切换缓冲或无缓冲时,行为不会改变。
$db->setAttribute(PDO::MYSQL_ATTR_USE_BUFFERED_QUERY, false)
和
$db->setAttribute(PDO::MYSQL_ATTR_USE_BUFFERED_QUERY, true)
使用
echo $db->getAttribute(constant('PDO::MYSQL_ATTR_USE_BUFFERED_QUERY'));
还显示设置更改。将设置移动到语句而不是连接为 https://www.php.net/manual/en/ref.pdo-mysql.php建议也没用。
$stmt = $db->prepare('select columns from stats where timestamp between ? and ?', array(PDO::MYSQL_ATTR_USE_BUFFERED_QUERY => false));
我还尝试将缓冲区设置移动到连接而没有任何影响:
$db = new PDO("mysql:host=".$dbhost.";dbname=".$dbname, $dbuser, $dbpass, array(PDO::ATTR_PERSISTENT => true, PDO::MYSQL_ATTR_USE_BUFFERED_QUERY => false));
取出第二个连接似乎允许无缓冲查询按预期运行:
ini_set('memory_limit', '1G');
$db = new PDO("mysql:host=".$dbhost.";dbname=".$dbname, $dbuser, $dbpass, array(PDO::ATTR_PERSISTENT => true, PDO::MYSQL_ATTR_USE_BUFFERED_QUERY => false));
$db->setAttribute(PDO::MYSQL_ATTR_USE_BUFFERED_QUERY, false);
//$db_ub = new PDO("mysql:host=".$dbhost.";dbname=".$dbname, $dbuser, $dbpass, array(PDO::ATTR_PERSISTENT => true));
//$db_ub->setAttribute(PDO::MYSQL_ATTR_USE_BUFFERED_QUERY, true);
$stmt = $db->prepare('select columns from stats where timestamp between ? and ?');
$stmt->execute(array('2019-01-25', '2019-11-25'));
while($row = $stmt->fetch(PDO::FETCH_ASSOC)) {
echo memory_get_usage() .PHP_EOL;
echo $row['id'] . PHP_EOL;
/*
$stmt2 = $db_ub->prepare('select somedata from users limit 1');
$stmt2->execute();
$row2 = $stmt2->fetch(PDO::FETCH_ASSOC);
$type = !empty($row2['somedate']) ? 5 : 4;
$result = $db_ub->prepare('insert ignore into newtable (old, type) values (?, ?)');
$result->execute(array($row['id'], $type));
*/
}
这个用法
memory_get_usage
不超过 379999
.如果我取消注释第二个连接并使其无缓冲,我会收到:
Cannot execute queries while other unbuffered queries are active. Consider using PDOStatement::fetchAll(). Alternatively, if your code is only ever going to run against mysql, you may enable query buffering by setting the PDO::MYSQL_ATTR_USE_BUFFERED_QUERY attribute.
缓冲的第二个连接如最初描述的那样执行,执行时消耗大量内存。如
ini_set('memory_limit'
高它工作如果低它错误。使用大 memory_limit
不是一个可行的解决方案。正在使用(
Red Hat Enterprise Linux Server release 7.3 (Maipo)
):php71u-pdo.x86_64 7.1.19-1.ius.centos7
将脚本移至较新的机器 (
Amazon Linux release 2 (Karoo)
):php73-pdo.x86_64 7.3.17-1.el7.ius
并有相同的行为。
最佳答案
PDO::ATTR_PERSISTENT
值不是 bool 值。它标识正在使用的连接,为多个连接使用唯一值。就我而言:
$db = new PDO("mysql:host=".$dbhost.";dbname=".$dbname, $dbuser, $dbpass, array(PDO::ATTR_PERSISTENT => 'unbuff', PDO::MYSQL_ATTR_USE_BUFFERED_QUERY => false));
$db_ub = new PDO("mysql:host=".$dbhost.";dbname=".$dbname, $dbuser, $dbpass, array(PDO::ATTR_PERSISTENT => 'buff', PDO::MYSQL_ATTR_USE_BUFFERED_QUERY => true));
关于php - PDO MYSQL_ATTR_USE_BUFFERED_QUERY 不生效,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/62096624/