MySQL 5.7 Percona 服务器频繁崩溃和重启

Question

几天以来，我的 MySQL 服务器崩溃并每隔几个小时重新启动一次。

我检查过的事情: 系统日志没有给我任何指示

MySQL错误日志指示:

    2018-08-22T10:28:25.602376Z 0 [Note] InnoDB: page_cleaner: 1000ms intended loop took 6702ms. The settings might not be optimal. (flushed=9, during the time.)
2018-08-22T10:34:21.112124Z 0 [Note] InnoDB: page_cleaner: 1000ms intended loop took 4263ms. The settings might not be optimal. (flushed=8, during the time.)
Killed
2018-08-22T10:37:48.520909Z mysqld_safe Number of processes running now: 0
2018-08-22T10:37:48.522288Z mysqld_safe mysqld restarted
2018-08-22T10:37:48.595417Z 0 [Warning] Could not increase number of max_open_files to more than 1024 (request: 65535)
2018-08-22T10:37:48.595512Z 0 [Warning] Changed limits: max_connections: 214 (requested 500)
2018-08-22T10:37:48.595518Z 0 [Warning] Changed limits: table_open_cache: 400 (requested 10240)
2018-08-22T10:37:48.774434Z 0 [Warning] The use of InnoDB is mandatory since MySQL 5.7. The former options like '--innodb=0/1/OFF/ON' or '--skip-innodb' are ignored.
2018-08-22T10:37:48.774462Z 0 [Warning] The syntax 'avoid_temporal_upgrade' is deprecated and will be removed in a future release
2018-08-22T10:37:48.775643Z 0 [Warning] 'NO_AUTO_CREATE_USER' sql mode was not set.
2018-08-22T10:37:48.778919Z 0 [Note] /usr/sbin/mysqld (mysqld 5.7.14-8-log) starting as process 27908 ...
2018-08-22T10:37:48.798784Z 0 [Warning] You need to use --log-bin to make --binlog-format work.
2018-08-22T10:37:48.807156Z 0 [Note] InnoDB: PUNCH HOLE support available
2018-08-22T10:37:48.807174Z 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2018-08-22T10:37:48.807180Z 0 [Note] InnoDB: Uses event mutexes
2018-08-22T10:37:48.807184Z 0 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
2018-08-22T10:37:48.807189Z 0 [Note] InnoDB: Compressed tables use zlib 1.2.8
2018-08-22T10:37:48.807194Z 0 [Note] InnoDB: Using Linux native AIO
2018-08-22T10:37:48.808810Z 0 [Note] InnoDB: Number of pools: 1
2018-08-22T10:37:48.813108Z 0 [Note] InnoDB: Using CPU crc32 instructions
2018-08-22T10:37:48.815587Z 0 [Note] InnoDB: Initializing buffer pool, total size = 28G, instances = 8, chunk size = 128M
2018-08-22T10:37:49.681807Z 0 [Note] InnoDB: Completed initialization of buffer pool
2018-08-22T10:37:49.934256Z 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2018-08-22T10:37:49.943100Z 0 [Note] InnoDB: Recovering partial pages from the parallel doublewrite buffer at /var/lib/mysql/xb_doublewrite
2018-08-22T10:37:50.039106Z 0 [Note] InnoDB: Highest supported file format is Barracuda.
2018-08-22T10:37:50.250493Z 0 [Note] InnoDB: Log scan progressed past the checkpoint lsn 1253064250683
2018-08-22T10:37:50.569591Z 0 [Note] InnoDB: Doing recovery: scanned up to log sequence number 1253069493248

此后出现数十条“正在恢复”的通知。

Munin 向我指出 InnoDB 池大小可能是问题所在。日志中有大量此类消息:

2018-08-22T10:39:10.793549Z 0 [Note] InnoDB: page_cleaner: 1000ms intended loop took 7779ms. The settings might not be optimal. (flushed=11, during the time.)

my.cnf 配置设置

# INNODB #
innodb-flush-method            = O_DIRECT
innodb-log-files-in-group      = 2
innodb-log-file-size           = 1G
innodb-flush-log-at-trx-commit = 2
innodb-file-per-table          = 1
innodb-buffer-pool-size        = 28G

我有点困惑如何解决重新启动的问题。

Answer 1

为您的 my.cnf [mysqld] 部分考虑的建议(直到您可以发布其他请求的信息)

innodb_fast_shutdown=0  # from 1 (YES) for clean (buffers flushed) to avoid recovery on start/restart
   yes shutdown will take a few seconds, maybe hours if
innodb_buffer_pool_pages_dirty are in the thousands or millions

innodb_lru_scan_depth=128  # from 1024 to conserve CPU every second  see refman
innodb_page_cleaners=4  # from 1 for additional page cleaning capacity
innodb_flushing_avg_loops=10  # from 30 to reduce the loop delay

您的打开文件限制也可能是 1024。您可以在操作系统命令提示符下使用 ulimit -a 进行查找。 ulimit -n 20000 提高限制将有助于减少文件/表的打开频率。发布您要求的其他信息将提供更多事实。