我一直在一个单独的服务器上工作来配置 mariadb,但查询仍然需要很长时间才能返回,而且大部分时间都不会返回。
Mariadb 版本:10.1.31-1 服务器有 50GB Ram 和 2200 Ghz * 10 核心 Centos7作为OP
目标数据库当前有非常大的 innodb 表并且没有很好地索引。只有 innodb 没有任何其他表类型。
目前我正在做一些嵌套的选择查询,并将结果保存到另一个空的克隆数据库中,插入或替换成。
举个例子:
REPLACE INTO db1.table1
SELECT * FROM db0.table1 WHERE ART_ID IN (
SELECT ART_ID FROM db0.table2 WHERE BRA_ID IN (
SELECT BRA_ID FROM db0.table3 WHERE BRAND IN (
SELECT BRAND FROM db0.table4 WHERE ID IN (... bunch of comma seperated ids)
)
)
);
唯一的连接来自本地和/或只有一个远程连接。 服务器的唯一任务是将庞大的数据过滤到另一个数据库,没有网络服务器等。
我把/etc/my.cnf 文件放在下面。我对这个文件做了很多测试,所以你可能会看到一些无用的添加。
问题是:cnf文件中是否有遗漏的配置;是否有任何其他配置提示等可以在合理的时间内运行这些查询。
它运行得更好,但在我更改配置或/并更新服务器后,一切都变慢了。 (是的,我在更新数据库版本后也更新了表格)
# The MySQL server
[mysqld]
performance_schema=0
bind-address = 0.0.0.0
max_connections = 32
socket = /var/lib/mysql/mysql.sock
skip-external-locking
max_allowed_packet = 1024M
connect_timeout=300
wait_timeout=1200
interactive_timeout=300
key_buffer_size = 2G
bulk_insert_buffer_size=256M
sort_buffer_size = 32M
#read_buffer_size = 8M
#read_rnd_buffer_size = 32M
myisam_sort_buffer_size = 32M
join_buffer_size=32M
#max_heap_table_size= 8G
tmp_table_size=8G
concurrent_insert=2
#external-locking=FALSE
#open_files_limit=50000
default-storage-engine=innodb
innodb_file_per_table=1
#lower_case_table_names=1
# Try number of CPU's*2 for thread_concurrency
innodb_write_io_threads=10
innodb_read_io_threads=20
#innodb_buffer_pool_instances=8
#thread_stack = 1M
#thread_cache_size = -1
innodb_buffer_pool_size = 16G
#query_cache_limit = 1073741824
#query_cache_size = 1073741824
#query_cache_type = 1
query_cache_size=0
query_cache_type=0
#innodb_additional_mem_pool_size = 20M
# Set .._log_file_size to 25 % of buffer pool size
innodb_fast_shutdown=0
#innodb_log_file_size = 1G
#innodb_log_buffer_size = 8M
#innodb_flush_log_at_trx_commit = 1
#innodb_lock_wait_timeout = 0
innodb_lock_wait_timeout = 1073741823
innodb_thread_concurrency=0
innodb_commit_concurrency=0
innodb_flush_method=O_DSYNC
innodb_log_compressed_pages = 0
#innodb_flush_neighbors = 0
#innodb_adaptive_hash_index_partitions = 8
log-bin=mysql-bin
server-id = 1
[mysqldump]
quick
max_allowed_packet = 16M
[mysql]
no-auto-rehash
[isamchk]
key_buffer = 384M
sort_buffer = 384M
read_buffer = 256M
write_buffer = 256M
[myisamchk]
key_buffer = 384M
sort_buffer = 384M
read_buffer = 256M
write_buffer = 256M
[mysqlhotcopy]
interactive-timeout
我使用了 percona 向导并合并了建议。 当前的my.cnf如下。
我昨天晚上(2018-03-05 22:41:30 UTC)运行了 php 应用程序,但 mysql 仍然无法返回任何结果。
应用程序仅添加限制 10000 和偏移量(我不想从头开始,我目前正在跟踪最后一个偏移量并从该点开始。)
目前我正计划删除并重新安装 mariadb :( (会是情绪 react )
# Generated by Percona Configuration Wizard (http://tools.percona.com/) version REL5-20120208
[mysql]
# CLIENT #
port = 9999
socket = /var/lib/mysql/mysql.sock
[mysqld]
# GENERAL #
port = 9999
user = mysql
default-storage-engine = InnoDB
socket = /var/lib/mysql/mysql.sock
pid-file = /var/lib/mysql/mysql.pid
# MyISAM # - there is no myisam table
key-buffer-size = 32M
myisam-recover-options = FORCE,BACKUP
# SAFETY #
max-allowed-packet = 16M
max-connect-errors = 1000000
skip-name-resolve
# DATA STORAGE #
datadir = /var/lib/mysql/
# BINARY LOGGING #
log-bin = /var/lib/mysql/mysql-bin
expire-logs-days = 14
sync-binlog = 1
# CACHES AND LIMITS #
tmp-table-size = 32M
max-heap-table-size = 32M
query-cache-type = 0
query-cache-size = 0
max-connections = 30
#thread-cache-size = 16
open-files-limit = 65535
table-definition-cache = 1024
table-open-cache = 2048
# INNODB #
innodb-flush-method = O_DIRECT
innodb-log-files-in-group = 2
innodb-log-file-size = 5G
innodb-flush-log-at-trx-commit = 1
innodb-file-per-table = 1
innodb-buffer-pool-size = 42G
innodb-buffer-pool-instances = 42
thread_cache_size=100
innodb_lru_scan_depth=100
innodb_purge_threads=4
innodb_read_io_threads=64
innodb_write_io_threads=64
innodb_thread_concurrency=0
max_seeks_for_key=32
max_write_lock_count=16
thread_concurrency=35
innodb_fast_shutdown=0
innodb_file_per_table=1
default-storage-engine=innodb
concurrent_insert=2
join_buffer_size=32M
tmp_table_size=8G
read_buffer_size = 8M
# LOGGING #
log-error = /var/lib/mysql/mysql-error.log
log-queries-not-using-indexes = 0
slow-query-log = 1
slow-query-log-file = /var/lib/mysql/mysql-slow.log
最佳答案
Question中的my.cnf数据与SHOW GLOBAL VARIABLES不匹配;值(value)观。
请使用此链接
https://mariadb.com/kb/en/library/configuring-mariadb-with-mycnf/
以帮助获得准确的匹配值以获得帮助。
然后,您的 my.cnf [mysqld] 部分需要考虑的建议包括:
innodb_buffer_pool_size=8G # from > 40G (95% of RAM will not work well)
innodb_lru_scan_depth=100 # from 1024 see refman every second digging too deep
log_warnings=2 # from 1 to include connection errors more detail
max_connect_errors=10 # from 1,000,000 no need to waste a million cycles
thread_cache_size=100 # from 16 CAP per V8 MySQ to be prepared for volume
have_symlink=NO # to protect you server, unless you NEED it
innodb_purge_threads=4 # from 1 to accommodate when needed
innodb_read_io_threads=64 # from 4 see dba.stackexhange.com Q 5666 9/12/11 Rolando
innodb_write_io_threads=64 # from 4 per Rolando to use multi-core
innodb_thrad_concurrency=0 # another part of multi-core enabling
max_seeks_for_key=32 # from huge number, not found in 32, will not be found
max_write_lock_count=16 # give RD opportunity after nn locks
thread_concurrency=35 # to limit concurrent processing load
@Erce,请在正确的事件 my.cnf 中应用。撤回我重做您的 mysqld 部分的提议。
关于mysql - 为专用的 mariadb 服务器找到正确的配置——用于巨大的 innodb 表,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/49094911/