这是一个非常简单的查询:
SELECT * FROM temp_company WHERE number NOT IN (SELECT number FROM company)
之前需要 15 分钟，但那是在缓冲池大小太小的 Mysql 安装上，15 分钟没问题，因为这是每月任务。我升级到 Mysql 5.7(从 5.1 或 5.2 之类的东西)，因为原始安装是 32 位，我无法将 innodb 缓冲池大小增加到该数据库所需的最小 10gb(我在一台机器上将其设置为 16GB 32GB RAM。一个月后我去运行这个查询，它在 6 小时后仍在运行。
上面的解释是:
id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
1 | PRIMARY | temp_company | | ALL | | | | | 3226661 | 100.00 | Using where |
2 | DEPENDENT SUBQUERY | company | | index | number | number | 33 | | 3383517 | 100.00 | Using where |
company 和 temp_company 的 PRIMARY 索引是 id，但 number 是它们匹配的内容，并且在两者中都是 KEY，但以上是否表明它没有使用 temp_company 表的索引？
我想尝试的另一个逻辑查询是:
EXPLAIN SELECT tc.* FROM temp_company tc
LEFT JOIN company c on c.number = tc.number
WHERE c.number IS NULL
这同样慢，EXPLAIN 是:
id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
1 | SIMPLE | tc | | ALL | | | | | 3226661 | 100.00 | |
2 | SIMPLE | c | | index | number | number | 33 | | 3383517 | 100.00 | Using where; Ising index; Using join buffer (block nested loop) |
任何帮助将不胜感激。也许 Mysql 改变了它查找索引的方式？
**更新 1--------
显示创建的: 公司
CREATE TABLE `company` (
`id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`count_telephone` mediumint(8) unsigned NOT NULL,
`count_fax` mediumint(8) unsigned NOT NULL,
`count_person` mediumint(8) unsigned NOT NULL,
`person_date` date DEFAULT NULL COMMENT 'Date the company_person relation was updated',
`count_email_address` mediumint(8) unsigned NOT NULL,
`name` varchar(255) DEFAULT NULL,
`url` varchar(255) DEFAULT NULL,
`url_date` date DEFAULT NULL,
`url_status` smallint(5) unsigned NOT NULL DEFAULT '0' COMMENT 'Failure count for crawling the URL',
`website_stamp_start` int(10) unsigned DEFAULT NULL,
`website_stamp` int(10) unsigned DEFAULT NULL,
`ch_url` varchar(255) DEFAULT NULL COMMENT 'Companies house URL',
`keywords_stamp_start` int(10) unsigned DEFAULT NULL,
`keywords_stamp` int(11) DEFAULT NULL,
`number` varchar(30) CHARACTER SET ascii COLLATE ascii_bin DEFAULT NULL,
`category` varchar(255) DEFAULT NULL,
`status` varchar(255) DEFAULT NULL,
`status_date` date DEFAULT NULL COMMENT 'Date the status field was updated',
`country_of_origin` varchar(80) DEFAULT NULL,
`dissolution_date` date DEFAULT NULL,
`incorporation_date` date DEFAULT NULL,
`account_ref_day` smallint(5) unsigned DEFAULT NULL,
`account_ref_month` smallint(5) unsigned DEFAULT NULL,
`account_next_due_date` date DEFAULT NULL,
`account_last_made_up_date` date DEFAULT NULL,
`account_category` varchar(255) DEFAULT NULL,
`returns_next_due_date` date DEFAULT NULL,
`returns_last_made_up_date` date DEFAULT NULL,
`mortgages_num_charges` smallint(5) unsigned DEFAULT NULL,
`mortgages_num_outstanding` smallint(5) unsigned DEFAULT NULL,
`mortgages_num_part_satisfied` smallint(5) unsigned DEFAULT NULL,
`mortgages_num_satisfied` smallint(5) unsigned DEFAULT NULL,
`partnerships_num_gen_partners` smallint(5) unsigned DEFAULT NULL,
`partnerships_num_lim_partners` smallint(5) unsigned DEFAULT NULL,
`ext_name` varchar(255) DEFAULT NULL,
`turnover` decimal(18,2) DEFAULT NULL,
`turnover_date` date DEFAULT NULL,
`trade_debtors` decimal(18,2) DEFAULT NULL,
`other_debtors` decimal(18,2) DEFAULT NULL,
`debtors_date` date DEFAULT NULL,
`real_turnover_band` int(11) DEFAULT NULL,
`est_turnover_band` int(11) DEFAULT NULL,
`ext_address_date` date DEFAULT NULL,
`care_of` varchar(255) DEFAULT NULL,
`po_box` varchar(60) DEFAULT NULL,
`line_1` varchar(255) DEFAULT NULL,
`line_2` varchar(255) DEFAULT NULL,
`town` varchar(60) DEFAULT NULL,
`county` varchar(60) DEFAULT NULL,
`country` varchar(60) DEFAULT NULL,
`post_code` varchar(20) DEFAULT NULL,
`DirScrapeID` int(10) DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `homepage_keywords_stamp` (`keywords_stamp`),
KEY `number` (`number`),
KEY `url` (`url`),
KEY `town` (`town`),
KEY `county` (`county`),
KEY `post_code` (`post_code`),
KEY `name` (`name`),
KEY `website_stamp` (`website_stamp`),
KEY `website_stamp_start` (`website_stamp_start`),
KEY `keywords_stamp_start` (`keywords_stamp_start`),
KEY `turnover` (`turnover`),
KEY `status` (`status`),
KEY `category` (`category`),
KEY `incorporation_date` (`incorporation_date`),
KEY `real_turnover_band` (`real_turnover_band`),
KEY `est_turnover_band` (`est_turnover_band`)
) ENGINE=InnoDB AUTO_INCREMENT=3706459 DEFAULT CHARSET=utf8
临时公司:
CREATE TABLE `temp_company` (
`id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`name` varchar(255) DEFAULT NULL,
`url` varchar(255) DEFAULT NULL,
`ch_url` varchar(255) DEFAULT NULL,
`number` varchar(30) DEFAULT NULL,
`category` varchar(255) DEFAULT NULL,
`status` varchar(255) DEFAULT NULL,
`country_of_origin` varchar(80) DEFAULT NULL,
`dissolution_date` date DEFAULT NULL,
`incorporation_date` date DEFAULT NULL,
`account_ref_day` smallint(5) unsigned DEFAULT NULL,
`account_ref_month` smallint(5) unsigned DEFAULT NULL,
`account_next_due_date` date DEFAULT NULL,
`account_last_made_up_date` date DEFAULT NULL,
`account_category` varchar(255) DEFAULT NULL,
`returns_next_due_date` date DEFAULT NULL,
`returns_last_made_up_date` date DEFAULT NULL,
`mortgages_num_charges` smallint(5) unsigned DEFAULT NULL,
`mortgages_num_outstanding` smallint(5) unsigned DEFAULT NULL,
`mortgages_num_part_satisfied` smallint(5) unsigned DEFAULT NULL,
`mortgages_num_satisfied` smallint(5) unsigned DEFAULT NULL,
`partnerships_num_gen_partners` smallint(5) unsigned DEFAULT NULL,
`partnerships_num_lim_partners` smallint(5) unsigned DEFAULT NULL,
`ext_name` varchar(255) DEFAULT NULL,
`turnover` decimal(18,2) DEFAULT NULL,
`turnover_date` date DEFAULT NULL,
`trade_debtors` decimal(18,2) DEFAULT NULL,
`other_debtors` decimal(18,2) DEFAULT NULL,
`debtors_date` date DEFAULT NULL,
`real_turnover_band` int(11) DEFAULT NULL,
`est_turnover_band` int(11) DEFAULT NULL,
`ext_address_date` date DEFAULT NULL,
`care_of` varchar(255) DEFAULT NULL,
`po_box` varchar(60) DEFAULT NULL,
`line_1` varchar(255) DEFAULT NULL,
`line_2` varchar(255) DEFAULT NULL,
`town` varchar(60) DEFAULT NULL,
`county` varchar(60) DEFAULT NULL,
`country` varchar(60) DEFAULT NULL,
`post_code` varchar(20) DEFAULT NULL,
`sic_code` varchar(10) DEFAULT NULL,
`DirScrapeID` int(10) DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `number` (`number`),
KEY `status` (`status`),
KEY `name` (`name`),
KEY `sic_code` (`sic_code`)
) ENGINE=InnoDB AUTO_INCREMENT=3297833 DEFAULT CHARSET=utf8
更新 2:查询的配置文件(限制为 5)
+-------------------------------+----------+
| Status | Duration |
+-------------------------------+----------+
| executing | 0.000001 |
| Sending data | 0.000112 |
| executing | 0.000001 |
| Sending data | 0.000111 |
| executing | 0.000001 |
| Sending data | 0.000110 |
| executing | 0.000001 |
| Sending data | 0.000110 |
| executing | 0.000001 |
| Sending data | 0.000110 |
| executing | 0.000001 |
| Sending data | 0.000111 |
| executing | 0.000001 |
| Sending data | 0.000111 |
| executing | 0.000001 |
| Sending data | 0.000112 |
| executing | 0.000001 |
| Sending data | 0.000112 |
| executing | 0.000001 |
| Sending data | 0.000112 |
| executing | 0.000001 |
| Sending data | 0.000112 |
| executing | 0.000001 |
| Sending data | 0.000112 |
| executing | 0.000001 |
| Sending data | 0.000112 |
| executing | 0.000001 |
| Sending data | 0.000113 |
| executing | 0.000001 |
| Sending data | 0.000114 |
| executing | 0.000001 |
| Sending data | 0.000114 |
| executing | 0.000001 |
| Sending data | 0.000114 |
| executing | 0.000001 |
| Sending data | 0.000115 |
| executing | 0.000001 |
| Sending data | 0.000116 |
| executing | 0.000001 |
| Sending data | 0.000115 |
| executing | 0.000001 |
| Sending data | 0.000115 |
| executing | 0.000001 |
| Sending data | 0.000116 |
| executing | 0.000001 |
| Sending data | 0.000116 |
| executing | 0.000001 |
| Sending data | 0.000115 |
| executing | 0.000001 |
| Sending data | 0.000115 |
| executing | 0.000001 |
| Sending data | 0.000116 |
| executing | 0.000001 |
| Sending data | 0.000116 |
| executing | 0.000001 |
| Sending data | 0.000117 |
| executing | 0.000001 |
| Sending data | 0.000117 |
| executing | 0.000001 |
| Sending data | 0.000117 |
| executing | 0.000001 |
| Sending data | 0.000118 |
| executing | 0.000001 |
| Sending data | 0.000118 |
| executing | 0.000001 |
| Sending data | 0.000118 |
| executing | 0.000001 |
| Sending data | 0.000118 |
| executing | 0.000001 |
| Sending data | 0.000118 |
| executing | 0.000001 |
| Sending data | 0.000118 |
| executing | 0.000001 |
| Sending data | 0.000120 |
| executing | 0.000001 |
| Sending data | 0.000120 |
| executing | 0.000001 |
| Sending data | 0.000121 |
| executing | 0.000001 |
| Sending data | 0.000123 |
| executing | 0.000001 |
| Sending data | 0.000121 |
| executing | 0.000001 |
| Sending data | 0.000120 |
| executing | 0.000001 |
| Sending data | 0.000121 |
| executing | 0.000001 |
| Sending data | 0.000121 |
| executing | 0.000001 |
| Sending data | 0.000121 |
| executing | 0.000001 |
| Sending data | 0.000122 |
| executing | 0.000001 |
| Sending data | 0.000123 |
| executing | 0.000001 |
| Sending data | 0.000124 |
| executing | 0.000001 |
| Sending data | 1.063880 |
| end | 0.000009 |
| query end | 0.000008 |
| closing tables | 0.000009 |
| freeing items | 0.000007 |
| Waiting for query cache lock | 0.000002 |
| freeing items | 0.000062 |
| Waiting for query cache lock | 0.000002 |
| freeing items | 0.000001 |
| storing result in query cache | 0.000002 |
| cleaning up | 0.000028 |
+-------------------------------+----------+
最佳答案
我不知道为什么它突然运行得更慢了，但我建议转换为一个连接，它应该表现更好:
SELECT t.*
FROM temp_company t
LEFT JOIN company c ON c.number = t.number
WHERE c.number is null
这是通过联接处理 not in (...) 的一种相当标准的方法，并且有效，因为 不 匹配的外部联接在连接表的列。
关于MySQL 升级后 MySQL NOT IN 查询速度变慢，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/21074717/