我们正在使用由 mod_jk 编织在一起的 apache 和 tomcat 来托管一个大型商业应用程序。今天，应用程序停止响应(我们的nagios 提示)，所以我看了一下。我发现内存中有大量的 apache 进程:
8275 ? S 0:00 /usr/sbin/apache2 -k start
8280 ? S 0:00 /usr/sbin/apache2 -k start
8285 ? S 0:00 /usr/sbin/apache2 -k start
8286 ? S 0:00 /usr/sbin/apache2 -k start
8287 ? S 0:00 /usr/sbin/apache2 -k start
只有很多很多。所以，我重新启动了 apache，希望它能自行清理。它工作了一点，然后又显示了大量的进程。我用过
while x=0; do ps ax|grep apache2|wc -l; sleep 2; done
监控进程，大约需要 3 分钟才能达到最大进程数，然后 apache 错误日志显示
[Wed Oct 19 09:43:01 2011] [error] server reached MaxClients setting, consider raising the MaxClients setting
[Wed Oct 19 09:50:43 2011] [notice] caught SIGTERM, shutting down
[Wed Oct 19 09:50:43 2011] [warn] No JkShmFile defined in httpd.conf. Using default /var/log/apache2/jk-runtime-status
[Wed Oct 19 09:50:44 2011] [warn] No JkShmFile defined in httpd.conf. Using default /var/log/apache2/jk-runtime-status
[Wed Oct 19 09:50:44 2011] [notice] mod_python: Creating 8 session mutexes based on 256 max processes and 0 max threads.
[Wed Oct 19 09:50:44 2011] [notice] mod_python: using mutex_directory /tmp
[Wed Oct 19 09:50:44 2011] [notice] Apache/2.2.14 (Debian) mod_jk/1.2.28 PHP/5.2.11-1 with Suhosin-Patch mod_python/3.3.1 Python/2.5.4 mod_ssl/2.2.14 OpenSSL/0.9.8o mod_perl/2.0.4 Perl/v5.10.1 configured -- resuming normal operations
[Wed Oct 19 09:52:53 2011] [error] server reached MaxClients setting, consider raising the MaxClients setting
因此，在谷歌搜索了一下后，我发现 MaxClient 设置的数字 256 应该对任何人都足够了。所以我们还有另一个问题。 apache 唯一写入的其他日志是 other_vhosts_access.log，它看起来像这样:
www.firstdomain.com:80 157.55.16.55 - - [19/Oct/2011:09:40:59 +0200] "GET /sortiments2.jsp?catalog_id=0&category_id=10376104&codesk=632983007112&codesk=632983005866&codesk=632983005880&codesk=632983005248 HTTP/1.1" 502 0 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
www.firstdomain.com:80 91.36.177.136 - - [19/Oct/2011:09:41:21 +0200] "GET / HTTP/1.1" 503 548 "http://www.office1837.de/?gclid=CMjy7rqf9KsCFUi_zAodq1e0sQ" "Mozilla/5.0 (Windows NT 6.1; rv:7.0.1) Gecko/20100101 Firefox/7.0.1"
www.firstdomain.com:80 195.68.198.11 - - [19/Oct/2011:09:40:18 +0200] "GET /article.jsp?article_id=111083460 HTTP/1.1" 502 522 "http://www.3m-buerosortiment.de/index2.php?option=com_virtuemart&page=shop.CC.php&ean=3134375221399&prodname=Post-it\xc2\xae Notes 654NGR&sku=654NGR" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Trident/4.0; BTRS6329; .NET CLR 2.0.50727)"
www.firstdomain.com:80 157.55.16.55 - - [19/Oct/2011:09:40:59 +0200] "GET /sortiments.jsp?catalog_id=0&category_id=11826686 HTTP/1.0" 502 0 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
www.firstdomain.com:80 65.52.104.90 - - [19/Oct/2011:09:41:52 +0200] "GET /robots.txt HTTP/1.1" 200 0 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
www.firstdomain.com:80 207.46.199.45 - - [19/Oct/2011:09:41:29 +0200] "GET /sortiments2.jsp?&category_id=9958253&catalog_id=0&codesk=4977766659659&codesk=4977766609449&codesk=4977766611091&codesk=4977766643900&codesk=4977766648523&imagesize=1&image_id=20997873 HTTP/1.1" 503 0 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
www.firstdomain.com:80 207.46.12.238 - - [19/Oct/2011:09:41:29 +0200] "GET /supplierSortiments2.jsp?category_id=10996252&supplier=Brother&codesk=4977766643870&codesk=4977766659567&codesk=4977766659871&codesk=4977766643900 HTTP/1.0" 503 0 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
www.firstdomain.com:80 207.46.12.238 - - [19/Oct/2011:09:41:29 +0200] "GET /sortiments2.jsp?catalog_id=127&category_id=10996604&codesk=8808979282463&codesk=8808987560621&codesk=8808987560621&codesk=8808993448104 HTTP/1.1" 503 0 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
www.seconddomain.com:80 87.173.152.50 - - [19/Oct/2011:09:41:29 +0200] "GET / HTTP/1.0" 503 542 "http://www.staehlin.de/home.php/main/index/buerowelt/buerobedarf" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; InfoPath.1; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)"
www.firstdomain.com:80 79.239.149.63 - - [19/Oct/2011:09:41:29 +0200] "GET / HTTP/1.1" 503 0 "http://www.streit.de/" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.0.3705; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)"
www.seconddomain.com:80 82.192.74.11 - - [19/Oct/2011:09:40:28 +0200] "GET /sortiments2.jsp?add_temp_article=true&article_domain=&article_id=315001460 HTTP/1.0" 502 484 "-" "TwengaBot-2.0 (http://www.twenga.com/bot.html)"
www.seconddomain.com:80 67.195.111.35 - - [19/Oct/2011:09:41:34 +0200] "GET /sortiments2.jsp?add_temp_article=true&article_domain=&article_id=111217102 HTTP/1.0" 503 541 "-" "Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)"
www.firstdomain.com:80 213.164.69.193 - - [19/Oct/2011:09:41:34 +0200] "GET / HTTP/1.1" 503 542 "http://www.resin.de/de/onlineshops.php" "Mozilla/5.0 (Windows NT 6.1; rv:7.0.1) Gecko/20100101 Firefox/7.0.1"
www.firstdomain.com:80 217.91.140.166 - - [19/Oct/2011:09:41:13 +0200] "GET /feed/offers.jsp?type=rss_2.0 HTTP/1.1" 200 968 "-" "Apple-PubSub/28"
www.seconddomain.com:80 82.192.74.5 - - [19/Oct/2011:09:47:53 +0200] "GET /article.jsp?article_id=124009100&category_id=12758254&catalog_id=154 HTTP/1.0" 500 1866 "-" "TwengaBot-2.0 (http://www.twenga.com/bot.html)"
www.firstdomain.com:80 82.192.74.5 - - [19/Oct/2011:09:41:55 +0200] "GET /redirector.jsp?&catalog_id=10&category_id=12752165 HTTP/1.0" 302 294 "-" "TwengaBot-2.0 (http://www.twenga.com/bot.html)"
www.firstdomain.com:80 87.139.37.126 - - [19/Oct/2011:09:40:32 +0200] "GET / HTTP/1.1" 503 543 "http://www.bueroschaal.de/" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0; Trident/4.0; SLCC1; .NET CLR 2.0.50727; InfoPath.2; .NET CLR 3.5.30729; OfficeLiveConnector.1.3; OfficeLivePatch.0.0; .NET CLR 3.0.30729)"
www.seconddomain.com:80 79.234.14.178 - - [19/Oct/2011:09:41:35 +0200] "GET /searchresult.jsp?position=90&searchstring=tacker HTTP/1.1" 503 0 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 2.0.50727; InfoPath.1; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; InfoPath.2)"
www.firstdomain.com:80 207.46.12.238 - - [19/Oct/2011:09:40:33 +0200] "GET /robots.txt HTTP/1.1" 502 0 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
www.seconddomain.com:80 88.130.168.74 - - [19/Oct/2011:09:41:43 +0200] "GET / HTTP/1.1" 503 540 "-" "Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10_4_11; de-de) AppleWebKit/533.19.4 (KHTML, like Gecko) Version/4.1.3 Safari/533.19.4"
www.firstdomain.com:80 195.68.198.10 - - [19/Oct/2011:09:40:43 +0200] "GET /article.jsp?article_id=940126050 HTTP/1.1" 503 547 "http://www.3m-buerosortiment.de/index2.php?option=com_virtuemart&page=shop.CC.php&ean=3134375221399&prodname=Post-it\xc2\xae Notes 654NGR&sku=654NGR" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Trident/4.0; BTRS6329; .NET CLR 2.0.50727)"
www.seconddomain.com:80 87.139.126.238 - - [19/Oct/2011:09:42:40 +0200] "POST /searchresult.jsp;jsessionid=CCA8181DB579814B8F317CCE61F9FC16 HTTP/1.1" 302 399 "http://salfer.firstdomain.com/" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; InfoPath.3; .NET4.0C; .NET4.0E)"
www.firstdomain.com:80 67.195.114.43 - - [19/Oct/2011:09:48:03 +0200] "GET /robots.txt HTTP/1.0" 200 423 "-" "Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)"
www.firstdomain.com:80 217.91.140.166 - - [19/Oct/2011:09:41:34 +0200] "GET /shoppingcart.jsp HTTP/1.1" 200 6328 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/534.51.22 (KHTML, like Gecko) Version/5.1.1 Safari/534.51.22"
www.firstdomain.com:80 207.46.199.203 - - [19/Oct/2011:09:42:59 +0200] "GET /robots.txt HTTP/1.1" 200 0 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
www.seconddomain.com:80 83.64.224.10 - - [19/Oct/2011:09:40:46 +0200] "GET /article.jsp?&searchstring=hp+17&article_id=101231800&imagesize=2&image_id=8301473 HTTP/1.1" 503 0 "-" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; GTB7.1; .NET CLR 1.1.4322)"
www.seconddomain.com:80 188.195.58.227 - - [19/Oct/2011:09:40:31 +0200] "GET / HTTP/1.1" 502 0 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET4.0C)"
www.firstdomain.com:80 207.46.12.238 - - [19/Oct/2011:09:40:32 +0200] "GET /article.jsp?article_id=5006056&category_id=10549957&catalog_id=148 HTTP/1.0" 502 0 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
www.seconddomain.com:80 84.138.10.184 - - [19/Oct/2011:09:41:57 +0200] "GET /index.jsp HTTP/1.1" 503 630 "-" "Mozilla/5.0 (Windows NT 5.1; rv:6.0) Gecko/20100101 Firefox/6.0"
那是很正常的路况。所以，从这一点开始，我不知道如何确定问题是什么。如果我能以某种方式看到/确定:
- 为什么 apache 会生成这些新进程以及
- 哪些请求启动了一个新进程。
如果需要，我已经准备好 netstat，但它只显示正常连接，正如我所期望的那样。
编辑 1:当问题开始时，mod_jk.log 开始发布这些消息:
[Wed Oct 19 10:09:23.336 2011] [8952:3623802672] [info] ajp_service::jk_ajp_common.c (2447): (ajp13_worker) sending request to tomcat failed (recoverable), (attempt=2)
[Wed Oct 19 10:09:23.336 2011] [8952:3623802672] [error] ajp_service::jk_ajp_common.c (2466): (ajp13_worker) connecting to tomcat failed.
[Wed Oct 19 10:09:23.336 2011] [8952:3623802672] [info] jk_handler::mod_jk.c (2615): Service error=0 for worker=ajp13_worker
[Wed Oct 19 10:09:26.500 2011] [9207:3623802672] [info] jk_open_socket::jk_connect.c (594): connect to 127.0.0.1:8009 failed (errno=110)
[Wed Oct 19 10:09:26.500 2011] [9207:3623802672] [info] ajp_connect_to_endpoint::jk_ajp_common.c (922): Failed opening socket to (127.0.0.1:8009) (errno=110)
[Wed Oct 19 10:09:26.500 2011] [9207:3623802672] [error] ajp_send_request::jk_ajp_common.c (1507): (ajp13_worker) connecting to backend failed. Tomcat is probably not started or is listening on the wrong port (errno=110)
[Wed Oct 19 10:09:26.500 2011] [9207:3623802672] [info] ajp_service::jk_ajp_common.c (2447): (ajp13_worker) sending request to tomcat failed (recoverable), because of error during request sending (attempt=2)
[Wed Oct 19 10:09:26.500 2011] [9207:3623802672] [error] ajp_service::jk_ajp_common.c (2466): (ajp13_worker) connecting to tomcat failed.
[Wed Oct 19 10:09:26.500 2011] [9207:3623802672] [info] jk_handler::mod_jk.c (2615): Service error=-3 for worker=ajp13_worker
最佳答案
Apache mod_status 会告诉您所有进程正在做什么。
我遇到过类似的情况，当我查看时，我发现有 1 个 IP 地址试图在几乎所有线程中下载相同的 1Mb 文件，所以我使用 Deny from [address] 阻止了该地址。 ，我的服务器现在运行顺畅。
关于apache - 许多 Apache 进程生成速度非常快，最多可在 3 分钟内达到极限，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/7818233/