linux - wget 一起使用 --timeout 和 --tries

标签 linux bash wget

您希望此命令在退出前需要多长时间?

wget --timeout=1 --tries=2 "http://www.google.com:81/not-there"

我希望超时为 1 秒,2 次尝试意味着 2 秒,但它需要 6.025 秒

wget --timeout=1 --tries=2 "http://www.google.com:81/not-there"
--2017-04-27 16:49:12--http://www.google.com:81/not-there
Resolving www.google.com (www.google.com)... 209.85.203.105, 209.85.203.103, 209.85.203.99, ...
Connecting to www.google.com (www.google.com)|209.85.203.105|:81... failed: Connection timed out.
Connecting to www.google.com (www.google.com)|209.85.203.103|:81... failed: Connection timed out.
Connecting to www.google.com (www.google.com)|209.85.203.99|:81... failed: Connection timed out.
Connecting to www.google.com (www.google.com)|209.85.203.104|:81... failed: Connection timed out.
Connecting to www.google.com (www.google.com)|209.85.203.106|:81... failed: Connection timed out.
Connecting to www.google.com (www.google.com)|209.85.203.147|:81... failed: Connection timed out.
Connecting to www.google.com (www.google.com)|2a00:1450:4009:80d::2004|:81... failed: Network is unreachable.

我不知道为什么它在退出前尝试了 6 次。

tries 好像是重试。如果我设置 --retry-connrefused 它至少会重试,但所花费的总时间仍然不是我所期望的。我希望能够决定在超时时应该尝试多少次。

编辑:

在@Socowi 的建议下,我尝试将 waitretry 与 retry-connrefused 结合使用并得到相同的行为:

$ wget --timeout=1 --waitretry=0 --tries=2 --retry-connrefused "http://www.google.com:81/not-there"
--2017-04-27 20:29:47--  http://www.google.com:81/not-there
Resolving www.google.com (www.google.com)... 2a00:1450:400b:c00::68, 209.85.203.99, 209.85.203.147, ...
Connecting to www.google.com (www.google.com)|2a00:1450:400b:c00::68|:81... failed: Connection timed out.
Connecting to www.google.com (www.google.com)|209.85.203.99|:81... failed: Connection timed out.
Connecting to www.google.com (www.google.com)|209.85.203.147|:81... failed: Connection timed out.
Connecting to www.google.com (www.google.com)|209.85.203.103|:81... failed: Connection timed out.
Connecting to www.google.com (www.google.com)|209.85.203.104|:81... failed: Connection timed out.
Connecting to www.google.com (www.google.com)|209.85.203.106|:81... failed: Connection timed out.
Connecting to www.google.com (www.google.com)|209.85.203.105|:81... failed: Connection timed out.
Retrying.

--2017-04-27 20:29:54--  (try: 2)  http://www.google.com:81/not-there
Connecting to www.google.com (www.google.com)|2a00:1450:400b:c00::68|:81... failed: Connection timed out.
Connecting to www.google.com (www.google.com)|209.85.203.99|:81... failed: Connection timed out.
Connecting to www.google.com (www.google.com)|209.85.203.147|:81... failed: Connection timed out.
Connecting to www.google.com (www.google.com)|209.85.203.103|:81... failed: Connection timed out.
Connecting to www.google.com (www.google.com)|209.85.203.104|:81... failed: Connection timed out.
Connecting to www.google.com (www.google.com)|209.85.203.106|:81... failed: Connection timed out.
Connecting to www.google.com (www.google.com)|209.85.203.105|:81... failed: Connection timed out.
Giving up.

编辑二

当两个相同的命令表现不同时,我真的很困惑。

root@8c59d6dd05fe:/var/www/html# wget --timeout=1 --waitretry=0 --tries=2 --retry-connrefused "http://www.google.com:81/not-there"
converted 'http://www.google.com:81/not-there' (ANSI_X3.4-1968) -> 'http://www.google.com:81/not-there' (UTF-8)
--2017-04-27 19:50:28--  http://www.google.com:81/not-there
Resolving www.google.com (www.google.com)... 216.58.211.164, 2a00:1450:4009:805::2004
Connecting to www.google.com (www.google.com)|216.58.211.164|:81... failed: Connection timed out.
Connecting to www.google.com (www.google.com)|2a00:1450:4009:805::2004|:81... failed: Cannot assign requested address.
Retrying.

--2017-04-27 19:50:29--  (try: 2)  http://www.google.com:81/not-there
Connecting to www.google.com (www.google.com)|216.58.211.164|:81... failed: Connection timed out.
Connecting to www.google.com (www.google.com)|2a00:1450:4009:805::2004|:81... failed: Cannot assign requested address.
Giving up.

root@8c59d6dd05fe:/var/www/html# wget --timeout=1 --waitretry=0 --tries=2 --retry-connrefused "http://www.google.com:81/not-there"
converted 'http://www.google.com:81/not-there' (ANSI_X3.4-1968) -> 'http://www.google.com:81/not-there' (UTF-8)
--2017-04-27 19:50:35--  http://www.google.com:81/not-there
Resolving www.google.com (www.google.com)... 209.85.203.104, 209.85.203.147, 209.85.203.106, ...
Connecting to www.google.com (www.google.com)|209.85.203.104|:81... failed: Connection timed out.
Connecting to www.google.com (www.google.com)|209.85.203.147|:81... failed: Connection timed out.
Connecting to www.google.com (www.google.com)|209.85.203.106|:81... failed: Connection timed out.
Connecting to www.google.com (www.google.com)|209.85.203.103|:81... failed: Connection timed out.
Connecting to www.google.com (www.google.com)|209.85.203.105|:81... failed: Connection timed out.
Connecting to www.google.com (www.google.com)|209.85.203.99|:81... failed: Connection timed out.
Connecting to www.google.com (www.google.com)|2a00:1450:400b:c03::68|:81... failed: Cannot assign requested address.
Retrying.

--2017-04-27 19:50:41--  (try: 2)  http://www.google.com:81/not-there
Connecting to www.google.com (www.google.com)|209.85.203.104|:81... failed: Connection timed out.
Connecting to www.google.com (www.google.com)|209.85.203.147|:81... failed: Connection timed out.
Connecting to www.google.com (www.google.com)|209.85.203.106|:81... failed: Connection timed out.
Connecting to www.google.com (www.google.com)|209.85.203.103|:81... failed: Connection timed out.
Connecting to www.google.com (www.google.com)|209.85.203.105|:81... failed: Connection timed out.
Connecting to www.google.com (www.google.com)|209.85.203.99|:81... failed: Connection timed out.
Connecting to www.google.com (www.google.com)|2a00:1450:400b:c03::68|:81... failed: Cannot assign requested address.
Giving up.

我以为我有点疯狂,但只有当@Socowi 在评论中指出每次重试的 IP 不同时,它才会点击。这取决于返回的可能 IP 的数量。我看到的神奇数字 7(重试)是 IP 选项的编号。当我只选择一个特定的 IP 时,它只会尝试一次。

最佳答案

重试次数错误

您的 wget 似乎将 URL 解析为多个 IP 地址,如 wget 的第二行所示的输出。然后使用指定的超时测试每个 IP。不幸的是,我没有找到任何选项来将 DNS 查找限制到一个地址或为所有 IP 一起设置总超时。但是你可以尝试使用 "<googles ip address>:81/not-there"而不是域名。

自动将域解析为您可以使用的单个 IP 地址

wget "http://$(getent hosts www.google.com | sed 's/ .*//;q'):81/not-there"

看似太长的超时

如您所知,设置 --retry-connrefusedwget即使在 »连接被拒绝« 错误后重试。指定的超时用于每次重试,但在重试之间会有一个暂停,每次重试后暂停时间会变长。

例子

wget --timeout=1 --tries=5 --retry-connrefused URL

做类似的事情

try to connect for 1 second
failed -> wait 1 second
try to connect for 1 second
failed -> wait 2 seconds
try to connect for 1 second
failed -> wait 3 second
try to connect for 1 second
failed -> wait 4 second
try to connect for 1 second

因此该命令需要 tries * timeout + 1 + 2 + ... + (tries - 1)秒。此行为在 man wget 中指定在选项下,它允许您更改它:)

--waitretry=seconds
If you don't want Wget to wait between every retrieval, but only between retries of failed downloads, you can use this option. Wget will use linear backoff, waiting 1 second after the first failure on a given file, then waiting 2 seconds after the second failure on that file, up to the maximum number of seconds you specify.

By default, Wget will assume a value of 10 seconds.

我想你想用类似的东西

wget --timeout=1 --waitretry=0 --tries=5 --retry-connrefused URL

这消除了两次重试之间的停顿,导致总时间为 timeout * tries .

关于linux - wget 一起使用 --timeout 和 --tries,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/43662591/

相关文章:

linux - 在 bash 中获取重定向位置

bash - 对下载文件的执行权限

linux - 使用 wget 时同时使用 -i 和 -O

python - 守护进程 python 脚本后找不到守护进程

Linux Debian 在启动时在 init 脚本中运行命令

linux - 查找命令结果,不会写入输出

regex - 为什么 ' grep "*.h"' and ' grep -E "*.h"' 在同一文件上的不同输出

linux - 如何停止在 bash 中通过 ctrl+c 终止脚本?

python - 检索完整的网页,包括动态加载的链接/图像

unix - 如何使用 wget --convert-links