任何带有选项的命令/wget?
对于多线程下载,是否需要递归同时进行网站?
最佳答案
我找到了一个不错的解决方案。
在http://www.linuxquestions.org/questions/linux-networking-3/wget-multi-threaded-downloading-457375/阅读原件
wget -r -np -N [url] & wget -r -np -N [url] & wget -r -np -N [url] & wget -r -np -N [url] &
copied as many times as you deem fitting to have as much processes downloading. This isn't as elegant as a properly multithreaded app, but it will get the job done with only a slight amount of over head. the key here being the "-N" switch. This means transfer the file only if it is newer than what's on the disk. This will (mostly) prevent each process from downloading the same file a different process already downloaded, but skip the file and download what some other process hasn't downloaded. It uses the time stamp as a means of doing this, hence the slight overhead.
It works great for me and saves a lot of time. Don't have too many processes as this may saturate the web site's connection and tick off the owner. Keep it around a max of 4 or so. However, the number is only limited by CPU and network bandwidth on both ends.
关于multithreading - 多个wget -r同时访问一个站点?,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/4745799/