hadoop - 同时处理与主机的多个连接

Question

如何同时处理与主机的多个连接？

Answer 1

来自 nutch-default.xml:

<property>
  <name>fetcher.threads.fetch</name>
  <value>10</value>
  <description>The number of FetcherThreads the fetcher should use.
    This is also determines the maximum number of requests that are 
    made at once (each FetcherThread handles one connection).</description>
</property>

<property>
  <name>fetcher.threads.per.host</name>
  <value>1</value>
  <description>This number is the maximum number of threads that
    should be allowed to access a host at one time.</description>
</property>

如上所述，连接数最多等于线程数。第一个属性控制连接总数，第二个属性控制每个主机的连接数 - 这是您需要设置的。