java - HttpClient 的多线程问题

Question

我遇到了 HttpClient 的多线程问题，我遇到了以下情况:

线程 A 将发出 url http://blap.com?param=2

线程 B 将发出 url http://blap.com?param=3

这在大约 98% 的时间都有效，但偶尔线程 A 会收到线程 B 的 url 的数据，反之亦然。

现在每个线程都在创建它自己的 HttpClient 实例，所以我认为理论上我不需要使用 MultiThreadedHttpConnectionManager。

我描述的行为是否合理，是否可以通过使用 MultiThreadedHttpConnectionManager 来解决？

我使用的是 java 1.6 和 apache http 客户端组件 4.0.3。

更新: 这是有问题的功能。

public void get_url(String strDataSet) throws SQLException, MalformedURLException, IOException
{

      String query;



        query = "select * from jobs where data_set='" + strDataSet + "'";

        ResultSet rs2 = dbf.db_run_query(query);
        rs2.next();


        HttpClient httpclient = new DefaultHttpClient();
        HttpResponse response;



            String strURL;
            strURL = rs2.getString("url_static");

            if (rs2.getString("url_dynamic")!=null && !rs2.getString("url_dynamic").isEmpty())
                strURL = strURL.replace("${dynamic}", rs2.getString("url_dynamic"));

            UtilityFunctions.stdoutwriter.writeln("Retrieving URL: " + strURL,Logs.STATUS2,"DG25");

            if (!strURL.contains(":"))
                UtilityFunctions.stdoutwriter.writeln("WARNING: url is not preceeded with a protocol" + strURL,Logs.STATUS1,"DG25.5");

            //HttpGet chokes on the ^ character

            strURL = strURL.replace("^","%5E");


            HttpGet httpget = new HttpGet(strURL); 


            /*
             * The following line fixes an issue where a non-fatal error is displayed about an invalid cookie data format.
             * It turns out that some sites generate a warning with this code, and others without it.
             * I'm going to kludge this for now until I get more data on which urls throw the
             * warning and which don't.
             * 
             * warning with code: www.exchange-rates.org
             */


                if (!(strCurDataSet.contains("xrateorg") || strCurDataSet.contains("google") || strCurDataSet.contains("mwatch")))
                {
                    httpget.getParams().setParameter("http.protocol.cookie-datepatterns", 
                            Arrays.asList("EEE, dd MMM-yyyy-HH:mm:ss z", "EEE, dd MMM yyyy HH:mm:ss z"));
                }







            response = httpclient.execute(httpget);




         HttpEntity entity = response.getEntity();

          BufferedReader in = new BufferedReader(
                    new InputStreamReader(
                    entity.getContent()));



      int nTmp;         

      returned_content="";




      while ((nTmp = in.read()) != -1)
        returned_content = returned_content + (char)nTmp;


      in.close();

      httpclient.getConnectionManager().shutdown();

      UtilityFunctions.stdoutwriter.writeln("Done reading url contents",Logs.STATUS2,"DG26");



}

更新: 我将问题缩小到以下行:

response = httpclient.execute(httpget);

如果我在该行周围放置一个线程锁，问题就消失了。问题是，这是最耗时的部分，我不希望一次只有一个线程能够检索 http 数据。

Answer 1

您的代码不是线程安全的。要解决眼前的问题，您需要将 HttpClient 声明为 ThreadLocal，但还有很多问题需要解决。