java - 在java中获取google的搜索结果

Question

我在这个程序中得到了 403 响应码，但我需要得到 200 才能返回搜索结果，我该怎么办？

      String url="http://www.google.com/search?q=";
      String charset="UTF-8";
      String key="java";
      String query = String.format("%s",URLEncoder.encode(key, charset));
      URLConnection con = new URL(url+ query).openConnection();
      BufferedReader in = new BufferedReader(new InputStreamReader(con.getInputStream()));
      String inputLine;
      while ((inputLine = in.readLine()) != null) 
      System.out.println(inputLine);
      in.close();

Answer 1

试试JSoup

Document document = Jsoup
        .connect("http://www.google.com/search?q=" + query)
        .userAgent("Mozilla/5.0 (Windows NT 6.1; WOW64; rv:5.0) Gecko/20100101 Firefox/5.0")     
        .get();

System.out.println(document.html());

要提取链接，请使用 selector api .

依赖:

<dependency>
  <!-- jsoup HTML parser library @ http://jsoup.org/ -->
  <groupId>org.jsoup</groupId>
  <artifactId>jsoup</artifactId>
  <version>1.7.3</version>
</dependency>