我有一个 url 数组,我想将我在数据库中读取的 url 中的信息存储起来。我的问题是我的数据列表太大 url 如果从上面读取序列化每个 url 存储在数据库中将需要时间。
我知道有一种方法可以使用线程来操作,但我不知道该怎么做,请帮助我。或者任何你的方法
try {
String lstUrls = "http://www.java2s.com/Tutorials/Java/Scala/index.htm\n"
+ "http://www.java2s.com/Tutorials/Java/Scala/0020__Scala_Variables.htm\n"
+ "http://www.java2s.com/Tutorials/Java/Scala/0040__Scala_Variable_Declarations.htm\n"
+ "http://www.java2s.com/Tutorials/Java/Scala/0060__Scala_Semicolons.htm\n"
+ "http://www.java2s.com/Tutorials/Java/Scala/0080__Scala_Code_Blocks.htm\n"
+ "http://www.java2s.com/Tutorials/Java/Scala/0090__Scala_Comments.htm\n"
+ "http://www.java2s.com/Tutorials/Java/Scala/0100__Scala_Type_Hierarchy.htm\n";
String[] urls = lstUrls.split("\n");
for (String url : urls) {
Document doc = Jsoup.connect(url).userAgent("Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/33.0.1750.152 Safari/537.36").get();
Elements select = doc.select("div.row");
String html = select.html();
System.out.println(html);
/*
insert html to database
*/
}
} catch (IOException ex) {
ex.printStackTrace();
}
最佳答案
要使用多线程来检索数据,您可以这样做:
Executor ex = Executors.newFixedThreadPool(3);
String lstUrls = "http://www.java2s.com/Tutorials/Java/Scala/index.htm\n"
+ "http://www.java2s.com/Tutorials/Java/Scala/0020__Scala_Variables.htm\n"
+ "http://www.java2s.com/Tutorials/Java/Scala/0040__Scala_Variable_Declarations.htm\n"
+ "http://www.java2s.com/Tutorials/Java/Scala/0060__Scala_Semicolons.htm\n"
+ "http://www.java2s.com/Tutorials/Java/Scala/0080__Scala_Code_Blocks.htm\n"
+ "http://www.java2s.com/Tutorials/Java/Scala/0090__Scala_Comments.htm\n"
+ "http://www.java2s.com/Tutorials/Java/Scala/0100__Scala_Type_Hierarchy.htm\n";
String[] urls = lstUrls.split("\n");
for (final String url : urls) {
try {
ex.execute(new Runnable() {
@Override
public void run() {
try {
Document doc = Jsoup
.connect(url)
.userAgent(
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/33.0.1750.152 Safari/537.36")
.get();
Elements select = doc.select("div.row");
String html = select.html();
System.out.println(html);
/*
* insert html to database
*/
} catch (Exception e) {
e.printStackTrace();
}
}
});
} catch (Exception e) {
e.printStackTrace();
}
}
这将使用 3 个线程并发处理 url,如果你想使用更多的 3 个线程,请更改此行 Executor ex = Executors.newFixedThreadPool(3);
并替换 3
用你想要的任何数字。
您可以找到更多关于 Executors here 的信息
关于java - Jsoup 将内容存入数据库,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/33193462/