我正在使用 HtmlUnit 来尝试抓取网页,因为它支持 Javascript。 (我宁愿使用 Jsoup,但不支持 JS)。
该问题与底层 NekoHtml 解析器的一项功能有关: “http://cyberneko.org/html/features/scanner/allow-selfclosing-iframe”
参见:http://nekohtml.sourceforge.net/settings.html
这显然可以在 Neko 中启用,但我使用的是 HtmlUnit。有没有办法配置 HTML 单元使用的底层 Neko 解析器来启用此功能?
尝试运行此代码时:
final WebClient webClient = new WebClient();
HtmlPage page = webClient.getPage(url.toString());
我收到此错误:
Caused by: com.gargoylesoftware.htmlunit.ObjectInstantiationException: unable to create HTML parser
at com.gargoylesoftware.htmlunit.html.HTMLParser$HtmlUnitDOMBuilder.<init>(HTMLParser.java:418)
at com.gargoylesoftware.htmlunit.html.HTMLParser$HtmlUnitDOMBuilder.<init>(HTMLParser.java:342)
at com.gargoylesoftware.htmlunit.html.HTMLParser.parse(HTMLParser.java:203)
at com.gargoylesoftware.htmlunit.html.HTMLParser.parseHtml(HTMLParser.java:179)
at com.gargoylesoftware.htmlunit.DefaultPageCreator.createHtmlPage(DefaultPageCreator.java:221)
at com.gargoylesoftware.htmlunit.DefaultPageCreator.createPage(DefaultPageCreator.java:106)
at com.gargoylesoftware.htmlunit.WebClient.loadWebResponseInto(WebClient.java:433)
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:311)
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:373)
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:358)
at
Caused by: org.xml.sax.SAXNotRecognizedException: Feature 'http://cyberneko.org/html/features/scanner/allow-selfclosing-iframe' is not recognized.
at org.apache.xerces.parsers.AbstractSAXParser.setFeature(Unknown Source)
at com.gargoylesoftware.htmlunit.html.HTMLParser$HtmlUnitDOMBuilder.<init>(HTMLParser.java:411)
... 41 more
最佳答案
尝试使用 FF 行为初始化 Web 客户端
WebClient webClient = new WebClient(BrowserVersion.FIREFOX_3_6);
并激活 JavaScript
webClient.setJavaScriptEnabled(true);
那就应该没问题了。
关于java - 使用 HtmlUnit 时,如何配置底层 NekoHtml 解析器?,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/11138875/