正如标题所说,我正在尝试从 javascript 链接下载带有 HTMLUnit 的文件。
我开始的页面是 https://ppair.uspto.gov/TruePassWebStart/AuthenticationChooser.html .当我在浏览器中单击链接“使用 Java Web Start 进行身份验证(新方法)”时,将下载一个 .jnlp 文件,然后运行该文件以打开一个 Java 程序窗口,要求提供身份验证信息。身份验证成功后,原始浏览器窗口会加载包含我将要抓取的信息的页面。
起始页的链接源代码片段是:
<tr>
<!-- onClick="return launchWebStart('authenticate');" -->
<td><a href="javascript:void(0)" id="webstart-authenticate" ><font size="5">Authenticate with Java Web Start (new method)</font></a>
</tr>
用于此过程的 javascript 文件位于 https://ppair.uspto.gov/TruePassWebStart/js/WebStart.js。 .基本上,javascript 获取一个 cookie,对其进行编码,并将其附加到 URL 以请求 jnlp 文件。我考虑过只模拟这个过程,但 HTMLUnit
文档不鼓励这样做。 (他们说“通过单击元素并四处移动焦点来像用户那样操作页面要好得多”,所以我正在尝试这样做。)
我在 HTMLUnit 中遇到的问题是,在我单击 () 适当的 anchor 后,我无法接收预期的 jnlp 文件。我已经尝试了从本网站上的其他问题中发现的几种不同的方法,包括:
HtmlUnit and JavaScript in links 和 HtmlUnit to invoke javascript from href to download a file
这是我使用的代码:
import java.io.IOException;
import java.io.InputStream;
import java.net.MalformedURLException;
import com.gargoylesoftware.htmlunit.*;
import com.gargoylesoftware.htmlunit.html.HtmlAnchor;
import com.gargoylesoftware.htmlunit.html.HtmlPage;
public class Test {
public static void main(String[] args) throws FailingHttpStatusCodeException, MalformedURLException, IOException {
WebClient webClient = new WebClient(BrowserVersion.FIREFOX_45);
// open starting webpage
HtmlPage page = webClient.getPage("https://ppair.uspto.gov/TruePassWebStart/AuthenticationChooser.html");
// id of the element where the link is
String linkID = "webstart-authenticate";
// identify the appropriate anchor
HtmlAnchor anchor = (HtmlAnchor) page.getElementById(linkID);
// click the anchor
Page p = anchor.click();
// get the InputStream for the response; print it out
InputStream is = p.getWebResponse().getContentAsStream();
int b = 0;
while ((b = is.read()) != -1) {
System.out.print((char)b);
}
webClient.close();
}
}
从上面的代码打印出来的是起始网页的 html,而不是预期的 jnlp 文件。控制台还每 3 秒打印一次来自 javascript WebConsole 的状态更新(至少如果我让代码等待足够长的时间),所以我知道 javascript 发生了一些事情(函数 launchWebStart 和 followMediator 在单独的 javascript 文件 WebStart 中.js):
Nov 21, 2016 2:53:25 PM com.gargoylesoftware.htmlunit.WebConsole info
INFO: launchWebStart
Nov 21, 2016 2:53:25 PM com.gargoylesoftware.htmlunit.WebConsole info
INFO: followMediator
Nov 21, 2016 2:53:25 PM com.gargoylesoftware.htmlunit.WebConsole info
INFO: responseReceived:200
WAIT
Nov 21, 2016 2:53:25 PM com.gargoylesoftware.htmlunit.WebConsole info
INFO: mediatorCallback: next wait
我还尝试使用 CollectingAttachmentHandler 对象,如 downloading files behind javascript button with htmlunit 中所述:
import java.io.IOException;
import java.net.MalformedURLException;
import java.util.List;
import com.gargoylesoftware.htmlunit.*;
import com.gargoylesoftware.htmlunit.attachment.Attachment;
import com.gargoylesoftware.htmlunit.attachment.CollectingAttachmentHandler;
import com.gargoylesoftware.htmlunit.html.HtmlAnchor;
import com.gargoylesoftware.htmlunit.html.HtmlPage;
public class Test2 {
public static void main(String[] args) throws FailingHttpStatusCodeException, MalformedURLException, IOException {
WebClient webClient = new WebClient(BrowserVersion.FIREFOX_45);
// open starting webpage
HtmlPage page = webClient.getPage("https://ppair.uspto.gov/TruePassWebStart/AuthenticationChooser.html");
// id of the element where the link is
String linkID = "webstart-authenticate";
// identify the appropriate anchor
HtmlAnchor anchor = (HtmlAnchor) page.getElementById(linkID);
CollectingAttachmentHandler attachmentHandler = new CollectingAttachmentHandler();
webClient.setAttachmentHandler(attachmentHandler);
attachmentHandler.handleAttachment(anchor.click());
List<Attachment> attachments = attachmentHandler.getCollectedAttachments();
int i = 0;
while (i < attachments.size()) {
Attachment attachment = attachments.get(i);
Page attachedPage = attachment.getPage();
WebResponse attachmentResponse = attachedPage.getWebResponse();
String content = attachmentResponse.getContentAsString();
System.out.println(content);
i++;
}
webClient.close();
}
}
此代码还打印出起始网页的内容。所以其他解决方案似乎都不适合我。我不知道我做错了什么。关于如何让它工作,我的想法已经用完了(我认为这很容易!)非常感谢任何建议!
最佳答案
这是一个基于你的 Test2 的工作版本
WebClient webClient = new WebClient(BrowserVersion.FIREFOX_45);
// open starting webpage
HtmlPage page = webClient.getPage("https://ppair.uspto.gov/TruePassWebStart/AuthenticationChooser.html");
// id of the element where the link is
String linkID = "webstart-authenticate";
// identify the appropriate anchor
HtmlAnchor anchor = (HtmlAnchor) page.getElementById(linkID);
CountDownLatch latch = new CountDownLatch(1);
webClient.setWebStartHandler(new WebStartHandler(){
@Override
public void handleJnlpResponse(WebResponse webResponse)
{
System.out.println("downloading...");
try (FileOutputStream fos = new FileOutputStream("/Users/Franklyn/Downloads/uspto-auth.authenticate2.jnlp"))
{
IOUtils.copy(webResponse.getContentAsStream(),fos);
} catch (IOException e)
{
throw new RuntimeException(e);
}
System.out.println("downloaded");
latch.countDown();
}
});
anchor.click();
latch.await();//wait downloading to finish
webClient.close();
那么为什么您的 Test2 不工作?因为响应的Content-Type对应的下载文件是application/x-java-jnlp-file,所以需要改用WebStartHandler。如果响应 header 包含名为“Content-Disposition”的 header 并且其值以“attachment”开头,那么您的 Test2 可能会正常工作。
关于javascript - 如何从 HTMLUnit 中的 Javascript 链接下载文件,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/40732739/