java - HTTP Post 到 API 返回 403 FORBIDDEN

Question

我必须将 XML 文件上传到 API。这是 API 由我从 API 颁发者那里获得的签名证书保护。

现在，我有两个用例。首先，我必须从此 API 下载一些文件。这与以下代码完美配合:

final Proxy proxy = new Proxy(Proxy.Type.HTTP, new InetSocketAddress(ip, port));

final URL url = new URL(linkToFile);
final HttpsURLConnection conn = (HttpsURLConnection) url.openConnection(proxy);
conn.setSSLSocketFactory(this.http.createSSLContext().getSocketFactory());
try (
            InputStream inputStream = zipUrlConn.getInputStream();
            ZipInputStream stream = new ZipInputStream(inputStream);) {

    // Do stuff with ZipInputStream here
}

createSSLContext() 方法如下所示:

public SSLContext createSSLContext() throws NoSuchAlgorithmException, CertificateException, IOException, KeyStoreException,
        UnrecoverableKeyException, KeyManagementException {

    final KeyStore clientStore = KeyStore.getInstance("PKCS12");
    clientStore.load(new FileInputStream(this.certificateResource.getFile()), this.p12PW.toCharArray());

    final KeyManagerFactory keyManagerFactory = KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm());
    keyManagerFactory.init(clientStore, this.p12PW.toCharArray());
    final KeyManager[] keyManagers = keyManagerFactory.getKeyManagers();

    final KeyStore trustStore = KeyStore.getInstance("JKS");
    trustStore.load(new FileInputStream(this.trustStoreResource.getFile()), this.trustStorePW.toCharArray());

    final TrustManagerFactory trustManagerFactory = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
    trustManagerFactory.init(trustStore);
    final TrustManager[] trustManagers = trustManagerFactory.getTrustManagers();

    final SSLContext sslContext = SSLContext.getInstance("TLS");
    sslContext.init(keyManagers, trustManagers, new SecureRandom());

    return sslContext;
}

我遵循了从发行者那里得到的指南，其中展示了如何使用 cUrl 命令执行此操作:

curl --cert Certificate-<id>.pem[:pem_password] https://api.url.com/

所以我基本上是在尝试用 java 重建这个命令，它是有效的。

现在是无法正常工作的部分，即文件上传。再一次，我得到了一个我必须重建的 cUrl 命令:

curl --cert Certificate-<id>.pem[:pem_password] -F upload=@<Path_to_file>\DS<PartnerId>_<Timestamp>.XML https://api.url.com/in/upload.php

我尝试了几种方法来实现这一点:

1. “普通”Java

首先，我尝试使用标准的 HttpsURLConnection 如下:

final Proxy proxy = new Proxy(Proxy.Type.HTTP, new InetSocketAddress("192.168.1.25", 3128));
final HttpsURLConnection connection = (HttpsURLConnection) new URL(ARGE_UPLOAD_URL).openConnection(proxy);

connection.setSSLSocketFactory(this.http.createSSLContext().getSocketFactory());
connection.setDoOutput(true);
connection.setRequestProperty("Accept-Charset", "UTF-8");
connection.setRequestProperty("Content-Type", "application/x-www-form-urlencoded;charset=UTF-8");

try (OutputStream outputStream = connection.getOutputStream()) {
    outputStream.write(Files.readAllBytes(new File("src/main/resources/XML/example.xml").toPath()));
}

final InputStream result = connection.getInputStream();

但这总是导致 java.io.IOException:服务器返回 HTTP 响应代码:403 URL:https://api.url.com/in/upload.php，即使我我正在使用相同的配置，通过它我可以从 API 下载。

1. Apache HttpClient

我发现一些资源声称 HttpClient 更容易配置和使用，所以我试了一下:

final CloseableHttpClient httpClient = HttpClientBuilder
            .create()
            .setSSLContext(this.http.createSSLContext())
            .setProxy(new HttpHost(InetAddress.getByName(ip), port))
            .build();
final HttpEntity requestEntity = MultipartEntityBuilder
            .create()
            .addBinaryBody("upload=@example.xml", new File("src/main/resources/XML/example.xml")) // Hardcoded for testing
            .build();

final HttpPost post = new HttpPost(https://api.url.com/in/upload.php);
post.setEntity(requestEntity);

try (CloseableHttpResponse response = httpClient.execute(post)) {
    this.logger.info(response.getStatusLine().toString());
    EntityUtils.consume(response.getEntity());
}

导致 HTTP/1.1 403 FORBIDDEN

1. HttpClient(FileEntity 而不是 MultipartEntity)

作为最后一件事，我尝试了一个 FileEntity:

final HttpPost httpPost = new HttpPost(https://api.url.com/in/upload.php);
httpPost.addHeader("content-type", "application/x-www-form-urlencoded;charset=utf-8");

final FileEntity fileEntity = new FileEntity(new File("src/main/resources/XML/example.xml"));
httpPost.setEntity(fileEntity);

System.out.println("executing request " + httpPost.getRequestLine() + httpPost.getConfig());
try (CloseableHttpResponse response = httpClient.execute(httpPost)) {
    final HttpEntity responseEntity = response.getEntity();

    System.out.println("Status: " + response.getStatusLine());
    if (responseEntity != null) {
        System.out.println("Entity: " + EntityUtils.toString(responseEntity));
    }
}

导致 Status: HTTP/1.1 403 FORBIDDEN

我只是不明白，尽管使用完全相同的配置，我如何能够从 API 下载，但不能上传到它。

如果您需要更多信息，我很乐意提供。

编辑

根据 oli 的建议，我使用Fiddler来抓取HTTPS请求。这是方法 1(普通 Java)的结果:

POST https://hrbaxml.arbeitsagentur.de/in/upload.php HTTP/1.1
Accept-Charset: utf-8
Accept: */*
Content-Type: application/x-www-form-urlencoded;charset=utf-8
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36
Host: hrbaxml.arbeitsagentur.de
Connection: keep-alive
Content-Length: 6738

这是通过 Google Chrome 手动上传的结果:

Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Encoding: gzip, deflate, br
Accept-Language: de-DE,de;q=0.9,en-US;q=0.8,en;q=0.7
Cache-Control: max-age=0
Connection: keep-alive
Content-Length: 6948
Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryxZxIJ5h19MEFbZQs
Cookie: cookie_consent=accepted
Host: hrbaxml.arbeitsagentur.de
Origin: https://hrbaxml.arbeitsagentur.de
Referer: https://hrbaxml.arbeitsagentur.de/in/
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36

编辑 2

补充一下，这是使用方法 2(带有 MultipartEntity 的 HttpClient)的结果:

POST https://hrbaxml.arbeitsagentur.de/in/upload.php HTTP/1.1
Content-Length: 7025
Content-Type: multipart/form-data; boundary=1sZvrqcGe-FuQ3r-_fFgt2SJtZ5_yo7Pfvq_
Host: hrbaxml.arbeitsagentur.de
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.5.5 (Java/1.8.0_161)
Accept-Encoding: gzip,deflate

--1sZvrqcGe-FuQ3r-_fFgt2SJtZ5_yo7Pfvq_
Content-Disposition: form-data; name="DSV000306700_2018-08-23_09-00-00.xml";    filename="DSV000306700_2018-08-23_09-00-00.xml"
Content-Type: application/octet-stream
Content-Transfer-Encoding: binary

编辑 3

我尝试从 Chrome 请求中复制所有 HTTP header ，因此我从 Java 发出的请求如下所示:

POST https://hrbaxml.arbeitsagentur.de/in/upload.php HTTP/1.1
Accept: Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Encoding: gzip, deflate, br
Accept-Language: de-DE,de;q=0.9,en-US;q=0.8,en;q=0.7
Cache-Control: max-age=0
Content-Type: multipart/form-data; boundary=1535095530678
Cookie: cookie_consent=accepted
Referer: https://hrbaxml.arbeitsagentur.de/in/
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36
Host: hrbaxml.arbeitsagentur.de
Connection: keep-alive
Content-Length: 6944

--1535095530678
Content-Disposition: form-data; name="uploadFile"; filename="DSV000306700_2018-08-23_09-00-00.xml"
Content-Type: application/xml
Content-Transfer-Encoding: binary

.. xml data ..

--1535095530678--

但仍然没有成功。还有其他可能的解决方案吗？也许这不是上传的问题而是其他原因？

Answer 1

我会捕获您使用 Java 应用程序发送到服务器的 HTTP 请求(例如使用 Wireshark)，并将其与您从浏览器发送的 HTTP 请求进行比较(您可以使用内置浏览器轻松捕获它工具，请尝试按 F12)。

我 100% 肯定您会看到一些差异，这对我来说总是有效。

编辑:

还有一个可能的问题。请尝试添加

connection.addRequestProperty("User-Agent", "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:25.0) Gecko/20100101 Firefox/25.0");

或您的浏览器在您的第一个实现中发送的同一个。还要确保你的 SSL 证书和加密算法没有问题(你使用默认的算法，你的情况就是它)。此外(如果没有帮助)您还可以检查协商的握手 key 的 key 长度。