java - Jsoup HTTP 获取 URL 时出错。进行后请愿时 Status=403

Question

我一直在寻找这个问题，据说这个问题可以使用用户代理解决，但事实并非如此。
我想做的是从请愿书中获取cookie，这是代码

note: i'm try to do the petition to https webpage

/*obtiene cookies de la peticion*/
        Connection.Response res = Jsoup.connect(liga).header("Content-Type","text/html;charset=UTF-8")
                .cookie("TALanguage", "ALL")
                .data("mode", "filterReviews")
                .data("filterRating", "")
                .data("filterSegment", "")
                .data("filterSeasons", "")
                .data("filterLang", "ALL")
                .referrer(liga)         
                .header("X-Requested-With", "XMLHttpRequest")
                .header("X-Puid",xpuid)
                .data("returnTo",returnTo)
                .userAgent("Mozilla/5.0 (Windows; U; WindowsNT 5.1; en-US; rv1.8.1.6) Gecko/20070725 Firefox/2.0.0.6")                           
                .method(Method.POST)
                .execute();

        doc = res.parse();


        Map<String, String> cookies = res.cookies();

程序在 .execute(); 行失败，日志中出现以下错误:

org.jsoup.HttpStatusException: HTTP error fetching URL. Status=403, URL=https://somepage.html

    at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:459)
    at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:434)
    at org.jsoup.helper.HttpConnection.execute(HttpConnection.java:181)
    at mx.oeste.crawler.htmlunit.obtenerComentarios(htmlunit.java:82)
    at mx.oeste.crawler.htmlunit.main(htmlunit.java:40)

Answer 1

尝试将内容类型 header 设置为“application/x-www-form-urlencoded”，如下所示:

Connection.Response res = Jsoup.connect(liga)
                               .header("Content-Type","application/x-www-form-urlencoded")
                               .cookie("TALanguage", "ALL")
                               .data("mode", "filterReviews")
                               .data("filterRating", "")
                               .data("filterSegment", "")
                               .data("filterSeasons", "")
                               .data("filterLang", "ALL")
                               .referrer(liga)         
                               .header("X-Requested-With", "XMLHttpRequest")
                               .header("X-Puid",xpuid)
                               .data("returnTo",returnTo)
                               .userAgent("Mozilla/5.0 (Windows; U; WindowsNT 5.1; en-US; rv1.8.1.6) Gecko/20070725 Firefox/2.0.0.6")                           
                               .method(Method.POST)
                               .execute();

如果不起作用，请尝试监视您最喜欢的浏览器，同时从请愿书中手动获取 cookie。您可以使用开发者工具来监视您的浏览器。