java - 无法将 cookie 从 Selenium webdriver 传递到 clojure 中的 httpclient cookie 存储

Question

我想使用 Selenium 登录网站，然后将 cookie 导出到 httpclient。

(defn convert-selemium-cookie-to-httpclient-cookie [x]
  (let [sf (SimpleDateFormat. "yyyy-MM-dd")
        fake-date (.parse sf "2018-08-06")]
    (doto
        (BasicClientCookie. (:name x) (:value x))
        (.setDomain (:domain x))
        (.setPath (:path x))
        (.setExpiryDate (if (:expiry x) (:expiry x) fake-date))
        (.setSecure (:secure? x)))))

(defn add-selenium-cookies-to-httpclient-cookie-store [x]
  (let [cs (clj-http.cookies/cookie-store)
        http-cookies (map convert-selemium-cookie-to-httpclient-cookie x)]
    (mapv (fn[x] (.addCookie cs x)) http-cookies)
    cs))

(def driver (new-driver {:browser :firefox}))
(def a (login driver ...)) ;; login
(def c (cookies driver)) ;; get the cookies
(def cs (add-selenium-cookies-to-httpclient-cookie-store c))
(println (client/get "some web site"
                     {:cookie-store cs
                      :client-params {"http.useragent"
                                      "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/534.20 (KHTML, like Gecko) Chrome/11.0.672.2 Safari/534.20"}}))

虽然我在 webdriver 中成功登录并且该站点在 session 中保留 cookie，但在将 cookie 复制到 cookie 存储后，client/get 仍然返回未登录的页面。我在中尝试过类似的事情python，它可以成功地将 cookie 从 webdriver 传输到请求(python 的 http 客户端)。不确定 java/clojure 的问题出在哪里。

Answer 1

您还没有指定依赖项的版本。我已经使用以下代码测试了您的代码:

[org.clojure/clojure "1.8.0"]
[clj-webdriver "0.7.2"]
[org.seleniumhq.selenium/selenium-server "3.0.0-beta4"]

Firefox 48.0.2
Mac OS Sierra 10.12

我的回答末尾的 REPL session 表明它有效(我使用 https://httpbin.org/ 对其进行了测试)。

要排除故障，我会检查以下内容:

• 打印驱动程序返回的 cookie 或检查浏览器控制台以查看登录页面的响应是否包含设置正确 cookie 的 header
• 打印你的 http-client cookiestore 包含的内容
• 为 http 客户端启用调试日志记录(或在您的服务器中启用调试日志记录)以查看发送到您的登录保护页面的实际请求以及该请求是否包含登录页面设置的 cookie。

在 REPL session 中，您可以看到从 selenium 返回的 cookie 包含第一个请求设置的 cookie。它们也存在于 http-client cookie 存储中，并且它们由 https://httpbin.org/cookies URL 正确返回，表明它们已通过 http-client 的请求发送。

(require '[clj-webdriver.taxi :as taxi])
(require '[clj-http.client :as client])
(require '[clj-http.cookies :as cookies])
(require '[clojure.pprint :refer [pprint]])
(import java.text.SimpleDateFormat)
(import org.apache.http.impl.cookie.BasicClientCookie)

(defn convert-selemium-cookie-to-httpclient-cookie [x]
  (let [sf (SimpleDateFormat. "yyyy-MM-dd")
        fake-date (.parse sf "2018-08-06")]
    (doto
      (BasicClientCookie. (:name x) (:value x))
      (.setDomain (:domain x))
      (.setPath (:path x))
      (.setExpiryDate (if (:expiry x) (:expiry x) fake-date))
      (.setSecure (:secure? x)))))

(defn add-selenium-cookies-to-httpclient-cookie-store [x]
  (let [cs (cookies/cookie-store)
        http-cookies (map convert-selemium-cookie-to-httpclient-cookie x)]
    (mapv (fn [x] (.addCookie cs x)) http-cookies) cs))

(def cookie-name (str "c1" (System/currentTimeMillis)))
(def cookie-value (str "v1" (System/currentTimeMillis)))

(pprint cookie-name)
;; "c11475935066134"

(pprint cookie-value)
;; "v11475935066814"

(def driver (taxi/new-driver {:browser :firefox}))

(taxi/to driver (format "https://httpbin.org/cookies/set?%s=%s" cookie-name cookie-value))

(def selenium-cookies (taxi/cookies driver))

(pprint selenium-cookies)
;; #{{:cookie
;;    #object[org.openqa.selenium.Cookie 0x4dc96ce8 "c11475935066134=v11475935066814; path=/; domain=httpbin.org"],
;;    :name "c11475935066134",
;;    :value "v11475935066814",
;;    :path "/",
;;    :expiry nil,
;;    :domain "httpbin.org",
;;    :secure? false}}

(def http-client-cookie-store (add-selenium-cookies-to-httpclient-cookie-store selenium-cookies))

(pprint http-client-cookie-store)
;; #object[org.apache.http.impl.client.BasicCookieStore 0x6dfa86f5 "[[version: 0][name: c11475935066134][value: v11475935066814][domain: httpbin.org][path: /][expiry: Mon Aug 06 00:00:00 CEST 2018]]"]

(def http-client-response
  (client/get
    "https://httpbin.org/cookies"
    {:cookie-store http-client-cookie-store}))

(pprint http-client-response)
;; {:status 200,
;;  :headers
;;  {"Server" "nginx",
;;   "Date" "Sat, 08 Oct 2016 13:58:01 GMT",
;;   "Content-Type" "application/json",
;;   "Content-Length" "64",
;;   "Connection" "close",
;;   "Access-Control-Allow-Origin" "*",
;;   "Access-Control-Allow-Credentials" "true"},
;;  :body
;;  "{\n  \"cookies\": {\n    \"c11475935066134\": \"v11475935066814\"\n  }\n}\n",
;;  :request-time 1001,
;;  :trace-redirects ["https://httpbin.org/cookies"],
;;  :orig-content-encoding nil}