python - 使用 urllib2 从受基本身份验证保护的 Jenkins 服务器获取 URL

Question

我正在尝试从 Jekins 服务器获取 URL。直到最近，我才能够使用此页面 (HOWTO Fetch Internet Resources Using urllib2) 中描述的模式创建一个密码管理器，该密码管理器使用用户名和密码正确响应 BasicAuth 挑战。一切都很好，直到 Jenkins 团队 changed their security model ，并且该代码不再有效。

# DOES NOT WORK!
import urllib2
password_mgr = urllib2.HTTPPasswordMgrWithDefaultRealm()
top_level_url = "http://localhost:8080"

password_mgr.add_password(None, top_level_url, 'sal', 'foobar')
handler = urllib2.HTTPBasicAuthHandler(password_mgr)
opener = urllib2.build_opener(handler)

a_url = 'http://localhost:8080/job/foo/4/api/python'
print opener.open(a_url).read()

堆栈跟踪:

Traceback (most recent call last):
  File "/home/sal/workspace/jenkinsapi/src/examples/password.py", line 11, in <module>
    print opener.open(a_url).read()
  File "/usr/lib/python2.7/urllib2.py", line 410, in open
    response = meth(req, response)
  File "/usr/lib/python2.7/urllib2.py", line 523, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python2.7/urllib2.py", line 448, in error
    return self._call_chain(*args)
  File "/usr/lib/python2.7/urllib2.py", line 382, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 531, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 403: Forbidden
[Finished in 0.0s with exit code 1]

问题似乎是 Jenkins 返回的不是预期的 401 代码，而是 403，urllib2 将其解释为对话结束。它从不实际发送密码。在 github 上冲浪后发现另一个开发人员的解决方案有效......

# WORKS... SORTA
def auth_headers(username, password):
   return 'Basic ' + base64.encodestring('%s:%s' % (username, password))[:-1]

auth = auth_headers('sal', 'foobar')
top_level_url = "http://localhost:8080"
a_url = 'http://localhost:8080/job/foo/4/api/python'
req = urllib2.Request(a_url)
req.add_header('Authorization', auth)
print urllib2.urlopen(req).read()

但这似乎并不令人满意。它不会费心检查域是否与用户名和密码相关...无论如何它只是发送我的登录详细信息!

任何人都可以建议一种使原始脚本起作用的方法吗？我想以可以登录 Jenkins 的方式使用 urllib2 密码管理器。

Answer 1

也请参阅此要点:https://gist.github.com/dnozay/194d816aa6517dc67ca1

当您需要访问需要认证的页面时，Jenkins 不会返回401 - retry HTTP 错误码；相反，它返回 403 - forbidden。在维基中，https://wiki.jenkins-ci.org/display/JENKINS/Authenticating+scripted+clients ，它表明使用命令行工具 wget 您需要使用 wget --auth-no-challenge 这正是因为该行为。

当您收到 403 - forbidden 时重试基本身份验证:

假设您定义了:

jenkins_url = "https://jenkins.example.com"
username = "johndoe@example.com"
api_token = "my-api-token"

您可以继承 urllib2.HTTPBasicAuthHandler 来处理 403 HTTP 响应。

import urllib2

class HTTPBasic403AuthHandler(urllib2.HTTPBasicAuthHandler):
    # retry with basic auth when facing a 403 forbidden
    def http_error_403(self, req, fp, code, msg, headers):
        host = req.get_host()
        realm = None
        return self.retry_http_basic_auth(host, req, realm)

然后是使用该处理程序的问题，例如您可以安装它，以便它适用于所有 urllib2.urlopen 调用:

def install_auth_opener():
    '''install the authentication handler.

    This handles non-standard behavior where the server responds with
    403 forbidden, instead of 401 retry. Which means it does not give you the
    chance to provide your credentials.'''
    auth_handler = HTTPBasic403AuthHandler()
    auth_handler.add_password(
        realm=None,
        uri=jenkins_url,
        user=username,
        passwd=api_token)
    opener = urllib2.build_opener(auth_handler)
    # install it for all urllib2.urlopen calls
    urllib2.install_opener(opener)

这是一个简单的测试，看看它是否工作正常。

if __name__ == "__main__":
    # test
    install_auth_opener()
    page = "%s/me/api/python" % jenkins_url
    try:
        result = urllib2.urlopen(page)
        assert result.code == 200
        print "ok"
    except urllib2.HTTPError, err:
        assert err.code != 401, 'BAD CREDENTIALS!'
        raise err

使用先发制人的身份验证。

这个答案中有一个很好的例子:https://stackoverflow.com/a/8513913/1733117 . 当您收到 403 forbidden 时，您无需重试，而是在 url 匹配时发送 Authorization header 。

class PreemptiveBasicAuthHandler(urllib2.HTTPBasicAuthHandler):
    '''Preemptive basic auth.

    Instead of waiting for a 403 to then retry with the credentials,
    send the credentials if the url is handled by the password manager.
    Note: please use realm=None when calling add_password.'''
    def http_request(self, req):
        url = req.get_full_url()
        realm = None
        # this is very similar to the code from retry_http_basic_auth()
        # but returns a request object.
        user, pw = self.passwd.find_user_password(realm, url)
        if pw:
            raw = "%s:%s" % (user, pw)
            auth = 'Basic %s' % base64.b64encode(raw).strip()
            req.add_unredirected_header(self.auth_header, auth)
        return req

    https_request = http_request