python - 如何使用 Python Requests 登录这个特定的网站？

Question

我正在尝试使用以下请求登录此网站，但它不起作用

The cookie never contains 'userid'.

我应该改变什么？我需要在发帖请求中添加标题吗？

import requests

payload = {
    'ctl00$MasterMainContent$LoginCtrl$Username': 'myemail@email.com',
    'ctl00$MasterMainContent$LoginCtrl$Password': 'mypassword',
    'ctl00$MasterMainContent$LoginCtrl$cbxRememberMe' : 'on',
}

with requests.Session() as s:
    login_page = s.get('http://www.bentekenergy.com/')
    response = s.post('http://benport.bentekenergy.com/Login.aspx', data=payload)
    if 'userid' in response.cookies:
       print("connected")
    else:
       print("not connected")

编辑 1(以下评论): 我不确定在请求 header 中放什么，下面是我尝试过的，但没有成功。

request_headers = {
    'Accept':'image/webp,image/*,*/*;q=0.8',
    'Accept-Encoding':'gzip, deflate, sdch, br',
    'Accept-Language':'en-US,en;q=0.8',
    'Connection':'keep-alive',
    'Cookie':'ACOOKIE=C8ctADJmMTc1YTRhLTBiMTEtNGViOC1iZjE0LTM5NTNkZDVmMDc1YwAAAAABAAAASGYBALlflFnvWZRZAQAAAABLAAC5X5RZ71mUWQAAAAA-',
    'Host':'statse.webtrendslive.com',
    'Referer':'https://benport.bentekenergy.com/Login.aspx',
    'User-Agent':'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36'
}

编辑 2(在 stovfl 回答之后):

我现在使用以下有效载荷，用表单中的值填充每个属性，并用用户名、密码和 rememberMe 完成它。 我还尝试在请求中使用以下 header 。 仍未连接

payload = {
    '__VIEWSTATE' : '',
    '__VIEWSTATEGENERATOR' : '',
    '__PREVIOUSPAGE' : '',
    '__EVENTVALIDATION' : '',
    'isAuthenticated' : 'False',
    'ctl00$hfAccessKey' : '',
    'ctl00$hfVisibility' : '',
    'ctl00$hfDateTime' : '',
    'ctl00$hfHash' : '',
    'ctl00$hfAnnouncementsUrl' : '',
    'ctl00$MasterMainContent$LoginCtrl$Username' : '',
    'ctl00$MasterMainContent$LoginCtrl$Password' : '',
    'ctl00$MasterMainContent$LoginCtrl$cbxRememberMe' : '',
}

request_headers = {
        'Accept':'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8',
        'Accept-Encoding':'gzip, deflate, br',
        'Accept-Language':'en-US,en;q=0.8',
        'Cache-Control':'max-age=0',
        'Connection':'keep-alive',
        'Content-Length':'7522',
        'Content-Type':'application/x-www-form-urlencoded',
        'Cookie':'',
        'Host':'benport.bentekenergy.com',
        'Origin':'https://benport.bentekenergy.com',
        'Referer':'https://benport.bentekenergy.com/Login.aspx',
        'Upgrade-Insecure-Requests':'1',
        'User-Agent':'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36'
}

with requests.Session() as s:
response = s.get('http://benport.bentekenergy.com/Login.aspx')
soup = BeautifulSoup(response.text, "html.parser")
if soup.find("input", {"name" : "ctl00$MasterMainContent$LoginCtrl$Username"}):
    print("not connected")
    soup = BeautifulSoup(response.text, "lxml")
    for element in soup.select("input"): 
       if element.get("name") in payload:
           payload[element.get("name")] = element.get("value")

    payload['ctl00$MasterMainContent$LoginCtrl$Username'] = 'myemail@email.com'
    payload['ctl00$MasterMainContent$LoginCtrl$Password'] = 'mypassword'
    payload['ctl00$MasterMainContent$LoginCtrl$cbxRememberMe'] = 'on'

    response = s.post('http://benport.bentekenergy.com/Login.aspx', data=payload, headers=request_headers)

    print (s.cookies)
    soup = BeautifulSoup(response.text, "html.parser")
    if soup.find("input", {"name" : "ctl00$MasterMainContent$LoginCtrl$Username"}):
            print("not connected")
    else:
            print("connected")

s.cookies 包含:

<RequestsCookieJar[<Cookie BenportState=q1k2r2eqftltjm55igy5mg55 for .bentekenergy.com/>, <Cookie RememberMe=True for .bentekenergy.com/>]>

编辑 3(回答!):

我加了

'__EVENTTARGET' : ''

在负载中并用值'ctl00$MasterMainContent$LoginCtrl$btnSignIn'填充它

现在我连接上了! 注意: header 不是必需的，只是有效载荷

Answer 1

Comment: ... found that there is a parameter '__EVENTTARGET' that was not in the payload. It needed to contain 'ctl00$MasterMainContent$LoginCtrl$btnSignIn'. Now I am connected!

是的，忽略了Submit Button , 有一个 Javascript :

href="javascript:WebForm_DoPostBackWithOptions(new WebForm_PostBackOptions("ctl00$headerLoginCtrl$btnSignIn",

相关:SO Answer How To see POST Data

---

Comment: ... based on your answer (Edit 2). Still not connected

1. 您正在使用 http而不是 https
   将自动重定向到 https .

2. <RequestsCookieJar已经改变，所以有一些进步。
   我仍然不确定您的认证支票:if soup.find("input", {"name"... .
   您检查页面内容了吗？
   有错误信息吗？

3. 不要使用 BeautifulSoup(...您的以下请求应使用 Session s重用分配的 Cookie .
   例如。 response = s.get('<url to some resticted page>

4. 尝试 request_headers 仅 'User-Agent'

---

Analysis <form>:
Login URL: https://benport.bentekenergy.com/Login.aspx
Form: action: /Login.aspx, method: post

如果value 不 empty表示:来自登录页面的预设值。

 1:input type:hidden   value:/wEPDwUKLT... id:__VIEWSTATE 
 2:input type:hidden   value:0BA31D5D      id:__VIEWSTATEGENERATOR 
 3:input type:hidden   value:2gILTn0H1S... id:__PREVIOUSPAGE 
 4:input type:hidden   value:/wEWDAKIr6... id:__EVENTVALIDATION 
 5:input type:hidden   value:False         id:isAuthenticated 
 6:input type:hidden   value:nu66O9eqvE    id:ctl00_hfAccessKey 
 7:input type:hidden   value:public        id:ctl00_hfVisibility 
 8:input type:hidden   value:08%2F16%2F... id:ctl00_hfDateTime 
 9:input type:hidden   value:3AB353573D... id:ctl00_hfHash 
10:input type:hidden   value://announce... id:ctl00_hfAnnouncementsUrl 
11:input type:text     value:empty         id:ctl00_MasterMainContent_LoginCtrl_Username 
12:input type:password value:empty         id:ctl00_MasterMainContent_LoginCtrl_Password 
13:input type:checkbox value:empty         id:ctl00_MasterMainContent_LoginCtrl_cbxRememberMe