python - 为什么我可以使用 python mechanize 登录亚马逊网站，但不能使用 requests 或 urllib2

Question

我可以使用从 here 中找到的以下 python 代码登录 amazon.com:

import mechanize 

br = mechanize.Browser()  
br.set_handle_robots(False)  
br.addheaders = [("User-agent", "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.13) Gecko/20101206 Ubuntu/10.10 (maverick) Firefox/3.6.13")]  

sign_in = br.open('https://www.amazon.com/gp/sign-in.html')  

br.select_form(name="sign-in")  
br["email"] = 'test@test.com' 
br["password"] = 'test4test'
logged_in = br.submit() 

orders_html = br.open("https://www.amazon.com/gp/css/history/orders/view.html?orderFilter=year-%s&startAtIndex=1000" % 2013)

但以下两段使用请求模块和 urllib2 不起作用。

import requests
import sys

username = "test@test.com"
password = "test4test"

login_data = ({
        'email' : fb_username,
        'password' : fb_password,
        'flex_password': 'true'})

url = 'https://www.amazon.com/gp/sign-in.html'

agent ={'User-agent', 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_4) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/21.0.1180.57 Safari/537.1'}

session = requests.session(config={'verbose': sys.stderr}, headers = agent)

r = session.get('http://www.amazon.com')

r1 = session.post(url, data=login_data, cookies=r.cookies)

r2 = session.post("https://www.amazon.com/gp/css/history/orders/view.html?orderFilter=year-2013&startAtIndex=1000", cookies = r1.cookies)

#

import urllib2
import urllib
import cookielib

amazon_username = "test@test.com"
amazon_password = "test4test"
url = 'https://www.amazon.com/gp/sign-in.html'

cookie = cookielib.CookieJar()
login_data = urllib.urlencode({'email' : amazon_username, 'password' : amazon_password,})

opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cookie))
opener.addheaders = [('User-agent', 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_4) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/21.0.1180.57 Safari/537.1')]

opener.open('www.amazon.com')

response = opener.open(url, login_data)

response = opener.open("https://www.amazon.com/gp/css/history/orders/view.html?orderFilter=year-%s&startAtIndex=1000" % 2013, login_data)

我在发布亚马逊登录表单时做错了什么？这是我第一次发布表格。感谢您的帮助。

我更喜欢使用 urllib2 或请求，因为我的所有其他代码都使用这两个模块。

此外，任何人都可以评论一下 mechanize、requests 和 urllib2 之间的速度性能，以及 mechanize 相对于其他两者的其他优势吗？

~~~~~~~~~~~新~~~~~~~~~~~~ 按照 C.C. 的指示，我现在可以使用 urllib2 登录。但是当我尝试对请求做同样的事情时，它仍然不起作用。谁能给我一个线索？

import requests
import sys

fb_username = "test@test.com"
fb_password = "xxxx"

login_data = ({
            'email' : fb_username,
            'password' : fb_password,
            'action': 'sign-in'})

url = 'https://www.amazon.com/gp/sign-in.html'

agent ={'User-agent', 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_4) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/21.0.1180.57 Safari/537.1'}

session = requests.session(config={'verbose': sys.stderr}, headers = agent)

r = session.get(url)

r1 = session.post('https://www.amazon.com/gp/flex/sign-in/select.html', data=login_data, cookies=r.cookies)

b = r1.text

Answer 1

关于您的 urllib2 方法，您遗漏了两件事。

首先，如果您查看 sign-in.html 的源代码，它表明

<form name="sign-in" id="sign-in" action="/gp/flex/sign-in/select.html" method="POST">

意味着表单应该提交给 select.html。

其次，除了邮箱和密码，您还需要选择您是否是现有用户:

<input id="newCust" type="radio" name="action" value="new-user"...>
...
<input id="returningCust" type="radio" name="action" value="sign-in"...>

它应该看起来像这样:

import cookielib
import urllib
import urllib2

amazon_username = ...
amazon_password = ...

login_data = urllib.urlencode({'action': 'sign-in',
                               'email': amazon_username,
                               'password': amazon_password,
                               })

cookie = cookielib.CookieJar()    
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cookie))
opener.addheaders = [('User-agent', ...)]

response = opener.open('https://www.amazon.com/gp/sign-in.html')
print(response.getcode())

response = opener.open('https://www.amazon.com/gp/flex/sign-in/select.html', login_data)
print(response.getcode())

response = opener.open("https://www.amazon.com/") # it should show that you are logged in
print(response.getcode())