当我使用python携带cookie发送模拟请求到https://www.amazon.com/gp/delivery/ajax/address-change.html回复 \n\n将被退回。但是当我使用 Charles 作为中间人代理时，同样的 http 消息正常响应。
同样，当你使用nodejs发送模拟请求时，也可以得到正常的响应。我在python中尝试了三个不同的网络请求库，requests、httpx和aiohttp得到了相同的结果。
对于\n\n的回复，我将问题锁定在python请求的tls握手包上。修改后urllib3.util.ssl_.DEFAULT_CIPHERS ，它仍然返回 \n\n 的响应.
和Wireshark抓包对比后发现，除了CIPHERS部分，Signature Algorithm部分也是固定的，三个请求库的Signature Algorithm部分都是一样的，用curl的方式，nodejs的方式， Charles的方式，Chrome获取TSL Client Hello包的方式。这是不一样的。
我想将 Python TSL Client Hello 包的签名算法部分模拟到 Chrome 中。在我DEBUG了python请求库的源码后，发现openssl so文件中似乎存在SSL Signature Algorithm部分控件。
这个问题困扰了我很久，希望能解决，非常感谢
import requests
from aiohttp import ClientSession
import httpx
cookies = {
'csm-hit': 'tb:s-B8ZK0QTPQCGWKHY3QDT5|1620287052879&t:1620287054928&adb:adblk_no',
'i18n-prefs': 'USD',
'lc-main': 'en_US',
'session-id': '143-0501748-3847056',
'session-id-time': '2082787201l',
'session-token': 'NxLWWkB7RnpUvmQEl7OcUzk44D9PnlSt/swrqvnSwBvry9WAPSeQt5U2hVCa7IeEEDwj+qzLHwrNhCnA+7pN8H7HELP5WYZuPjtTJ1d8jrTxLueLIQB+wh+3e+1c1vRrfYDa4FTsdm6jN2QR55zq0ybhNJt0jrXCTdlaktZ+e0tHPIjQnCsu1lidMvyOksR+',
'skin': 'noskin',
'sp-cdn': 'L5Z9:CN',
'ubid-main': '134-5202210-0613519',
}
headers = {
'Host': 'www.amazon.com',
'user-agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4094.1',
'accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9',
'accept-language': 'zh-CN,zh;q=0.9',
'anti-csrftoken-a2z': 'gBtJDelwICZ60r+pGBgwbzjAf4Wr+LTRIoyWRyMAAAAMAAAAAGC1xeJyYXcAAAAA',
'content-type': 'application/x-www-form-urlencoded;charset=utf-8',
}
data = 'locationType=LOCATION_INPUT&zipCode=90001&storeContext=generic&deviceType=web&pageType=Gateway&actionSource=glow&almBrandId=undefined'
url = 'https://www.amazon.com/gp/delivery/ajax/address-change.html'
# url = 'https://www.python-spider.com/nginx'
your_proxy_url = 'http://127.0.0.1:8888'
# your_proxy_url = ''
#
with httpx.Client(
# http2=True,
# proxies=your_proxy_url,
verify=False) as client:
# This HTTP request will be tunneled instead of forwarded.
response = client.post(url=url, headers=headers, cookies=cookies, data=data)
print(response.status_code)
print(response.text)
# cert='/Users/yangyanhui/lbs/spider/amazon/amazon_cookie_pool/charles-ssl-proxying-certificate.pem'
response = requests.post(url, headers=headers, cookies=cookies, data=data)
print(response.status_code)
print(response.text)
import aiohttp, asyncio
# asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy()) # 加上这一行
async def main(): # aiohttp必须放在异步函数中使用
async with ClientSession(cookies=cookies, headers=headers) as session:
async with session.post(url, data=data,
# proxy=your_proxy_url,
verify_ssl=False) as resp:
print(await resp.text())
print(resp.status)
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
curl -H 'Host: www.amazon.com' -H 'Cookie: csm-hit=tb:s-B8ZK0QTPQCGWKHY3QDT5|1620287052879&t:1620287054928&adb:adblk_no; i18n-prefs=USD; lc-main=en_US; session-id=143-0501748-3847056; session-id-time=2082787201l; session-token=NxLWWkB7RnpUvmQEl7OcUzk44D9PnlSt/swrqvnSwBvry9WAPSeQt5U2hVCa7IeEEDwj+qzLHwrNhCnA+7pN8H7HELP5WYZuPjtTJ1d8jrTxLueLIQB+wh+3e+1c1vRrfYDa4FTsdm6jN2QR55zq0ybhNJt0jrXCTdlaktZ+e0tHPIjQnCsu1lidMvyOksR+; skin=noskin; sp-cdn=L5Z9:CN; ubid-main=134-5202210-0613519' -H 'user-agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4094.1' -H 'accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9' -H 'accept-language: zh-CN,zh;q=0.9' -H 'anti-csrftoken-a2z: gBtJDelwICZ60r+pGBgwbzjAf4Wr+LTRIoyWRyMAAAAMAAAAAGC1xeJyYXcAAAAA' -H 'content-type: application/x-www-form-urlencoded;charset=utf-8' --data-binary "locationType=LOCATION_INPUT&zipCode=90001&storeContext=generic&deviceType=web&pageType=Gateway&actionSource=glow&almBrandId=undefined" --compressed 'https://www.amazon.com/gp/delivery/ajax/address-change.html'
最佳答案
也许考虑使用有一个包的 golang 可以修改你的 tls 指纹
关于ssl - 使用python爬虫发送亚马逊修改邮政编码接口(interface)时遇到疑似TLS指纹风控，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/68032039/