我从 https://www.topgear.com/car-reviews/ferrari/laferrari 中提取了图像链接在bash中使用scrapy，有没有简单的方法可以在没有管道的情况下保存它们？
scrapy shell https://www.topgear.com/car-reviews/ferrari/laferrari
response.xpath('//div[@class="carousel__content-inner"]//img/@srcset').extract()
['https://www.topgear.com/sites/default/files/styles/fit_980x551/public/cars-car/carousel/2015/02/buyers_guide_-_laf_-_front.jpg?itok=KiD7ErMe 980w',
'https://www.topgear.com/sites/default/files/styles/fit_980x551/public/cars-car/carousel/2015/02/buyers_guide_-_laf_-_rear.jpg?itok=JMYaaJ5L 980w',
'https://www.topgear.com/sites/default/files/styles/fit_980x551/public/cars-car/carousel/2015/02/buyers_guide_-_laf_-_interior.jpg?itok=4Z0zIdH_ 980w',
'https://www.topgear.com/sites/default/files/styles/fit_980x551/public/cars-car/carousel/2015/02/buyers_guide_-_laf_-_side.jpg?itok=OKl2MOJ2 980w']
感谢您的帮助。
最佳答案
您可以使用scrapy选择器https://docs.scrapy.org/en/latest/topics/selectors.html 和请求库:
from scrapy.selector import Selector
import requests
from tqdm import tqdm
headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:50.0) Gecko/20100101 Firefox/50.0'}
response = requests.get('https://www.topgear.com/car-reviews/ferrari/laferrari', headers=headers)
links = Selector(text=response.text).xpath('//div[@class="carousel__content-inner"]//img/@srcset').getall()
for i, image_url in tqdm(enumerate(links)):
try:
response = requests.get(image_url, headers=headers)
except:
pass
else:
if response.status_code == 200:
with open('{:02}.jpg'.format(i), 'wb') as f:
f.write(response.content)
关于python-3.x - 如何用scrapy保存图片，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/59734613/