我已经构建了一个网络爬虫,它成功地从我正在查看的网页中提取了几乎所有我需要的东西。目标是提取与在特定 URL 中找到的所有咖啡相关联的特定图像的 URL。
我定义的完成scraping的rake任务如下:
mechanize = Mechanize.new
mechanize.get(url) do |page|
page.links_with(:href => /products/).each do |link|
coffee_page = link.click
bean = Bean.new
bean.acidity = coffee_page.css('[data-id="acidity"]').text.strip.gsub("acidity ","")
bean.elevation = coffee_page.css('[data-id="elevation"]').text.strip.gsub("elevation ","")
bean.roaster_id = "2"
bean.harvest_season = coffee_page.css('[data-id="harvest"]').text.strip.gsub("harvest ","")
bean.price = coffee_page.css('.price-wrap').text.gsub("$","")
bean.roast_profile = coffee_page.css('[data-id="roast"]').text.strip.gsub("roast ","")
bean.processing_type = coffee_page.css('[data-id="process"]').text.strip.gsub("process ","")
bean.cultivar = coffee_page.css('[data-id="cultivar"]').text.strip.gsub("cultivar ","")
bean.flavor_profiles = coffee_page.css('.price-wrap+ p').text.strip
bean.country_of_origin = coffee_page.css('#pdp-order h1').text.strip
bean.image_url = coffee_page.css('img data-featured-product-image').attr('src')
if bean.country_of_origin == "Origin Set" || bean.country_of_origin == "Gift Card (online use only)"
bean.destroy
else
ap bean
end
end
end
现在我需要的信息都在页面上,我正在寻找像下面这样找到的图像 URL,但是对于源页面上的所有单独的 coffee_pages。它需要足够通用才能提取此图片源,但仅此而已。我已经尝试了许多不同的 css 选择器,但一切都不是 nil 就是空白。
<img src="//cdn.shopify.com/s/files/1/2220/0129/products/ceremony-product-gummy-bears_480x480.jpg?v=1551455589" alt="Burundi Kiryama" data-product-featured-image style="display:none">
我所在的 coffee_page 在这里:https://shop.ceremonycoffee.com/products/burundi-kiryama
最佳答案
你需要改变
bean.image_url = coffee_page.css('img data-featured-product-image').attr('src')
到
bean.image_url = coffee_page.css('#mobile-only>img').attr('src')
如果可以,请始终使用附近的标识符来定位要访问的元素。
关于ruby-on-rails - 无法识别正确的 CSS 选择器以使用 Mechanize 进行抓取,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/55235229/