我正在尝试使用 python 中的网络爬虫来打印 facebook 推荐者的数量。例如,天空新闻的这篇文章( http://news.sky.com/story/1330046/are-putins-little-green-men-back-in-ukraine ) Facebook 上大约有 60 条推荐。我想用网络爬虫在python程序中打印这个数字。 我尝试这样做,但它没有打印任何内容:
import requests
from bs4 import BeautifulSoup
def get_single_item_data(item_url):
source_code = requests.get(item_url)
plain_text = source_code.text
soup = BeautifulSoup(plain_text)
# if you want to gather information from that page
for item_name in soup.findAll('span', {'class': 'pluginCountTextDisconnected'}):
try:
print(item_name.string)
except:
print("error")
get_single_item_data("http://news.sky.com/story/1330046/are-putins-little-green-men-back-in-ukraine")
最佳答案
Facebook 建议在 iframe 中加载。
您可以按照该页面的 iframe
src 属性,然后加载 span.pluginCountTextDisconnected 的文本:
import requests
from bs4 import BeautifulSoup
url = 'http://news.sky.com/story/1330046/are-putins-little-green-men-back-in-ukraine'
r = requests.get(url) # get the page through requests
soup = BeautifulSoup(r.text) # create a BeautifulSoup object from the page's HTML
url = soup('iframe')[0]['src'] # search for the iframe element and get its src attribute
r = requests.get('http://' + url[2:]) # get the next page from requests with the iframe URL
soup = BeautifulSoup(r.text) # create another BeautifulSoup object
print(soup.find('span', class_='pluginCountTextDisconnected').string) # get the directed information
第二个 requests.get 是这样编写的,因为 src
属性返回 //www.facebook.com/plugins/like.php?href=http%3A%2F% 2Fnews.sky.com%2Fstory%2F1330046&send=false&layout=button_count&width=120&show_faces=false&action=recommend&colorscheme=light&font=arial&height=21
。我添加了 http://
并忽略了前导 //
。
关于python - python 中的 facebook 网络爬虫,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/25675998/