python - 如何获取li标签内的链接？

Question

我有这个代码:

import urllib
from bs4 import BeautifulSoup
url = "http://download.cnet.com/windows/"
pageHtml = urllib.urlopen(url)
soup = BeautifulSoup(pageHtml)
for a in soup.select("div.catFlyout a[href]"):
    print "http://download.cnet.com"+a["href"]

但是这段代码没有给出正确的输出。 正确的输出应该是这样的:

http://download.cnet.com/windows/security-software/
http://download.cnet.com/windows/browsers/
http://download.cnet.com/windows/business-software/
..
..
http://download.cnet.com/windows/video-software/

Answer 1

列表中有一些相对和绝对链接，仅当链接以 http 开头时才添加基本 url:

for a in soup.select("div.catFlyout a[href]"):
    if not a["href"].startswith("http"):
        print "http://download.cnet.com"+a["href"]
    else:
        print a["href"]

或者，使用 urlparse 检查链接是否是绝对的(取自 here ):

import urllib
import urlparse
from bs4 import BeautifulSoup

def is_absolute(url):
    return bool(urlparse.urlparse(url).scheme)

url = "http://download.cnet.com/windows/"
pageHtml = urllib.urlopen(url)
soup = BeautifulSoup(pageHtml)
for a in soup.select("div.catFlyout a[href]"):
    if not is_absolute(a['href']):
        print "http://download.cnet.com"+a["href"]
    else:
        print a["href"]