python - BeautifulSoup:获取文本，创建字典

Question

我正在收集有关中央银行研究出版物的信息，到目前为止，对于美联储，我有以下 Python 代码:

START_URL = 'https://ideas.repec.org/s/fip/fedgfe.html'
page = requests.get(START_URL)
soup = BeautifulSoup(page.text, 'html.parser')
for paper in soup.findAll("li",class_="list-group-item downfree"):
    print(paper.text)

这为许多出版物中的第一个生成以下内容:

2018-070 Reliably Computing Nonlinear Dynamic Stochastic Model Solutions: An Algorithm with Error Formulasby Gary S. Anderson

我现在想把这个转换成一个Python字典，最终会包含大量论文:

Papers = {
  'Date': 2018 - 070,
  'Title': 'Reliably Computing Nonlinear Dynamic Stochastic Model Solutions: An Algorithm with Error Formulas',
  'Author/s': 'Gary S. Anderson'
  }

Answer 1

我在提取所有后代后得到了很好的结果，并且只挑选了那些 NavigableStrings 的后代。 .确保从 bs4 导入 NavigableString。我还使用了 numpy 列表理解，但您也可以使用 for 循环。

START_URL = 'https://ideas.repec.org/s/fip/fedgfe.html'
page = requests.get(START_URL)
soup = BeautifulSoup(page.text, 'html.parser')

papers = []
for paper in soup.findAll("li",class_="list-group-item downfree"):
    info = [desc.strip() for desc in paper.descendants if type(desc) == NavigableString]
    papers.append({'Date': info[0], 'Title': info[1], 'Author': info[3]})

print(papers[1])

{'Date': '2018-069',
 'Title': 'The Effect of Common Ownership on Profits : Evidence From the U.S. Banking Industry',
 'Author': 'Jacob P. Gramlich & Serafin J. Grundl'}