python - 从 ClinicalTrials.gov 抓取数据

Question

我正在开发一个小型 Python 函数来从 clinicalTrials.gov 中抓取数据。 。我希望从每个研究记录中抓取该研究针对的条件。例如，对于 this学习记录我想要以下内容:

conditions = ['Rhinoconjunctivitis', 'Rhinitis', 'Conjunctivitis'. 'Allergy']

但是，每一个研究记录中，都有不同数量的条件。我编写了以下获取数据的脚本:

page = requests.get('https://clinicaltrials.gov/ct2/show/study/NCT00550550')
soup = BeautifulSoup(page.text, 'html.parser')
studyDesign = soup.find_all(headers='studyInfoColData')
condition = soup.find(attrs={'class':'data_table'}).find_all('span')
for each in condition:
    print(each.text.encode('utf-8').strip())

像这样:

b'Condition or disease'
b'Intervention/treatment'
b'Phase'
b'Rhinoconjunctivitis'
b'Rhinitis'
b'Conjunctivitis'
b'Allergy'
b'Drug: Placebo'
b'Biological: SCH 697243'
b'Drug: Loratadine Syrup 1 mg/mL Rescue Treatment'
b'Drug: Loratadine 10 mg Rescue Treatment'
b'Drug: Olopatadine 0.1% Rescue Treatment'
b'Drug: Mometasone furoate 50 mcg Rescue Treatment'
b'Drug: Albuterol 108 mcg Rescue Treatment'
b'Drug: Fluticasone 44 mcg Rescue Treatment'
b'Drug: Prednisone 5 mg Rescue Treatment'
b'Phase 3'

我现在如何才能只获得病情而没有干预/治疗信息？

Answer 1

您可以仅将第一个table与data_table类一起使用，并在td中提取span元素:

import requests
from bs4 import BeautifulSoup

page = requests.get('https://clinicaltrials.gov/ct2/show/study/NCT00550550')
soup = BeautifulSoup(page.text, 'html.parser')
studyDesign = soup.find("table", {"class" : "data_table"}).find('td')
conditions = [ t.text.strip() for t in studyDesign.find_all('span') ]
print(conditions)

给出:

[u'Rhinoconjunctivitis', u'Rhinitis', u'Conjunctivitis', u'Allergy']