我有一系列类似于“Saturday, December 27th 2014”的字符串,我想丢弃“Saturday”并使用名称“141227”(年+月+日)保存文件。到目前为止,一切正常,除了我无法让 daypos 或 Yearpos 的正则表达式工作。他们都给出了相同的错误:
Traceback (most recent call last): File "scrapewaybackblog.py", line 17, in daypos = byline.find(re.compile("[A-Z][a-z]*\s")) TypeError: expected a character buffer object
什么是字符缓冲区对象?这是不是说明我的表情有问题?这是我的脚本:
for i in xrange(3, 1, -1):
page = urllib2.urlopen("http://web.archive.org/web/20090204221349/http://www.americansforprosperity.org/nationalblog?page={}".format(i))
soup = BeautifulSoup(page.read())
snippet = soup.find_all('div', attrs={'class': 'blog-box'})
for div in snippet:
byline = div.find('div', attrs={'class': 'date'}).text.encode('utf-8')
text = div.find('div', attrs={'class': 'right-box'}).text.encode('utf-8')
monthpos = byline.find(",")
daypos = byline.find(re.compile("[A-Z][a-z]*\s"))
yearpos = byline.find(re.compile("[A-Z][a-z]*\D\d*\w*\s"))
endpos = monthpos + len(byline)
month = byline[monthpos+1:daypos]
day = byline[daypos+0:yearpos]
year = byline[yearpos+2:endpos]
output_files_pathname = 'Data/' # path where output will go
new_filename = year + month + day + ".txt"
outfile = open(output_files_pathname + new_filename,'w')
outfile.write(date)
outfile.write("\n")
outfile.write(text)
outfile.close()
print "finished another url from page {}".format(i)
我还没有弄清楚如何使 12 月 = 12,但那是另一次了。请帮我找到合适的职位。
最佳答案
不要用正则表达式解析日期字符串,而是用 dateutil
解析它。 :
from dateutil.parser import parse
for div in soup.select('div.blog-box'):
byline = div.find('div', attrs={'class': 'date'}).text.encode('utf-8')
text = div.find('div', attrs={'class': 'right-box'}).text.encode('utf-8')
dt = parse(byline)
new_filename = "{dt.year}{dt.month}{dt.day}.txt".format(dt=dt)
...
或者,您可以使用datetime.strptime()
解析字符串,但您需要注意 suffixes :
byline = re.sub(r"(?<=\d)(st|nd|rd|th)", "", byline)
dt = datetime.strptime(byline, '%A, %B %d %Y')
re.sub()
此处查找 st
或 nd
或 rd
或 th
字符串after a digit并将后缀替换为空字符串。之后的日期字符串将匹配 '%A, %B %d %Y'
格式,请参阅:
一些附加说明:
- 您可以将
urlopen()
的结果直接传递给BeautifulSoup
构造函数 - 使用 CSS Selector 而不是按类名
find_all()
div.blog-box
- 要加入系统路径,请使用
os.path.join()
- 使用
with
context manager处理文件时
修复版本:
import os
import urllib2
from bs4 import BeautifulSoup
from dateutil.parser import parse
for i in xrange(3, 1, -1):
page = urllib2.urlopen("http://web.archive.org/web/20090204221349/http://www.americansforprosperity.org/nationalblog?page={}".format(i))
soup = BeautifulSoup(page)
for div in soup.select('div.blog-box'):
byline = div.find('div', attrs={'class': 'date'}).text.encode('utf-8')
text = div.find('div', attrs={'class': 'right-box'}).text.encode('utf-8')
dt = parse(byline)
new_filename = "{dt.year}{dt.month}{dt.day}.txt".format(dt=dt)
with open(os.path.join('Data', new_filename), 'w') as outfile:
outfile.write(byline)
outfile.write("\n")
outfile.write(text)
print "finished another url from page {}".format(i)
关于python - 使用 Regex 和 BeautifulSoup 解析 Python 中的字符串,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/27672665/