我知道我在从 Unicode 转换时遇到问题,但我不确定问题出在哪里。
我正在从 HTML 文件目录中提取有关最近欧洲之旅的数据。一些位置名称具有非 ASCII 字符(例如 é、ô、ü)。我正在使用正则表达式从文件的字符串表示中获取数据。
如果我打印我找到的位置,它们会打印字符,因此编码必须正确:
Le Pré-Saint-Gervais, France
Hôtel-de-Ville, France
我使用 SQLAlchemy 将数据存储在 SQLite 表中:
Base = declarative_base()
class Point(Base):
__tablename__ = 'points'
id = Column(Integer, primary_key=True)
pdate = Column(Date)
ptime = Column(Time)
location = Column(Unicode(32))
weather = Column(String(16))
high = Column(Float)
low = Column(Float)
lat = Column(String(16))
lon = Column(String(16))
image = Column(String(64))
caption = Column(String(64))
def __init__(self, filename, pdate, ptime, location, weather, high, low, lat, lon, image, caption):
self.filename = filename
self.pdate = pdate
self.ptime = ptime
self.location = location
self.weather = weather
self.high = high
self.low = low
self.lat = lat
self.lon = lon
self.image = image
self.caption = caption
def __repr__(self):
return "<Point('%s','%s','%s')>" % (self.filename, self.pdate, self.ptime)
engine = create_engine('sqlite:///:memory:', echo=False)
Base.metadata.create_all(engine)
Session = sessionmaker(bind = engine)
session = Session()
我遍历文件并将每个文件的数据插入数据库:
for filename in filelist:
# open the file and extract the information using regex such as:
location_re = re.compile("<h2>(.*)</h2>",re.M)
# extract other data
newpoint = Point(filename, pdate, ptime, location, weather, high, low, lat, lon, image, caption)
session.add(newpoint)
session.commit()
我在每次插入时都看到以下警告:
/usr/lib/python2.5/site-packages/SQLAlchemy-0.5.4p2-py2.5.egg/sqlalchemy/engine/default.py:230: SAWarning: Unicode type received non-unicode bind param value 'Spitalfields, United Kingdom'
param.append(processors[key](compiled_params[key]))
当我尝试对表格执行任何操作时,例如:
session.query(Point).all()
我得到:
Traceback (most recent call last):
File "./extract_trips.py", line 131, in <module>
session.query(Point).all()
File "/usr/lib/python2.5/site-packages/SQLAlchemy-0.5.4p2-py2.5.egg/sqlalchemy/orm/query.py", line 1193, in all
return list(self)
File "/usr/lib/python2.5/site-packages/SQLAlchemy-0.5.4p2-py2.5.egg/sqlalchemy/orm/query.py", line 1341, in instances
fetch = cursor.fetchall()
File "/usr/lib/python2.5/site-packages/SQLAlchemy-0.5.4p2-py2.5.egg/sqlalchemy/engine/base.py", line 1642, in fetchall
self.connection._handle_dbapi_exception(e, None, None, self.cursor, self.context)
File "/usr/lib/python2.5/site-packages/SQLAlchemy-0.5.4p2-py2.5.egg/sqlalchemy/engine/base.py", line 931, in _handle_dbapi_exception
raise exc.DBAPIError.instance(statement, parameters, e, connection_invalidated=is_disconnect)
sqlalchemy.exc.OperationalError: (OperationalError) Could not decode to UTF-8 column 'points_location' with text 'Le Pré-Saint-Gervais, France' None None
我希望能够正确地存储并返回原始字符完整的位置名称。任何帮助将不胜感激。
最佳答案
我发现这篇文章在某种程度上帮助解释了我的麻烦:
http://www.amk.ca/python/howto/unicode#reading-and-writing-unicode-data
通过使用“编解码器”模块然后按如下方式更改我的程序,我能够获得所需的结果:
打开文件时:
infile = codecs.open(filename, 'r', encoding='iso-8859-1')
打印位置时:
print location.encode('ISO-8859-1')
我现在可以查询和操作表中的数据,而不会出现之前的错误。我只需要在输出文本时指定编码即可。
(我仍然不完全理解这是如何工作的,所以我想是时候了解更多关于 Python 的 unicode 处理的信息了...)
关于python - SQLAlchemy 的 Unicode 问题,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/966352/