背景:我正在我的 Python/AppEngine 项目中获取数据并创建一个 .tsv 文件,以便我可以使用 d3.js 创建图表。现在我正在为每个页面加载编写 CSV;我想将该文件存储在 Google Cloud Storage 中一次,然后从那里读取它。
每次加载页面时我当前如何写入文件!:
def get(self): ## this gets called when loading myfile.tsv from d3.js
datalist = MyEntity.all()
self.response.headers['Content-Type'] = 'text/csv'
writer = csv.writer(self.response.out, delimiter='\t')
writer.writerow(['field1', 'field2'])
for eachco in datalist:
writer.writerow([eachco.variable1, eachco.variable2])
虽然效率低下,但效果很好。
使用this Google Cloud Storage documentation ,我一直在努力让这样的事情发挥作用:
def get(self):
filename = '/bucket/myfile.tsv'
datalist = MyEntity.all()
bucket_name = os.environ.get('BUCKET_NAME', app_identity.get_default_gcs_bucket_name())
write_retry_params = gcs.RetryParams(backoff_factor=1.1)
writer = csv.writer(self.response.out, delimiter='\t')
gcs_file = gcs.open(filename, 'w', content_type='text/csv', retry_params=write_retry_params)
gcs_file.write(writer.writerow(['field1', 'field2']))
for eachco in datalist:
gcs_file.write(writer.writerow([eachco.variable1, eachco.variable2]))
gcs_file.close()
但我得到:
TypeError: Expected str but got <type 'NoneType'>.
我认为 csv.writer 的输出将是一个字符串,所以我不确定为什么会收到 TypeError。
所以我可以想到两种情况:
- 我的代码中出现了一些问题,将 tsv 写入 云储存。不过,迭代并将 TSV/CSV 文件写入 Cloud Storage 应该很简单,对吧?
- 我以完全错误的方式解决了这个问题 完全,甚至应该使用 BlobStore 或 db.TextProperty() 存储此 .tsv 数据。 (文件没那么大;绝对 远低于 1MB)
如果有任何帮助,我将不胜感激!
编辑 - 完整回溯
Traceback (most recent call last):
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.1/webapp2.py", line 1530, in __call__
rv = self.router.dispatch(request, response)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.1/webapp2.py", line 1278, in default_dispatcher
return route.handler_adapter(request, response)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.1/webapp2.py", line 1102, in __call__
return handler.dispatch()
File "/mydirectory/myapp/handlers.py", line 21, in dispatch
webapp2.RequestHandler.dispatch(self)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.1/webapp2.py", line 572, in dispatch
return self.handle_exception(e, self.app.debug)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.1/webapp2.py", line 570, in dispatch
return method(*args, **kwargs)
File "/mydirectory/myapp/thisapp.py", line 384, in get
gcs_file.write(writer.writerow(['field1', 'field2']))
File "lib/cloudstorage/storage_api.py", line 754, in write
raise TypeError('Expected str but got %s.' % type(data))
TypeError: Expected str but got <type 'NoneType'>.
最佳答案
您仍在尝试根据响应创建编写器:
writer = csv.writer(self.response.out, delimiter='\t')
您需要写入 GCS 文件。像这样的事情:
datalist = MyEntity.all()
bucket_name = os.environ.get('BUCKET_NAME', app_identity.get_default_gcs_bucket_name())
filename = os.path.join(bucket_name, 'myfile.tsv')
write_retry_params = gcs.RetryParams(backoff_factor=1.1)
gcs_file = gcs.open(filename, 'w', content_type='text/csv', retry_params=write_retry_params)
writer = csv.writer(gcs_file, delimiter='\t')
writer.writerow(['field1', 'field2'])
for eachco in datalist:
writer.writerow([eachco.variable1, eachco.variable2])
gcs_file.close()
注释:
- 未实际测试
- 我还调整了文件名以使用
bucket_name
- 如果您在
get()
请求中执行此操作,您可能需要检查该文件是否已存在,如果存在,请使用它,否则您仍会在每次请求时生成它。或者,您可以将此代码移至任务或.tsv
上传处理程序中。
关于python - 编写 CSV 以存储在 Google Cloud Storage 中,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/39524581/