Python + BigQuery + ResponseNotReady()

Question

我正在使用“bigquery_service = build('bigquery', 'v2', http=http)”，它在我的笔记本电脑中正确执行，但是当我在服务器上执行此查询时，出现以下错误。

File "/usr/lib64/python2.6/threading.py", line 532, in __bootstrap_inner
    self.run()
  File "/usr/lib64/python2.6/threading.py", line 484, in run
    self.__target(*self.__args, **self.__kwargs)
  File "QueryAPI.py", line 117, in localMain
    exportDataToGCS(canonicalDate);
  File "QueryAPI.py", line 177, in exportDataToGCS
    bigquery_service = build('bigquery', 'v2', http=http)
  File "/homeBigQuery/src/oauth2client/util.py", line 132, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/homeBigQuery/src/googleapiclient/discovery.py", line 198, in build
    resp, content = http.request(requested_url)
  File "/homeBigQuery/src/oauth2client/util.py", line 132, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/homeBigQuery/src/oauth2client/client.py", line 538, in new_request
    redirections, connection_type)
  File "/homeBigQuery/src/httplib2/__init__.py", line 1570, in request
    (response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)
  File "/homeBigQuery/src/httplib2/__init__.py", line 1317, in _request
    (response, content) = self._conn_request(conn, request_uri, method, body, headers)
  File "/homeBigQuery/src/httplib2/__init__.py", line 1286, in _conn_request
    response = conn.getresponse()
  File "/usr/lib64/python2.6/httplib.py", line 980, in getresponse
    raise ResponseNotReady()

Answer 1

你可以尝试this 3rd party library for BigQuery 。它是 Google BigQuery API 的一个薄包装。

登录 BigQuery:

import bigquery as bq

with open('my_key.pem', 'r') as key:
    key_content = key.read()

bq_client = bq.get_client(project_id='foo',
                          service_account='bar',
                          private_key=key_content)

提交加载作业:

schema = [{'name':'foo', 'type':'STRING'}]
job = bq_client.import_data_from_uris(source_uris=['gs://foo/bar'],
                                        dataset='foo',
                                        table='bar',
                                        schema=schema,
                                        source_format=bq.client.JOB_FORMAT_CSV,
                                        write_disposition=bq.client.JOB_WRITE_TRUNCATE,
                                        field_delimiter=',',
                                        skip_leading_rows=1)


try:
    job_resource = bq_client.wait_for_job(job, timeout=60)        
except bq.errors.BigQueryTimeoutException:
    logging.critical('BigQuery loading timeout.')