python - 尝试使用执行 ID 检索对象时 AWS Athena python 连接 S3 错误

Question

当尝试读取 CSV 形式的 S3 对象时，响应是 AWS Athena 查询的执行 ID:

def run_query(query, database, s3_output):
    client = boto3.client('athena')
    response = client.start_query_execution(
        QueryString=query,
        QueryExecutionContext={
            'Database': database
            },
        ResultConfiguration={
            'OutputLocation': s3_output,
            }
        )
    print('Execution ID: ' + response['QueryExecutionId'])
    return response

response = run_query(query1, db, s3_output)
result = get_exec_status(response)
print(result)

s3_resource = boto3.resource('s3')
s3_client = boto3.client('s3')

def read_s3(path):
    path = path.replace("s3://", "")
    bucket, key = path.split('/', 1)
    s3_client.copy_object(Bucket=bucket, CopySource=path, Key=".csv")
    s3_client.delete_object(Bucket=bucket, Key=key)

read_s3("s3://"+ response + ".csv")

Error:
 File "athena_connect.py", line 67, in <module>
    read_s3("s3://"+ response + ".csv")
  File "athena_connect.py", line 64, in read_s3
    s3_client.copy_object(Bucket=bucket, CopySource=path, Key=".csv")
botocore.errorfactory.NoSuchKey: An error occurred (NoSuchKey) when calling the CopyObject operation: The specified key does not exist.

但是，当 响应='somekey' 这段代码工作正常。可能出了什么问题？

Answer 1

错误是:

The specified key does not exist

这意味着程序正在尝试读取 Amazon S3 中不存在的对象。

这一行:

read_s3("s3://"+ response + ".csv")

期望response是一个包含文件 key 的字符串。

但是，response 之前被用作字典:

print('Execution ID: ' + response['QueryExecutionId'])

因此，最好使用:

read_s3("s3://"+ response['QueryExecutionId'] + ".csv")