我正在尝试通过 AWS CLI 命令运行一组 Athena SQL 语句,我需要了解以下内容
SQL 查询集在一个文件中,我需要通过 python 脚本为每个查询顺序迭代以下 CLI 命令中的 --query-string
我可以使用 FOR 循环从文件中获取 sql 命令,但我不知道如何在 AWS CLI 命令中对其进行参数化。
AWS CLI 命令
aws athena start-query-execution \
--query-string **"select count(*) from tablename;"** \
--query-execution-context Database=test45 \
--result-configuration OutputLocation=s3://test/ \
--output text
Python代码
import boto3
client = boto3.client('athena')
fd = open('Athenacli.sql', 'r')
sqlFile = fd.read()
fd.close()
sqlCommands = sqlFile.split(';')
for command in sqlCommands:
print command
response = client.start_query_execution(
QueryString=command,
QueryExecutionContext={
'Database': 'Test45'
},
ResultConfiguration={
'OutputLocation': 's3://test/'
}
)
print response
Athenacli.sql 的文件内容
select count(*) CNT from Test45.table1;
错误信息
Traceback (most recent call last):
pydev_imports.execfile(file, globals, locals) # execute the script
File "C:/Python27/mypycode/CLIFrame.py", line 18, in <module>
'OutputLocation': 's3://test/'
File "C:\Users\AppData\Roaming\Python\Python27\site-packages\botocore\client.py", line 253, in _api_call
return self._make_api_call(operation_name, kwargs)
File "C:\Users\AppData\Roaming\Python\Python27\site-packages\botocore\client.py", line 531, in _make_api_call
api_params, operation_model, context=request_context)
File "C:\Users\AppData\Roaming\Python\Python27\site-packages\botocore\client.py", line 586, in _convert_to_request_dict
api_params, operation_model)
File "C:\Users\AppData\Roaming\Python\Python27\site-packages\botocore\validate.py", line 291, in serialize_to_request
raise ParamValidationError(report=report.generate_report())
botocore.exceptions.ParamValidationError: Parameter validation failed:
Invalid length for parameter QueryString, value: 0, valid range: 1-inf
Process finished with exit code 1
最佳答案
#!/bin/bash
while IFS='' read -r sql || [[ -n "$sql" ]]; do
aws athena start-query-execution \
--query-string "$sql" \
--query-execution-context Database=test45 \
--result-configuration OutputLocation=s3://test/ \
--output text
done < "$1"
您可以调用您的文件
run_athena_query.sh
:$ chmod +x run_athena_query.sh
$ ./run_athena_query.sh <file_with_query>
关于sql - AWS CLI Athena,Python - 以编程方式传递查询,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/44296510/