我正在尝试编写一个 Python 脚本来访问 Amazon Redshift,以在 Redshift 中创建一个表并将数据从 S3 复制到 Redshift 表。
我的代码是:
import psycopg2
import os
#import pandas as pd
import requests
requests.packages.urllib3.disable_warnings()
redshift_endpoint = os.getenv("END-point")
redshift_user = os.getenv("user")
redshift_pass = os.getenv("PASSWORD")
port = 5439
dbname = 'DBNAME'
conn = psycopg2.connect(
host="",
user='',
port=5439,
password='',
dbname='')
cur = conn.cursor()
aws_key = os.getenv("access_key") # needed to access S3 Sample Data
aws_secret = os.getenv("secret_key")
#aws_iam_role= os.getenv('iam_role') #tried using this too
base_copy_string= """copy %s from 's3://mypath/%s'.csv
credentials 'aws_access_key_id= %s aws_access_secrect_key= %s'
delimiter '%s';""" # the base COPY string that we'll be using
#easily generate each table that we'll need to COPY data from
tables = ["employee"]
data_files = ["test"]
delimiters = [","]
#the generated COPY statements we'll be using to load data;
copy_statements = []
for tab, f, delim in zip(tables, data_files, delimiters):
copy_statements.append(base_copy_string % (tab, f, aws_key, aws_secret, delim)%)
#create Table
cur.execute(""" create table employee(empname varchar(30),empno integer,phoneno integer,email varchar(30))""")
for copy_statement in copy_statements: # execute each COPY statement
cur.execute(copy_statement)
conn.commit()
for table in tables + ["employee"]:
cur.execute("select count(*) from %s;" % (table,))
print(cur.fetchone())
conn.commit() # make sure data went through and commit our statements permanently.
当我运行此命令时,我在 cur.execute(copy_statement) 处收到错误
**Error:** error: Invalid credentials. Must be of the format: credentials 'aws_iam_role=...' or 'aws_access_key_id=...;aws_secre
t_access_key=...[;token=...]'
code: 8001
context:
query: 582
location: aws_credentials_parser.cpp:114
process: padbmaster [pid=18692]
我的代码有问题吗?或者是 AWS access_key 问题?
我什至尝试使用iam_role,但出现错误:
IAM role cannot assume role even in Redshift
我通过附加 S3FullAccess 策略拥有托管 IAM 角色权限。
最佳答案
您的脚本中有一些错误。
1) 更改base_copy_string如下:
base_copy_string= """copy %s from 's3://mypath/%s.csv' credentials 'aws_access_key_id=%s;aws_secret_access_key=%s' delimiter '%s';""" # the base COPY string that we'll be using
必须在凭据中添加 ;
以及单引号的其他格式问题。它是 aws_secret_access_key
,而不是 aws_access_secrect_key
。
我建议您使用 iam-roles 而不是凭据。 http://docs.aws.amazon.com/redshift/latest/dg/loading-data-access-permissions.html
2) 修改copy_statements.append如下(删除最后多余的%
):
copy_statements.append(base_copy_string % (tab, f, aws_key, aws_secret, delim))
更正这些问题并重试。
关于python - 从Python访问Redshift时出现"Invalid credentials"错误,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/46616058/