Airflow 允许我们通过命令行添加连接信息 airflow connections .这有助于通过 ansible 或其他开发运营工具自动部署 Airflow 装置。
尚不清楚如何通过命令行将与谷歌云平台(服务帐户)的连接添加到 ariflow。
最佳答案
pre airflow 1.9 以下示例概述了如何使用 DAG 添加连接信息:https://gist.github.com/yu-iskw/42f9f0aa6f2ff0a2a375d43881e13b49
def add_gcp_connection(ds, **kwargs):
""""Add a airflow connection for GCP"""
new_conn = Connection(
conn_id=<CONNECTION_ID>,
conn_type='google_cloud_platform',
)
scopes = ['https://www.googleapis.com/auth/cloud-platform']
conn_extra = {
"extra__google_cloud_platform__scope": ",".join(scopes),
"extra__google_cloud_platform__project":
"<GCP_PROJECT_NAME>",
"extra__google_cloud_platform__key_path":
"<GCP_CREDENTIALS_ABSOLUTE_PATH.json>"
}
conn_extra_json = json.dumps(conn_extra)
new_conn.set_extra(conn_extra_json)
session = settings.Session()
session.add(new_conn)
session.commit()
从 Airflow 1.9 向前一 jar :
airflow connections -a \
--conn_id=<CONNECTION_ID> \
--conn_type=google_cloud_platform \
--conn_extra='{ "extra__google_cloud_platform__key_path":" '`
`'<GCP_CREDENTIALS_ABSOLUTE_PATH.json>", '`
`'"extra__google_cloud_platform__project": '`
`'"<GCP_PROJECT_NAME>", '`
`'"extra__google_cloud_platform__scope": '`
`'"https://www.googleapis.com/auth/cloud-platform"}'
关于python - 通过命令行将 GCP 凭据添加到 Airflow ，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/50040717/