我正在尝试在 AWS EC2 实例上安装 Airflow 。网络上的各种来源似乎很好地记录了该过程,但是,在“pip install” Airflow 之后我遇到了问题;执行命令 'airflow initdb' 时出现以下错误:
[2019-09-25 13:22:02,329] {__init__.py:51} INFO - Using executor SequentialExecutor
Traceback (most recent call last):
File "/home/cloud-user/.local/bin/airflow", line 22, in <module>
from airflow.bin.cli import CLIFactory
File "/home/cloud-user/.local/lib/python2.7/site-packages/airflow/bin/cli.py", line 68, in <module>
from airflow.www_rbac.app import cached_app as cached_app_rbac
File "/home/cloud-user/.local/lib/python2.7/site-packages/airflow/www_rbac/app.py", line 26, in <module>
from flask_appbuilder import AppBuilder, SQLA
File "/home/cloud-user/.local/lib/python2.7/site-packages/flask_appbuilder/__init__.py", line 5, in <module>
from .base import AppBuilder
File "/home/cloud-user/.local/lib/python2.7/site-packages/flask_appbuilder/base.py", line 5, in <module>
from .api.manager import OpenApiManager
File "/home/cloud-user/.local/lib/python2.7/site-packages/flask_appbuilder/api/__init__.py", line 11, in <module>
from marshmallow_sqlalchemy.fields import Related, RelatedList
File "/home/cloud-user/.local/lib/python2.7/site-packages/marshmallow_sqlalchemy/__init__.py", line 1, in <module>
from .schema import TableSchemaOpts, ModelSchemaOpts, TableSchema, ModelSchema
File "/home/cloud-user/.local/lib/python2.7/site-packages/marshmallow_sqlalchemy/schema.py", line 101
class TableSchema(ma.Schema, metaclass=TableSchemaMeta):
^
我的安装步骤如下:
sudo apt update -y
sudo apt install python-pip -y
pip install apache-airflow
export AIRFLOW_HOME=~/.local/bin
export PATH=$PATH:$AIRFLOW_HOME
cd $AIRFLOW_HOME
mkdir dags/
airflow initdb
airflow scheduler &
airflow webserver &
我也尝试按照一些在线说明安装 Postgres:
sudo apt-get install postgresql postgresql-contrib -y
或者:pip install apache-airflow[postgres]
关于可能需要的任何配置文件更改没有太多说明。
虽然我停留在 initdb 这一步,但最终目标是将数据库分离到它自己的 RDS 实例中。
我还阅读了一些文档,其中指出可以在虚拟环境中运行 Airflow ,但这是必需的吗?
最佳答案
那是因为 marshmallow-sqlalchemy
版本升级了,0.19.0 破坏了与 py2 和 py3.5 的兼容性
运行以下
pip install marshmallow-sqlalchemy==0.17.1
它应该可以工作。
关于 Airflow Jira 的相关问题:https://issues.apache.org/jira/browse/AIRFLOW-5430
关于amazon-ec2 - 为什么 Apache airflow 使用命令 : 'airflow initdb' ? 失败,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/58103825/