airflow - Airflow DAG运行已触发,但从未执行?

标签 airflow apache-airflow

我发现自己处于以下情况:我手动触发了DAG Run(通过airflow trigger_dag datablocks_dag)运行,并且Dag Run出现在界面中,但随后会一直保持“ Running”状态,而实际上并未执行任何操作。

当我在UI中检查此DAG Run时,看到以下内容:

enter image description here

我已将start_date设置为datetime(2016, 1, 1),并且schedule_interval设置为@once。通过阅读文档,我的理解是,由于start_date <现在,将触发DAG。 @once确保它仅发生一次。

我的日志文件说:

[2017-07-11 21:32:05,359] {jobs.py:343} DagFileProcessor0 INFO - Started process (PID=21217) to work on /home/alex/Desktop/datablocks/tests/.airflow/dags/datablocks_dag.py
[2017-07-11 21:32:05,359] {jobs.py:534} DagFileProcessor0 ERROR - Cannot use more than 1 thread when using sqlite. Setting max_threads to 1
[2017-07-11 21:32:05,365] {jobs.py:1525} DagFileProcessor0 INFO - Processing file /home/alex/Desktop/datablocks/tests/.airflow/dags/datablocks_dag.py for tasks to queue
[2017-07-11 21:32:05,365] {models.py:176} DagFileProcessor0 INFO - Filling up the DagBag from /home/alex/Desktop/datablocks/tests/.airflow/dags/datablocks_dag.py
[2017-07-11 21:32:05,703] {models.py:2048} DagFileProcessor0 WARNING - schedule_interval is used for <Task(BashOperator): foo>, though it has been deprecated as a task parameter, you need to specify it as a DAG parameter instead
[2017-07-11 21:32:05,703] {models.py:2048} DagFileProcessor0 WARNING - schedule_interval is used for <Task(BashOperator): foo2>, though it has been deprecated as a task parameter, you need to specify it as a DAG parameter instead
[2017-07-11 21:32:05,704] {jobs.py:1539} DagFileProcessor0 INFO - DAG(s) dict_keys(['example_branch_dop_operator_v3', 'latest_only', 'tutorial', 'example_http_operator', 'example_python_operator', 'example_bash_operator', 'example_branch_operator', 'example_trigger_target_dag', 'example_short_circuit_operator', 'example_passing_params_via_test_command', 'test_utils', 'example_subdag_operator', 'example_subdag_operator.section-1', 'example_subdag_operator.section-2', 'example_skip_dag', 'example_xcom', 'example_trigger_controller_dag', 'latest_only_with_trigger', 'datablocks_dag']) retrieved from /home/alex/Desktop/datablocks/tests/.airflow/dags/datablocks_dag.py
[2017-07-11 21:32:07,083] {models.py:3529} DagFileProcessor0 INFO - Creating ORM DAG for datablocks_dag
[2017-07-11 21:32:07,234] {models.py:331} DagFileProcessor0 INFO - Finding 'running' jobs without a recent heartbeat
[2017-07-11 21:32:07,234] {models.py:337} DagFileProcessor0 INFO - Failing jobs without heartbeat after 2017-07-11 21:27:07.234388
[2017-07-11 21:32:07,240] {jobs.py:351} DagFileProcessor0 INFO - Processing /home/alex/Desktop/datablocks/tests/.airflow/dags/datablocks_dag.py took 1.881 seconds


是什么原因引起的?

我是否误解了start_date的工作方式?

还是日志文件中看起来令人担忧的schedule_interval WARNING行可能是问题的根源?

最佳答案

问题是dag已暂停。

在您提供的屏幕截图中,将其翻转到On,然后按此操作即可。

气流开始时,这是常见的“陷阱”。

关于airflow - Airflow DAG运行已触发,但从未执行?,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/45047059/

相关文章:

python - 使用 apache-airflow (cloud-composer) 调度从 BigQuery 到 MongoDB 的负载

airflow - 我如何在 Airflow 中使用 --conf 选项

python - Airflow 默认on_failure_callback

python - 如何配置 Apache Airflow 以使用 MS SQL Server Windows 身份验证

kubernetes - 如何在 Kubeflow 中安排作业?

airflow - 在 BigQueryOperator 中将参数添加为 template_fields

python - Airflow 设置 dag 运行注意事项

python - Airflow - ModuleNotFoundError : No module named 'kubernetes'

Airflow 任务卡在 "queued"状态并且永远不会运行

airflow - 如何在使用 Airflow 实现的工作流中等待 DAG 任务中的异步事件?