airflow UI with celery interaction not as expected

Home / Uncategorized / airflow UI with celery interaction not as expected

Question:
There’s two unexpected behavior when interact with airflow UI, does anybody have some similar experience and have some insight upon them?

1. why marking a running task as success will restart the task?

Try to mark a running task as success and it worked to termninate the currently task with the icon turn deep green, but later the task icon turn yellow and wait for restart again. Normally the meaning of success means no need to restart again.

airflow webserver UI job status

2. Trigger a job in web UI does not work even with celery?

I try to trigger a job in web UI, the reponse message shows it work and will be start at any moment, but in fact not. I check the celery worker and no message is received.

`Sent TaskInstance: mars_quality_daily_v2.quality_comment 2017-08-07 04:00:00 [up_for_retry] to the message queue, it should start any moment now.

run airflow task from web ui

To achieve my initial purpose, I need to clear the task state, and the task will restart immediately without my click. but it’s totally out of control now.

airflow version: 1.7.1.3

python version: 2.7.13

Maybe the question is too specific for this website but I find no other place to solve the puzzles. The airflow project has a chatting room instead of github issues, but the chatting room is full of people asking without any answer.


Answer:

Read more

Leave a Reply

Your email address will not be published. Required fields are marked *