Kubernetes Pod Operator Task keeps Pod with Pending/Running state in detele_succeded_pod #59083
Description
Apache Airflow version
3.1.3
If "Other Airflow 2/3 version" selected, which one?
No response
What happened?
Hi!
We have detected that the Airflow Worker does not delete the Kubernetes Pod Operator for Pending/Running State while using the on_finish_action with DELETE_SUCCEEDED_POD.
What you think should happen instead?
I believe it should delete Pods with Pending state. Otherwise, they will consume resources. Only the failed should be Kept, so, we should change the logic to DELETE_NON_FAILED_POD.
How to reproduce
In case you spin up a KPO Pod without machine using DELETE_SUCCEEDED_POD for on_finish_action, and Airflow Worker timeout in the spin up, the Pod will be kept there waiting for a machine and if the machine become available it will start to run
Operating System
apache:airflow-3.1.3 on Kubernetes
Versions of Apache Airflow Providers
Kubernetes Provider only.
Deployment
Official Apache Airflow Helm Chart
Deployment details
No response
Anything else?
No response
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct