-
Notifications
You must be signed in to change notification settings - Fork 14.8k
New issue
Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? # to your account
Fetch intermediate log async GKEStartPod #39348
Conversation
9dd6061
to
e594417
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. Can you add the test case with last_log_time and last_log_time?
Any updates on this? |
I'll try address the pending review by end of this week |
Bump on this 🙏 |
@pankajastro Any ETA on this? Thanks |
786ba83
to
3c60132
Compare
CI is green now. Hopefully, we will merge soon. In the meantime, could you please test it once? |
* Fetch intermediate log in async GKEStartPod This PR introduces a parameter that enables the retrieval of intermediate logs for the GKEStartPod asynchronous operator. Add param last_log_time and logging_interval in GKEStartPodTrigger serialize Add optional param last_log_time in method invoke_defer_method Example DAG: start_pod = GKEStartPodOperator( task_id="start_pod", project_id=PROJECT_ID, location=LOCATION, cluster_name=GKE_CLUSTER_NAME, do_xcom_push=True, namespace=GKE_NAMESPACE, image="ubuntu:jammy", cmds=["sh", "-c", "timeout 300 bash -c 'while true; do echo \"meow\"; sleep 30; done'"], name="test-sleep", in_cluster=False, on_finish_action="delete_pod", deferrable=True, get_logs=True, logging_interval=5, gcp_conn_id=GCP_CONN_ID )
* Fetch intermediate log in async GKEStartPod This PR introduces a parameter that enables the retrieval of intermediate logs for the GKEStartPod asynchronous operator. Add param last_log_time and logging_interval in GKEStartPodTrigger serialize Add optional param last_log_time in method invoke_defer_method Example DAG: start_pod = GKEStartPodOperator( task_id="start_pod", project_id=PROJECT_ID, location=LOCATION, cluster_name=GKE_CLUSTER_NAME, do_xcom_push=True, namespace=GKE_NAMESPACE, image="ubuntu:jammy", cmds=["sh", "-c", "timeout 300 bash -c 'while true; do echo \"meow\"; sleep 30; done'"], name="test-sleep", in_cluster=False, on_finish_action="delete_pod", deferrable=True, get_logs=True, logging_interval=5, gcp_conn_id=GCP_CONN_ID )
This PR introduces a parameter that enables the retrieval of intermediate logs for the GKEStartPod asynchronous operator.
last_log_time
andlogging_interval
in GKEStartPodTrigger serializelast_log_time
in methodinvoke_defer_method
Example DAG:
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rst
or{issue_number}.significant.rst
, in newsfragments.