Skip to content

Don't Kill RM or Proxy on user job failure #146

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 24 additions & 2 deletions xpk.py
Original file line number Diff line number Diff line change
Expand Up @@ -247,7 +247,7 @@ class CapacityType(enum.Enum):
failurePolicy:
maxRestarts: {args.max_restarts}
successPolicy:
operator: "All"
operator: "Any"
targetReplicatedJobs:
- "main"
replicatedJobs:
Expand All @@ -260,7 +260,7 @@ class CapacityType(enum.Enum):
labels:
xpk.google.com/workload: {args.workload}
spec:
backoffLimit: 4
backoffLimit: 0
completions: {system.vms_per_slice}
parallelism: {system.vms_per_slice}
template:
Expand Down Expand Up @@ -300,6 +300,7 @@ class CapacityType(enum.Enum):
metadata:
labels:
xpk.google.com/workload: {args.workload}
app: pathways-rm
spec:
backoffLimit: 0
completions: 1
Expand Down Expand Up @@ -349,6 +350,7 @@ class CapacityType(enum.Enum):
metadata:
labels:
xpk.google.com/workload: {args.workload}
app: pathways-proxy
spec:
backoffLimit: 0
completions: 1
Expand Down Expand Up @@ -391,6 +393,26 @@ class CapacityType(enum.Enum):
path: /tmp
type: DirectoryOrCreate
name: shared-tmp
---
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
name: rm-pdb
spec:
minAvailable: 1
selector:
matchLabels:
app: pathways-rm
---
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
name: proxy-pdb
spec:
minAvailable: 1
selector:
matchLabels:
app: pathways-proxy
"""

script_dir_dockerfile = """FROM {base_docker_image}
Expand Down
Loading