-
Notifications
You must be signed in to change notification settings - Fork 6
Conversation
Otherwise, we assume a task has timeout when in reality it could've been externally interrupted.
I'll cherry pick changes to the new CLI after this change is merged. |
cmd/agent/main.go
Outdated
@@ -209,6 +209,7 @@ func main() { | |||
sig := <-signalChannel | |||
|
|||
if sig == os.Interrupt || sig == syscall.SIGTERM { | |||
log.Printf("execution canceled by '%v' signal!", sig) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We already know this because we have the log.Printf("Captured %v...", sig)
below.
@@ -52,6 +52,7 @@ type StepResult struct { | |||
|
|||
var ( | |||
ErrStepExit = errors.New("executor step requested to terminate execution") | |||
ErrTimedOut = errors.New("timed out") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we not change the error message to avoid changing tests and potentially breaking things for people expecting "Timed out" string in the instruction's logs?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Linting will fail I think. Errors should be lowercase
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's add a //nolint:
then.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can do something like this though 967a6a5
@@ -215,7 +216,7 @@ func (executor *Executor) RunBuild(ctx context.Context) { | |||
// Normal timeout-bounded context | |||
timeout := time.Duration(response.TimeoutInSeconds) * time.Second | |||
|
|||
timeoutCtx, timeoutCtxCancel := context.WithTimeout(ctx, timeout) | |||
timeoutCtx, timeoutCtxCancel := context.WithTimeoutCause(ctx, timeout, ErrTimedOut) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Perhaps we can skip using the WithTimeoutCause()
if we're never using the context.Cause()
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Didn't read docs that deep and thought ctx.Err()
will have it set. See 349540a
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry, but I'm still not sure why we need it if we already have context.DeadlineExceeded
that serves the same purpose.
@@ -61,15 +60,16 @@ func ShellCommandsAndWait( | |||
|
|||
select { | |||
case <-ctx.Done(): | |||
handler([]byte("\nTimed out!")) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It feels like a proper approach here would be branch on ctx.Err()
:
diff --git a/internal/executor/shell.go b/internal/executor/shell.go
index 4545003..9e419cd 100644
--- a/internal/executor/shell.go
+++ b/internal/executor/shell.go
@@ -61,7 +61,15 @@ func ShellCommandsAndWait(
select {
case <-ctx.Done():
- handler([]byte("\nTimed out!"))
+ var message string
+
+ if errors.Is(ctx.Err(), context.DeadlineExceeded) {
+ message = "Timed out!"
+ } else {
+ message = fmt.Sprintf("Context terminated unexpectedly: %v", ctx.Err())
+ }
+
+ handler([]byte("\n" + message))
processdumper.Dump()
@@ -69,7 +77,7 @@ func ShellCommandsAndWait(
handler([]byte(fmt.Sprintf("\nFailed to kill a timed out shell session: %s", err)))
}
- return cmd, TimeOutError
+ return cmd, ctx.Err()
case <-done:
var forcePiperClosure bool
This will keep the old behavior in place and won't munge the error message.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
IMO the cause approach is better. It's clear what was the cause. DeadlineExceeded
might come from somehting else in the future, for example.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Plus what kind of error we need to return? ctx.Err()
or TimeOutError
?
What do you think about also printing the value of This should provide some additional clues. |
Don't think the issue is with |
@@ -61,15 +60,16 @@ func ShellCommandsAndWait( | |||
|
|||
select { | |||
case <-ctx.Done(): | |||
handler([]byte("\nTimed out!")) | |||
errorMessage := fmt.Sprintf("%v!", context.Cause(ctx)) | |||
handler([]byte("\n" + strings.ToUpper(errorMessage[:1]) + errorMessage[1:])) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You can probably use cases.Title()
here to skip the gory details of string manipulation.
From cirruslabs/cirrus-ci-agent#363 by creating a `diff` and replacing `iternal/` with `internal/agent/`
From cirruslabs/cirrus-ci-agent#363 by creating a `diff` and replacing `iternal/` with `internal/agent/`
Otherwise, we assume a task has timeout when in reality it could've been externally interrupted.