Skip to content

Exit codes

submit and supervise

These two run the foreground supervisor, and their exit code reports how supervision ended.

Code Meaning
0 The job Succeeded and nothing is outstanding.
2 The job Failed (including give-up), and its escalation was delivered. The failure was printed.
3 The job was Cancelled.
4 The supervisor gave up on infrastructure: for longer than supervise.giveup (10 minutes by default) the record could not be read, or the loop made no progress, for example because the scheduler stayed unreachable. The job is left Unattended.
5 An outcome was reached, but something was still outstanding when the post-outcome wait ended.
128 + n Ended by signal n after draining. SIGINT gives 130, SIGTERM 143, SIGHUP 129.
1 dunscaith itself failed.

Codes 2, 3 and 0 are the outcome. 4, 5 and the signal codes say supervision was cut short.

Exit 2 also means a wiring failure at startup

If dunscaith cannot even construct its command tree it exits 2 before running anything. That is a defect in the build, not a failed job. It prints a message to stderr, unlike a Failed job.

Which code wins

When more than one applies, the highest in this list is reported:

  1. A signal (128 + n)
  2. 4, the supervisor gave up
  3. 5, work outstanding
  4. The outcome's code (0, 2 or 3)

A Failed job with an object still stopping exits 5, and the outcome is printed above the exit.

The signal code is not an exit call. After the drain the process re-raises the signal and dies by it, which the shell reports as 128 + n. That holds during the post-outcome wait too: a signal there ends the wait and gives 128 + n, never 5. A drain that itself fails exits with that error's code instead.

Exit 5

Once the job has an outcome, the foreground supervisor keeps working for supervise.wait (two minutes by default) on whatever is left. It then prints what remains and exits 5. Outstanding means one or more of:

  • clean-up: objects of the job are still stopping on the scheduler;
  • escalation delivery: an escalation is undelivered, for example because the escalation command keeps failing. This applies whatever the outcome, so a Failed job whose escalation never reached your command exits 5, not 2;
  • settlement: an unmatched attempt or a foreign object is waiting for you to run abandon.

Run dunscaith reconcile, which carries it on from the current host's records, or dunscaith supervise <job>.

Other reasons for exit 1

  • job <id> is already supervised: another supervisor holds the job's lock.
  • record settled without an outcome: the record is settled but names no outcome.

All other commands

0 for success, 1 for any error, including a usage error such as an unknown flag.

A command that was forwarded to another host exits with the remote command's own status. If the remote command failed with status 1, you see forwarded command failed on <host> with status 1 and exit 1; any other status is passed through.

watch and status are ended by a signal like any other command. watch exits 0 when the job reaches an outcome, and 1 when its record is not there.

In a script

dunscaith submit job.yaml
case $? in
  0) echo succeeded ;;
  2) echo failed ;;
  3) echo cancelled ;;
  4) echo "gave up; job is unattended: run dunscaith supervise" ;;
  5) echo "outcome reached with work outstanding: run dunscaith reconcile" ;;
  1) echo "dunscaith error" ;;
  *) echo "ended by signal $(( $? - 128 ))" ;;
esac