Jobs from the CLI

Everything about jobs lives under aq job.

Why it is a group and not top-level verbs

aq run already means "push this directory to a box and run a command on it with my terminal attached", and aq logs already tails a box. Those are daily commands, and aq run mybox and aq run myjob are the same string, so there is no argument shape that could tell them apart. So jobs got a group rather than breaking two verbs people use.

Running

aq job run <pod> <version> -- <command...>
aq job run --image <ref> -- <command...>

One command, one submission: it creates the job and starts it in one call. Pick a source with <pod> <version> positionals (one of your saved pod versions, which runs on that version's own saved hardware) or --image <ref> (a container image; --gpu-model or --any-gpu is then required to say what it can run on). --on <alias> pins a version-source job to a box you already attached instead of renting one. --name sets a display name; without it one is the source's own name. See aq job run --help for the full flag set.

By default aq job run streams your command's log to your terminal until the job reaches succeeded, failed, or cancelled, and exits non-zero unless it succeeded. Pass --detach to print the job id and return immediately instead of streaming.

There is no separate create step: a job exists from the moment you run it, and it has exactly one execution. To run the same command again, use rerun - it makes a new job, not a second run of the old one.

Listing and watching

aq job ls                        # your jobs: name, status, GPU, duration, cost
aq job logs <job>                # print the log so far
aq job logs <job> -f             # keep printing as it is written
aq job logs <job> --attempt 2    # one specific attempt

A job that moved between machines has several attempts, and --attempt picks one. The default is the latest, not all of them joined together: concatenating would put the timestamps out of order in the middle with nothing marking the seam.

If we cannot reach the machine to read its log, aq says so on stderr rather than printing nothing. An empty log and an unreadable one are different, and silence would claim the first.

Pulling outputs

aq job pull <job> [dest]

Downloads the job's run's landed artifacts (its log object and every declared output) into dest (default: ./<job>-<runId>/), one file per artifact key. Re-running skips a file already downloaded at the same size, so an interrupted pull picks up where it left off.

Stopping

aq job cancel <job>

Stops the job's run. Billing stops when the machine started only for it is released.

Re-running

aq job rerun <job>

Makes a new job with the same source, command, and hardware, and starts it immediately. The new job gets its own id, its own status, and its own logs; the job you re-ran is untouched.

Removing

aq job rm <job>

Removes the job record.