Concepts
Jobs
A job is the persistent record Mobius writes when an automation step, an agent tool call, or an LLM generation needs to be executed outside the runtime. Workers claim jobs and report results. Most of the time you'll watch runs, not jobs. Jobs become interesting when a step is suspended longer than it should be: was the job claimed? Is the worker heartbeating? Did it report a duplicate?
When you'd look at a job
Three diagnostic patterns cover most cases:
- A step has been suspended too long. Check that a worker is
online (
mobius worker-sessions list). If no worker is registered for the action or queue the step needs, that's your answer. - A step keeps retrying. The job's
claim_attemptis going up without completing. Either the worker is crashing inside the handler or the action is throwing a non-retryable error that the runtime is treating as retryable. - Two workers seem to be doing the same thing. They aren't.
Mobius fences with a per-job lease token; only one worker's
terminal report is accepted. If you see this in logs it's worth
confirming with the run's event stream, where each step has exactly
one
action.resultevent.
Status
| Status | Meaning |
|---|---|
pending | Waiting for a worker to claim it. |
claimed | Held by a worker session. |
completed | Worker reported success. |
failed | Worker reported a failure, or attempts ran out. |
cancelled | Parent run was cancelled, or an operator cancelled the job. |
A reaper sweeps jobs whose deadlines pass: pending jobs go back to the queue, claimed jobs whose lease lapses get reclaimed, and runaway jobs fail when the execution deadline passes.
Kinds
Two shapes of work flow through the same job pipeline:
| Kind | Carries | Streaming? |
|---|---|---|
action_execution | An action name and a spec object. | No. |
llm_generation | A provider, model, and generation spec. | Yes (token deltas). |
The agent loop runs in Mobius Cloud; tool calls and LLM generations are sent out as jobs to whichever worker advertises support for the relevant action or provider/model pair.
Inspecting jobs
The CLI doesn't expose raw jobs as a top-level command, because almost everything you need is on the run:
mobius runs get run_01...
mobius runs list-steps run_01...
mobius runs stream run_01...The step record carries the job_id, and the run event stream shows
action.called, action.result, action.failed, and (for generations)
streaming token events. That's usually enough.
Retries
The action defines the retry policy. Each attempt bumps claim_attempt,
resets the lease, and pushes a fresh job to the queue. If the worker
reports a terminal failure that the action classifies as non-retryable
(validation errors, bad input), the runtime skips retry and marks the
step failed.
Idempotency
If you care about not running the same external side effect twice, set
idempotency at the action layer: handle re-claims with a stable
external ID, dedupe inside the action implementation, or annotate the
action as idempotent so retries are explicitly allowed. The job-level
lease fencing protects the wire protocol, not your downstream system.