Long-Running Job Queue

The job queue lets agents and API clients enqueue work that takes longer than a single HTTP request. Jobs run in the background, track progress from 0–100, and store their result in PostgreSQL when complete. The queue uses FOR UPDATE SKIP LOCKED for safe concurrent worker claiming — multiple workers can run without double-processing.

Job lifecycle

API

Enqueuing a job

# Enqueue an agent task as a background job
curl -X POST "http://localhost:3000/jobs" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "type": "agent_run",
    "payload": {
      "agentId": "research-agent",
      "uid": "user_01j...",
      "message": "Summarize all Slack threads from last week"
    },
    "maxRetries": 2,
    "timeoutSeconds": 600
  }'

# Response 202 Accepted:
# { "id": "job_01j...", "status": "pending" }

Polling for results

Poll GET /jobs/:id until status is completed or failed. The progress field (0–100) is updated by the worker during execution and can be used to show a progress bar.

# Poll for job status
curl "http://localhost:3000/jobs/job_01j..." \
  -H "Authorization: Bearer $TOKEN"

# While running:
# { "job": { "status": "running", "progress": 42, "workerId": "wkr_...", ... } }

# When done:
# { "job": { "status": "completed", "progress": 100, "result": { ... } } }

Listing jobs

# List failed jobs
curl "http://localhost:3000/jobs?status=failed&limit=10" \
  -H "Authorization: Bearer $TOKEN"

Cancelling a job

Only pending jobs can be cancelled. Once a worker has claimed the job (status=running), cancellation is not supported — the worker will run to completion or timeout.

# Cancel a pending job (only works while status=pending)
curl -X DELETE "http://localhost:3000/jobs/job_01j..." \
  -H "Authorization: Bearer $TOKEN"

Retries

When a job fails, it is retried automatically up to maxRetries times. Each retry resets the status to pending and increments retryCount. After all retries are exhausted, the status becomes failed with the last error message stored in the error field.

Background worker

The background worker starts and stops with the gateway lifecycle. It polls the queue every few seconds using FOR UPDATE SKIP LOCKED to safely claim jobs in a multi-worker deployment.

The built-in job type is agent_run. When the worker claims an agent_run job, it calls the same agent loop as the POST /chat route — passing agentId, uid, and message from the job payload. The full response is stored in the job's result field when the turn completes. This means any agent that works via the chat API also works asynchronously via the job queue with no extra configuration.

Additional job types can be registered in src/jobs/worker.ts by adding a handler to the handlers map.