Lab

GPU job queue

Daily driver

A router on my Mac that sends AI work to GPU machines at home over a private network.

Queue

queue depth 2 Refresh

node-1critique + taskbusy

Running

1760883302-9f3a7c21task

qwen3-coder:30bpreempted

elapsed 12:41

PreemptDelete

Queued

1760883355-1d8e44b0task

qwen3-coder:30bpreempted

elapsed 12:41

PreemptDelete

node-2critiquebusy

Running

1760883310-c7a2e5f0critique

deepseek-r1:32bpreempted

elapsed 02:07

PreemptDelete

Queued

1760883371-6be0a913critique

gemma3:27bpreempted

elapsed 02:07

PreemptDelete

The queue page: one running job per node, a reorderable list behind it, and preempt and delete by job id. Mockup with sample data.

$ gpuq critique plan.md

fan-out 2 critique nodes

node-1 1760883310-4be01d97 qwen3-coder:30b done 2 findings 58s

node-2 1760883310-c7a2e5f0 deepseek-r1:32b done 3 findings 41s

$ gpuq task "add a retry budget to the export job"

1760883402-9f3a7c21 -> node-1 (first free node by rank)

preview https://router.example.invalid/jobs/1760883402-9f3a7c21

$ gpuq critique --parent 1760883402-9f3a7c21 --node node-1

error 409 a critique never runs on its parent task's node

not retried

$ curl -s https://router.example.invalid/health

{"ok":true,"queue_depth":2}

A client run: a critique fans out to two nodes, a task returns a job id and a preview link, and a refused placement fails loudly. Mockup with sample data.

How it works

A client sends a job. It tells the router on my Mac whether it is a critique or a long task.
The router picks nodes. A critique fans out to every critique node at once. A task goes to the first free node by rank.
A guard checks the placement. A critique never lands on its parent task's node, and the router refuses it with a 409.
Each node runs one job at a time. The rest wait in that node's queue, which I can reorder, preempt or clear.
The result comes back and the model unloads. The job id and preview link return to the client, and the model is released with keep_alive 0 to free VRAM.

Built with

NodeThe router on my Mac and the job client that talks to it.
OllamaServes the models on each GPU node, unloaded after every job.
TailscaleThe private network between my Mac and the GPU machines at home.
Plain HTML and JSThe queue page, with no framework: node columns, drag handles, preempt and delete.

Decisions

Fail loudly. A refused or failed job returns an error and is never retried silently, so a wrong placement shows up instead of hiding behind a retry.
One job per node, and models unload after each job. A cold start costs a little time, but VRAM is always free and a node is never half busy.
Delete and preempt exist because of a stuck queue. Once the head of the queue sat on a finished job for 49 minutes while /health looked fine, and deleting that job by id fixed it. Now /health reports queue depth, and the queue page can remove or jump any job by id.