Taskmaster

What Taskmaster Does

Taskmaster is a task supervisor that sits above your agents and watches their work. You define a list of tasks, set a success criteria for each one, and hit Run. Taskmaster fires the agents — either launching a new tab or taking over an existing chat that’s already running — reads every final response, and evaluates it against what you said done looks like. If an agent doesn’t pass, Taskmaster tells it exactly what was wrong and sends it back to try again.

Every task has one of three states at any moment: Complete, In Progress, or Needs User Input — for the rare case where the agent genuinely can’t move forward without a human decision.

You don’t babysit. You come back to results.

The QA Loop

The difference between Taskmaster and a plain task queue is the evaluation step. Most orchestration tools mark a task done when the agent finishes. Taskmaster asks whether it should be.

The Stress Test — Failure Caught in the Wild

Taskmaster can queue unlimited tasks and run up to 40 concurrently. For this test: 40 tasks running simultaneously, each one instructing its agent to use Parallel Dispatch to fetch 20 YouTube transcripts — 800 agents total. The screenshot below is mid-run, showing exactly what the QA loop looks like under that kind of load.

The rightmost panel is the moment worth reading. Taskmaster evaluated the agent’s output, diagnosed that 14 of 20 workers couldn’t access the YouTube tools, identified exactly which tasks succeeded and which failed, and generated a targeted correction prompt — all automatically. The agent is being sent back with a precise brief on what to fix. Nobody typed anything.

What You Configure Per Task

• Directive — what the agent should do. Plain language, as specific as you want.
• Success criteria — how Taskmaster knows it’s done. "All 20 transcript files must exist on disk." The evaluator reads this literally.
• Evaluation model — you choose which model judges the output. Use a cheap fast model for evaluation and save the expensive one for the actual work.
• Agent model — override the model per task, or leave blank to use the chat tab’s default.
• Cost limit — set a spending cap per task when using cloud models. Local models run free, no cap needed.
• Mode — launch a new agent tab, or hand the task to an existing chat that’s already running.
• Sub-tasks & execution order — nest child tasks under a parent. Run them sequentially or all at once. Dependencies enforce order across top-level tasks too.

The Journal

Every agent response, every evaluation decision, every correction prompt — logged. After any run you can open the journal and read the exact reasoning the evaluator used for each task. If something failed, you know why and at which step.

Why It Exists

I built Taskmaster because life doesn’t let me sit at a desk all day — and even when it does, some jobs are too long to watch. I needed a way to start a serious overnight run — build a decompiler from scratch, process a thousand files, research a topic end to end — set what done looks like, and wake up to either a completed task or a clear failure report with exactly where it went wrong and why.

Not a vague "something failed." A diagnosis. A log. A record of every decision the evaluator made and every correction it sent back.

You can use it however you want. Queue up a week of work before you go to bed. Set it on something that would take you three days manually and see if it’s done before breakfast. Hand off the parts of your job that are slow and repetitive and go do the parts that actually need you.