Campaign modes

A campaign is one piece of research: what to improve, reach or find out, and how to judge it. Thermite runs three kinds, called modes. This page explains how each mode works, what ends it, and how to choose one.

	Hill climb	Goal	Research
The question	How far can this number go without breaking anything else?	Can the project reach these goals?	What can be learned about this question in this time?
An agent’s work	One change to the code, and any sources, claims and notes it records on the way	One change to the code, and any sources, claims and notes it records on the way	Reading sources and recording claims, with quotes, and findings
What checks it	The judge, by timing the change against the champion with statistics	The judge, by each goal’s own test; a person accepts goals that need judgement	Code that finds every quote, word for word, in a page the rig archived, and checks each finding against verified claims
The best result	The champion: the best confirmed version	The champion: the version that meets the most goals	The findings, the verified claims and the report
It ends when	Never on its own: you pause it	Every required goal is met, or you accept that they cannot be	Its time window is over
The dashboard’s headline	The best gain over the starting point	Goals met	Verified claims

What every mode shares

Each campaign has a contract: a file that says what the campaign must achieve, what must not get worse and which files the agents may change. A contract has a version number; a changed contract is a new version, never an edit of a running one. A run is the campaign on one rig, and one campaign can have runs on several rigs. Write a contract and the contract reference cover every field.

In every mode, code decides when a campaign is done, never an agent. No agent can close a run, mark a goal met or end the research early.

Nothing ends a campaign by default. There is no limit on turns, attempts, failures or time unless you set one. The contract can set three budgets: hours since the run was created (paused time counts too), hours of judging on the device, and money spent on the model (counted only when the rig is told the model’s prices). A spent budget pauses the run; it never closes it. The Stop after (hours) field on the New campaign page sets the first. Set a budget or a stop time explains all three, and Why Thermite runs until the job is done explains why there are no defaults.

A rig works on one run at a time. Starting or resuming a run on a rig pauses the run that was running there; a paused run can be resumed later.

Research in every mode

Reading is not only for research campaigns. A change to code often rests on something read: a paper, upstream code, an issue or a specification. So workers in hill-climb and goal runs can fetch web pages through the rig’s archiving fetcher, read and search what the run archived, and record a claim with its exact quotes or a research note with the sources it rests on. Code checks their claims as it checks a research run’s (Research). Findings, hypotheses and report sections stay with research runs: a hill-climb or goal run refuses them.

Such a run’s research folder holds INDEX.md, sources/, claims/ and notes/, which the harness writes from the run’s records and keeps current, and knowledge/, the past research described below. Once the run has recorded anything there, each worker’s brief points to the folder, and tells the worker to search it before fetching a page again. The fetcher uses fixed values in these runs: it respects robots.txt, and fetches at most 6 pages a minute from each site and at most 5,000,000 bytes a page. In Pi, the worker has the tools fetch, read_source, search_corpus, add_claim and add_note; in OpenCode, Claude Code and Codex, the same five as thermite-trial research commands.

Field notes and past research

In every mode, the workers, the planner (where the run has one) and the advisors keep field notes as they work: short notes of what they observed, decided and learned, which the run’s Journal tab shows. Every run’s records also go into its campaign’s folder in Thermite’s knowledge base on control. Later runs of the campaign read that folder, and other campaigns’ folders when the owner grants them, in their research folder’s knowledge/. Journals and the knowledge base explains both.

Hill climb

A hill climb pushes one number, the primary metric, up or down: operations per second, run time, a score that the kit’s check reports. The contract says which way is better and the smallest gain that counts. It can also name:

• guards: other metrics that must not get worse, each with how far it may slip in one step and in total since the start, so a faster change that loses quality is not a win;
• hard constraints: pass-or-fail limits, such as a longest allowed run time;
• test cases that the judge measures every time, and hidden test cases that it runs only to confirm a change that already looks better. The kit and the contract call a test case a cell; the dashboard calls it a case.

How a change is judged. The judge builds the change and checks that it works on every test case. Then it times the change and the champion together, in blocks that alternate between the two on the same device, and decides with statistics set in the contract. A change that looks better is timed again in fresh blocks and run on the hidden test cases. Once a day, when it has nothing else to do, the judge times the champion against itself to learn how noisy the rig is, and uses that in every later decision. Why agents never judge follows a change through every step.

Who promotes. A confirmed win becomes the new champion at once, unless the run waits for the owner’s approval of every promotion. That rule is chosen when the run starts, or comes from the rig’s default. A surprisingly large gain first goes to an audit: by default, a gain above 15%, or above three times the median of the run’s earlier confirmed gains once it has three. So does a change whose agent declared known flaws. An audit can hold a result back, but never pass one. A gain above the 15% trigger, a change with declared flaws, and anything the auditor hands on to the owner then wait for the owner’s decision. On a rig with no auditor configured, audits wait for the owner too. Decide what waits for you shows those decisions.

When it ends. Never on its own. A faster program can always be tried, so no code closes a hill climb. It stops when you pause it, when a budget you set pauses it, or when another run starts on its rig. Its champion is ready to download at any time.

Goal

A goal campaign works until a list of goals is met. Each goal has its own test, which the contract names:

• a code test: a set of the kit’s test cases, called a verifier, that must all pass the kit’s check. A kit can order its verifiers and run an expensive one only after the cheap ones it depends on have passed;
• a metric threshold: a number, such as items per second, that must be above or below a value. The judge runs each test case several times (five by default) and counts the goal as met only when the whole uncertainty range is on the passing side;
• a review: a goal that needs judgement, such as the quality of documentation. The rig’s review panel, a group of models, reviews the champion’s change without knowing who wrote it, and needs at least two reviewers who all pass it (two to five in the advisors file, three when its model is chosen on Edit rig). Even then, the goal is met only when the owner accepts it.

The contract also says what the finished work must be, and can list results that do not count, such as a special case that only passes the tests. That list goes to the auditor and the reviewers; code does not check it.

What counts as progress. A change makes progress when it meets a goal that the champion does not, and loses none that the champion meets. A change that could not check a goal the champion meets is not counted as worse, but not as progress either: its verdict is inconclusive. A change that makes progress is checked again in fresh runs, and on hidden test cases for the goals it gained, before it is promoted. If the contract has an adversarial checklist, a list of ways a change could fake success for the auditor to look for, every promotion is audited first.

The goal report. When the run starts, and after each promotion, the judge checks every goal on the champion. The planner and the workers see that report, so they know which goals are still open, and the next round of work waits for it, for up to two hours.

When it ends. In one of two ways:

• Every required goal is met, or, if the contract says so, at least a set number of them. Because this ending is final, the judge checks the champion twice, independently, and closes the run only if both checks agree.
• You accept that the goals cannot be met. Thermite proposes this ending, called “not possible”, only when the contract allows it, and only with evidence: a number of different approaches, set in the contract, each tried at least three times with no progress. Those approaches are then set aside. You accept the proposal, which closes the run, or reject it, which reopens them; a new proposal then needs more failed approaches than the last one. No agent can propose it.

A goal campaign has no time limit of its own. Budgets pause it, as in any mode.

Your first goal campaign runs one from start to finish.

Research

A research campaign studies a topic for a set time. The contract gives the topic, the questions, starting URLs and which web sites to include or leave out. There is no project to change and no kit: the work is reading.

How the work is checked. Workers fetch pages through the rig’s archiving fetcher, which keeps a copy of everything it reads. By default it respects robots.txt. It limits how fast it fetches from each site and how large a page may be, and never fetches private or local addresses. A page a worker reads any other way verifies nothing until it is fetched through the archiving fetcher. A worker records a claim with exact quotes from those archived pages. Code then checks it:

• a claim is verified only if every quote appears word for word in the archived text, and every number in the claim appears in its quotes. Otherwise it stays a draft, with its problems listed;
• a verified claim whose quotes come from at least two different web sites counts as independently sourced. The contract can ask for more sites. The report lists these claims apart from those that rest on one site.

When verified claims together show something, a worker records a finding: the conclusion in one sentence, a summary, why it matters, and the verified claims behind it. Code checks a finding too. It counts only if every claim it cites is verified and every number in it appears in those claims. Otherwise it stays a draft finding, with its problems listed. The report gives each finding with its support, and says how many of its claims are independently sourced.

Workers can also write research notes, propose hypotheses that cite verified claims, and draft report sections. The report shows those sections as worker sections, never as findings, and flags any number in them that is in no archived source and no verified claim.

The research folder. Each run has a folder of files that its workers search and add to. The harness writes most of it from its records and keeps it current while the workers record. The workers can read what the harness writes and the lanes’ saved files, and change only their own lane’s working area:

INDEX.md          the questions, what is known so far, and where everything is
REPORT.md         the newest report
sources/          every archived page as text: the words a claim's quotes must match
claims/ findings/ hypotheses/ notes/ report/
                  every claim (verified or draft), finding, hypothesis, note and report section
lanes/<lane>/     each lane's files, as they were when its last trial ended
work/<lane>/      the lane's working area: syntheses, outlines, drafts, lists of what to read next
knowledge/        past research from the folders the campaign may read; its README.md says what is where

A lane’s working area is kept for its next trial, so a worker builds on what the trials before it learned. When a trial ends, the harness saves the lane’s files, records which trial wrote or removed each one and when, and shows them to the other lanes under lanes/. These files are working material, not results: only claims and findings recorded through the research tools are checked and reported.

What a worker has. It depends on the rig’s worker harness, the program the agents run in:

• In Pi, a worker has the research tools (fetch, read an archived source, search the archive, record a claim, a note, a hypothesis or a finding, write a report section, and see the progress) and file tools: list, read and search the folder, and write and remove files in its own working area. It has no shell. Its idea tool proposes another research task for the queue, and its note tool writes a field note.
• In OpenCode v2, a worker keeps OpenCode’s own tools, including its shell and files, as a hill-climb worker does. It starts in its lane’s working area and runs the research tools as thermite-trial research commands, proposes another task with thermite-trial idea, and writes a field note with thermite-trial note.

A Claude subscription runs only in Claude Code, which doesn’t run research workers, so a research campaign needs another model. On a rig whose workers run in Codex, research workers run in Pi, with the same model. OpenCode’s own free models work only when the agents run in OpenCode v2. A research campaign needs a model to do anything at all.

How the work is planned. Research is planned by fixed rules, not by a model. Each research task is an experiment in the run’s experiment queue: a kind of task (explore, find primary sources, look for contradictions, or consolidate) for one of the campaign’s questions. Each round, when the queue holds fewer tasks than the run has lanes (lines of search, each with one worker), the research plan adds tasks: the questions with the least coverage first. When an hour passes with no new source and no newly verified claim, primary sources and contradictions come first. Each lane then takes the next task in the queue, whoever added it, so the owner and the teacher pair can add tasks too. A worker’s idea for another task waits until the owner queues it or drops it, because no model plans a research run. The report shows what was new in the last hour. The owner can message the lanes and the teacher pair of a research run, but it has no planner, auditor or review panel to message (How the parts fit together).

What it produces. The final report and the whole research folder: the archived sources, every claim, finding, hypothesis and note, and the lanes’ files. A digest of the report comes every hour by default. The rig’s export after the final report carries the folder, without knowledge/, and every export carries the run’s field notes and research notes (Set up the results repository says what reaches the repository).

When it ends. When its time window is over. The window is the contract’s maximum duration, or its minimum duration when no maximum is set. Only running time counts: time spent paused does not. When the time is up, the trials still running finish, the final report is written and the run is closed. The report has no pass or fail; it is what the claims and findings support.

Some things are not built yet:

• Web search in the research tools: workers start from the campaign’s starting URLs and follow the links of pages already archived (goal 046).
• A model planner for research (goal 047).

Choosing a mode

• You have a number to make better, and things that must not break: a hill climb.
• You have a list of conditions that the work must meet, and you want to know if it can: a goal campaign.
• You have a question, and a few hours or days to spend on it: a research campaign.

The first step of the New campaign page, Aim, offers the three as Hill climb, Reach a list of goals and Research a question. Your own project starts a hill climb there.