Autonomous post-training research
Athanor
An Agent-Native System for Autonomous Post-Training Research
Yufa Zhou, Yichen Wang, Jiakun Fan, Ziqi Shu, Zhichao Liu, Anru R. Zhang, Neiwen Ling, Zijun Wei
ByteDance · Duke University · The University of Chicago · Virginia Tech
Motivation
Frontier labs say AI has started to build AI
“More than 80% of the code we merge … was authored by Claude.”
“We have now reached the goal … of having an automated research intern.”
“As models build future models, the rate of improvement may become staggeringly fast.”
Athanor · 2 · anthropic.com/institute · openai.com/index/research-acceleration-view-inside-openai · darioamodei.com
★ The question
Agents get stronger every few months.
What should people build around them?
Possible directions
Tune the harness for this model?
prompts and strategies that beat a past version;
but the next model generation may not need them
Design its action space like a domain-specific language?
ours, v0.1–v0.3: each next model outgrew them
Teach it how research should go?
plugins, skills, approval steps, preregistration gates, workflows
Our answer: build the feedback and the environment
a frozen evaluator, a sealed held-out exam, every score tied to its recipe
Agent = model + harness, trained together by the labs; teaching it our human-designed research procedure does not scale.
Our bet, as in the bitter lesson: people set the goal and build the environment, and the better agent does the rest.
“The two methods that seem to scale arbitrarily in this way are search and learning.” (Rich Sutton)
Athanor · 3 · left: schematic · cf. Rich Sutton, “The Bitter Lesson”, 2019
The task · AutoPostTrain
Four agents post-train one raw base model with public data,
real GPUs and no research advice: how far can they go alone?
Athanor · 4 · agents: Claude Opus 5.5 in Claude Code · one campaign: these 104 hours · dev tasks: the agents can have their models scored on them any time · held-out: a matching task for each dev task, never seen, scored once at the end · at hour 15.8 they are told only the team’s best submission counts
★ The main result
In 104 hours, from 15.9 to Qwen’s own instruct model
Four agents, a public base model and public data, no research advice.
Athanor · 5 · y: mean score on the six dev tasks · dots: every full evaluation during the campaign, colored by agent · labels: where each step first raised the best · final line: remeasured after the campaign with a longer output limit, every model alike, mean of four evaluations
★ The main result · other base models
Three other base models, one agent per run, 72 hours:
each run beats its maker’s instruct-level release
The same environment carries over to other base models; the reasoning releases stay ahead, as on Qwen3-4B.
Few Olmo and Qwen3-8B Claude Code points: those agents mostly ran diagnostics on some tasks; only full evaluations are plotted.
Athanor · 6 · left: development suite, best checkpoint so far (mean of its seeds) · right: held-out suite, mean of 4 seeds · one run per base: Opus 5.5 in Claude Code, 8 GPUs, 72 h; on Qwen3-8B also GPT-6.1 Sol in Codex, 72 h · grey: instruct-level release (Qwen3-8B has none: its hybrid release in non-thinking mode) · dashed: reasoning release
How it works
Each agent runs its own research loop; the GPUs do the slow part
The agent thinks for minutes, then sleeps while the GPUs train for hours and evaluate in about 26 minutes; every step lands in the record.
Athanor · 7 · each agent runs this loop on its own node, in parallel with its teammates
How it works · two kinds of long-horizon
The horizon here is the clock, not the tokens:
22 of 24 hours went to waiting for jobs
fix a code bug
tests answer in seconds
~10 s
analyze data
a query answers in seconds to minutes
~1 min
post-training
a training job answers in hours
each try holds a whole GPU node
~5 h
lab science
an experiment answers in days to months
~weeks
The run is bounded by GPU-hours and the clock, not by the agent’s turns. Harness tricks (memory, compaction, subagents) speed up the turns, so they touch little of wall-clock time; that is the structure of AutoPostTrain.
Athanor · 8 · one 24-hour run, Opus 5.5, Qwen3-4B, to scale · 22 of 24 h: gaps ≥ 15 min; drawn: the three longest · campaign: 87% (slide 31) · task bars: typical, log scale 1 s–1 month
How it works · six agents
Six agents, same 24-hour task:
runs of one agent can differ more than the generations do
Dots: single 24-hour runs; bars: their mean. Opus 4.8’s runs span 26.0 to 38.5; the same setting run twice differs by up to 9 points (35.0 and 26.0).
Athanor · 9 · one agent, Qwen3-4B base, 24 h each, with Athanor; lower than slide 5, which is four agents over 104 h · Opus 4.8’s 38.5 ran on an earlier harness · Instruct-2507 is 47.5 under the evaluator these runs used
Part 1 of 4
What the agents did
Take away: they research like researchers, and the data picked in hour 1 and its first fine-tune carry most of the gain.
1 · What the agents did
2 · What Athanor is
3 · Cost and survival
4 · What we learned building it
Athanor · 10
What the agents did · the result in detail
Level with Qwen’s instruct model on average, with different strengths; Qwen’s reasoning model stays ahead
Math
Logic
Code
Knowledge
Instruction
Action
AIME, HMMT 2025
MMLU-Pro
ZebraLogic
LiveCodeBench
IFEval
ALFWorld
team submission 67.3 · held-out 48.2
Qwen3-4B-Instruct-2507 66.5 · held-out 47.9
Qwen3-4B-Thinking-2507 80.5 · held-out 63.0
Qwen3-4B-Base 15.9 · held-out 9.2
Ahead: action, code, knowledge
the biggest lead: multi-step action (ALFWorld)
Behind: logic, math
most of the logic gap is in the larger puzzles
Why these six tasks
Different capabilities: math, logic, code, knowledge, instructions, action
Not saturated: the base scores 0–35, so methods can pull apart
Verifiable: a rule checks every answer, no judge model
Athanor · 11 · final measurement · held-out: AIME/HMMT 2026, ZebraLogic other half, LiveCodeBench 2025, SuperGPQA, IFBench, new ALFWorld tasks
★ What the agents did · the recipe
The recipe: five known steps, composed across all four agents
1 · SFT
mostly one public reasoning corpus, all six tasks
→ 49.7
2 · RL
RL that punishes answers stuck in loops
→ 61.7
3 · merge across agents
average two or three agents’ checkpoints
→ 61.8
4 · weight extrapolation
push the merge to 1.2–1.3× as far from the base
→ 65.5
5 · self-distill
keep the shortest correct of four samples
→ 66.1
Agent 1
Agent 2
Agent 3
Agent 4
0
24
48
72
96
↑ told: scored as a team
submission
Athanor · 12 · cards: the submission’s lineage · grey: fetched from a teammate · six later 24-h runs: first fine-tune = 72–98% of held-out gain
★ What the agents did · 1 · Agent 3
Agent 3 reads every cut-off answer:
in code and logic, most are stuck in repetition loops
HOUR 10
finds the loops
HOURS 10–13
RL misses them
HOURS 12–15
writes a fix
HOURS 16–22
the team adopts it
answers cut off at the length limit: how many are loops
code
331 of 351
logic
238 of 292
math
46 of 486
■ repetition loop
■ just too long
what a loop looks like: one logic-puzzle answer
… So April’s birthday is in house 3. So April is the person in house 3. So April’s birthday is in house 3. So April is the person in house 3. So April’s birthday is in house 3. …
the same two sentences, 529 times, until the output limit cuts the answer off
It reads every answer its model left unfinished. When the model always takes its most likely next word (greedy decoding), it repeats itself until it runs out of space.
Athanor · 13 · cut-off answers of one evaluation, by task; the excerpt is verbatim from the same evaluation
What the agents did · 1 · Agent 3
A training–evaluation mismatch:
RL never sees the loops the evaluator scores
HOUR 10
finds the loops
HOURS 10–13
RL misses them
HOURS 12–15
writes a fix
HOURS 16–22
the team adopts it
RL learns from
sampled answers (temperature 0.8)
→
they rarely loop: nothing to punish
The evaluator scores
greedy answers on code and logic
→
they loop until cut off
After 25 RL steps, greedy accuracy has not improved: four tasks within 3 points, math down from 36 to 23. The agent stops the job.
“Sampled rollouts (T .8) rarely loop, so GRPO on them does not touch greedy loops. Stopped grpo1 at step 25.”
its note: T .8 = temperature 0.8; GRPO = the RL method; grpo1 = its first RL job
Agent 3 could find this because it had the full record: every rollout and every graded answer, not only a score.
Athanor · 14 · greedy accuracy on the agent’s own held-back problems, step 0 against step 25 · quote verbatim from its notes
★ What the agents did · 1 · Agent 3
The fix: decode a quarter of the rollouts greedily,
so a loop loses reward
HOUR 10
finds the loops
HOURS 10–13
RL misses them
HOURS 12–15
writes a fix
HOURS 16–22
the team adopts it
SEES · hour 12
its training rollouts, sampled at 0.8: only 2–7% of the cut-off ones loop
→
INFERS
“So training never saw the greedy failure mode.”
→
ACTS
rewrites the RL trainer’s sampling step: one in four training answers is greedy
four rollouts of one problem: three sampled, one greedy
sampled
reward 1
sampled
reward 1
sampled
reward 0
greedy, loops
reward 0 → pushed down
number of looping answers on validation, before → after 20 RL steps
86 → 18
greedy accuracy: math 33 → 53 · code 22 → 34 · knowledge 44 → 55
The looping answer now scores 0 next to answers that score 1, so RL learns to stop looping.
Athanor · 15 · GRPO step 20 against step 0, greedy validation · a new RL job on a rebuilt problem set, so step 0 differs from slide 14 · quote verbatim from Agent 3’s session
What the agents did · 1 · Agent 3
It writes the diagnosis and the code into its notes;
within hours all three teammates copy it
HOUR 10
finds the loops
HOURS 10–13
RL misses them
HOURS 12–15
writes a fix
HOURS 16–22
the team adopts it
49.7 → 57.3
Agent 3’s own score with the fix, full evaluation at step 40
Hour 19.3: Agent 1’s RL job with the fix reaches 52.2 → 56.5 in 25 steps.
Agent 1: “… 25 steps with r3’s 25% greedy rollouts … 56.50 … Thanks r3”
Athanor · 16 · hour 0 = the start of the campaign · times from the agents’ transcripts · quotes verbatim · r3 = Agent 3
What the agents did · 2 · told once; how to share was their own
Told they are scored as a team,
they start sharing within a minute
56 s
from the notice to the first checkpoint fetched from a teammate
Agent 4: “averaging two EQUAL-strength RL models from different lineages gives ~+2.7” (its own estimate)
Agent 4: “great find, r3. Running (not duplicating yours)”
By the end it is one search: after hour 84, 28 of 32 new checkpoints build on all four agents.
Athanor · 17 · told at hour 15.8 · r3 = Agent 3
★ What the agents did · 3 · merging
After hour 21, every new best is a merge of several agents’ checkpoints, or was trained from one
● parents
● the merge
x: dev score
hour 25 · Agent 1’s RL + Agent 3’s RL
58.7 and 59.6
61.5
hour 31 · Agent 1’s RL + Agent 3’s RL
59.7 and 59.6
61.1
hour 41 · the hour-25 merge + Agent 4’s RL
61.5 and 61.5
62.3
57
59
61
63
A merge beats both parents
In these three merges, the average scores 0.8 to 1.9 points above the better parent.
A merge is cheap and fast
Minutes of CPU, then one evaluation: the fastest loop on offer.
It takes a teammate’s gain
139 of the 287 links in the checkpoints’ family tree pass through a teammate’s checkpoint.
Athanor · 18 · links: the lineage graph of slide 22 · dots: single evaluations, which vary by about half a point (rescoring noise) · merges: equal-weight averages
★ What the agents did · 4
They re-run a winner before trusting it: 19 of 23 come out lower
Where the rescoring noise comes from: the same checkpoint, scored twice, differs by about half a point. Answers of math benchmarks are sampled, with a new seed each evaluation, and even the greedy tasks shift a little between runs.
Why: the top score also picked the most luck
score = true quality + noise
E[noise | picked as top] > E[noise of a re-run] = 0
so the re-run is lower on average: regression to the mean
The harder the pick, the bigger the fall:
a new team best fell 0.54, a non-best 0.20.
Not a bug: in a later campaign, 27 of 36
re-runs were lower too; with no picking,
the first score is no higher than the rest.
Athanor · 19 · a re-run is one more full evaluation of the same checkpoint, on new seeds · colors: the four agents · right: an illustration, not data
What the agents did · 5
Weight extrapolation: pushing a weight direction further from the raw base model trades knowledge for logic
θ = θraw + f · (θmerge − θraw)
• θraw: Qwen3-4B-Base, before any training
• θmerge: an average of three agents’ trained models
• f = 1 is the merge; f > 1 pushes further along what training changed
Logic rises from 56.4 to 63.0 at 1.2; knowledge falls from 66.6 to 63.8 at 1.3.
It keeps 1.2, the best trade, and re-runs it before relying on it.
Athanor · 20 · Agent 3’s sweep, one evaluation each · merge: an average of three agents’ checkpoints
What the agents did · 6
They test their own ideas fast and drop what fails
Case 1 · one fine-tune, then dropped
Agent 2 · hours 31–32
the idea
Fine-tune on a solver program’s step-by-step traces.
what it did
One fine-tune: logic 49 → 27. It reads the answers:
“model imitates trace style but asserts unjustified deductions”
Dropped after one fine-tune.
Case 2 · dropped before any training
Agent 1 · hour 95
the idea
Fine-tune on the model’s own shortest correct answers to big puzzles.
what it did
It counts first: of 157 solved 5×5 puzzles, 3 are under 8k tokens.
“no short correct traces to learn from”
Dropped without spending a GPU hour.
Why it is cheap
Every answer is kept, so a check is a few lines of code, not a training job.
Athanor · 21 · quotes verbatim from the agents’ notes
★ What the agents did · 7
Where they slow down:
late jobs are mostly the kinds that score fastest
Hours from submitting a job to its first score
Merge weights
0.8 h
RL
2.7 h
Train on new data
8.6 h
median, this campaign
The data picked in hour 1 is never replaced
Here and in all nine other runs we checked, on four base models. Late jobs are mostly merges and RL on data already used.
Correctness is checked only where it is cheap
Agent 3, hour 1: “That math row … looks like noise, but the pass-rate filter should catch it.” The filter never runs. In other runs, a check is one line in a data script; corpus labels are mostly taken on trust.
The real lineage at the close: 174 evaluated checkpoints, 287 links. Colored: the submission and the 22 checkpoints it was built from. White: 105 branches that ended with no child.
Athanor · 22 · lineage graph at the campaign’s close; dots are full evaluations, x = hour, y = dev score
Part 2 of 4
What Athanor is
Our design philosophy: hold fixed only what the agent cannot supply for itself, namely feedback it cannot fake and a record complete enough to audit every finding.
1 · What the agents did
2 · What Athanor is
3 · Cost and survival
4 · What we learned building it
Athanor · 23
★ What Athanor is · the whole system
The agent decides; services run it and write it down
Research is the decisions; everything else is a service that returns its result in full, scored by something the agent cannot change.
Athanor · 24 · outside the loop, the operator, a person or an agent acting for them, runs the nodes and scores the held-out suite at the end
What Athanor is · where it sits
Athanor holds fixed only what the feedback needs;
the agent builds the rest
the agent builds it
Athanor provides it
someone else fixes it
agent can change
agent cannot
From scratch
e.g. PostTrainBench
Athanor
Hosted training API
e.g. RSIGym
research code
agent writes
agent writes
agent writes, inside the API
trainer
agent writes
starter given, agent may rewrite
fixed: LoRA via Tinker, set knobs
evaluator
benchmark script
frozen service
provider service
data cleaning
agent, on trust
service removes test overlap
agent, on trust
GPUs
one GPU, 10 hours
whole nodes, used any way
hidden behind the API
Athanor · 25 · refs: Rank et al., PostTrainBench, ICML 2026 · Meng et al., RSIGym, arXiv:2610.10310, 2026
What Athanor is · the agent, untouched
The agent runs unmodified, Claude Code or Codex;
everything we built is the environment around it
the environment: everything we built, as services
workspace
swappable
starters for SFT, RL, merging
data
swappable
catalog, decontamination
sandbox
fixed
what it sees and reaches
Claude Code
native, unmodified
>_
Codex
native, unmodified
stock binaries, default tools: no plugins, no MCP servers, no custom tools
command
fixed
athanor: job, eval, data, artifact
services
fixed
queue, run, recover
record
fixed
journal, lineage
evaluator
swappable
scoring frozen; suites and base models plug in
platform, store
swappable
where nodes and bytes come from
operator
fixed
runs and recovers the nodes
What we set on the agent: how it starts
which Claude model, its permissions in the sandbox, a hook that records each session start
Around it: the whole environment
sandbox, CLI command, services, record, evaluator, GPUs
Swappable pieces drop in
agent, benchmarks, base model, starters, data, GPU platform
Every piece is a service behind one interface; what we built is the environment, not a wrapper around the agent.
Athanor · 26 · swappable: another implementation drops in behind the same interface
What Athanor is · four commands
Four commands, one per thing post-training makes;
everything in between is a service
one per thing it makes
the agent decides
in between: hidden, recorded, opened on demand
the agent reads
athanor job
runs a training job
your code, data, GPUs
queue
launch
watch for stalls
recover
checkpoints + logs
athanor eval
makes a score
a checkpoint
serve
sample
grade
aggregate
scores + every graded answer
athanor data
makes a dataset
a public dataset
download
decontaminate
save a clean copy
a clean dataset
athanor artifact
keeps a checkpoint
a checkpoint
copy to store
check bytes
link to its job
saved · fetched · submitted
Each command runs in the background with a fixed start and end. Decisions stay serial; the work they launch runs in parallel, and each service can be made faster on its own.
Athanor · 27 · opened on demand: every step in between is recorded · athanor submit only marks one saved artifact as the team’s answer
What Athanor is · lineage
Every command leaves a record: together, the lineage graph
174 checkpoints, 287 links
this campaign’s graph: each evaluated checkpoint with what it came from
22 checkpoints behind the submission
built by all four agents, traced back to the base model
139 of 287 links
pass through a teammate’s checkpoint
Every finding about the agents in this talk is read from this record: each score traces back, step by step, to the base model, so a submission can be audited.
Athanor · 28 · edge names as the lineage graph records them; the commit shown is illustrative · counts: the lineage graph of slide 22
What Athanor is · progressive disclosure
Progressive disclosure: push a short fact and a pointer;
the agent pulls the raw record when it needs it
At each milestone
the agent publishes the checkpoint and a note, so a reset, a new node or a teammate picks up from there
The same rule as the manuals
AGENTS.md first, a command’s --help when it is used, the full docs when needed
A small window onto a huge record, with pointers: the way an agent already works with files, paths and --help.
Athanor · 29 · the record: journal · lineage · transcripts · workspaces · example: Agent 3 at hour 12, notice shortened
Part 3 of 4
What it costs, and what it survives
Take away: GPUs are the cost; a lost node costs minutes, not the research.
1 · What the agents did
2 · What Athanor is
3 · Cost and survival
4 · What we learned building it
Athanor · 30
Cost and survival · what it costs
Most of the climb is bought early; the last points to the instruct release cost the most
GPUs are the cost
the agents’ inference is 9% of $11.8k
The agents mostly wait
87% of session time waiting on jobs; 3.6% reasoning
In this campaign, jobs almost always held the GPUs
jobs hold 93% of all GPU hours
Athanor · 31 · 87% waiting: the four agents over the whole campaign (slide 8 shows one 24-hour run) · money: GPU hours held by jobs at $3.49 each, plus inference as Claude Code counts it · curve: best single evaluation so far, lucky draws included (slide 19), so the submission’s own first score, 66.1, sits below it · the submission remeasured after the campaign with a longer output limit, mean of four evaluations, is 67.3
★ Cost and survival · long-horizon reliability
Over 104 hours the platform took back three of four nodes; each affected agent resumed from its record
Agent 1
Agent 2
Agent 3
Agent 4
Records live off the node, so a lost node costs minutes, not the research.
The same records are the data behind every finding about the agents in this talk.
Athanor · 32 · node lost: the platform reclaimed it, the operator restarted the session on a new node · red block: the outage, to scale
Part 4 of 4
What we learned building it
Where the philosophy came from: each stronger agent outgrew our design; what stayed is what it cannot do for itself.
1 · What the agents did
2 · What Athanor is
3 · Cost and survival
4 · What we learned building it
Athanor · 33
★ What we learned · how we got here
Five months, four versions:
each stronger agent outgrew our last design
● Anthropic
● OpenAI
● open weights
← released before June
GPT-6.1 Sol
Kimi K3.1 · Oct 23
Claude Fable 5
Claude Sonnet 5
Claude Opus 5
Claude Fable 5.1
Claude Opus 5.5
GPT-6
GLM-5.2
Kimi K3
DeepSeek V4.1
← GPT-5.5 · Apr
GPT-5.6
DeepSeek V4-Pro · GLM-5.3
v0.1
exploring: a menu of 5 tools
v0.2
typed form
per recipe ×
v0.3
a framework of scaffolding ×
v1.0
services, records,
free code ✓
Each version fit the agents of its month; the next, stronger agent found it a limit. What lasted was what assumed least.
Athanor · 34 · public release dates
What we learned · v0.3, August–September
What we polished most, the agent never used:
it worked from the plain services and the evaluator’s code
The setup: every part sat open in the sandbox and none was required; the action space was not fixed to our interface. Three 24-hour runs on v0.3 (Opus 5 once, 5.5 twice); we counted what each used.
Lesson: ask a bold question; run the plainest baseline first; then build only what it needs.
Athanor · 35 · uses: references in the transcripts; 86, 50 and 12 are from the Opus 5 run, 2,603 tool calls · services: filter, evaluation, publish, jobs · source: engineering retrospective, 23 September
What we learned · in use
Left alone, they use the environment as designed:
the shell first, our commands by choice
shell: athanor
1,588
shell: other
1,541
Write
56
Edit
12
Read
10
other
11
tool calls of the four agents, whole campaign; half of the 3,129 shell calls run an athanor command
Every training job via athanor job
though the GPUs sit in their own shell
Teammates’ files read 292 times
read straight from their files, not through a viewer we built
57 checkpoints fetched from teammates
all through athanor artifact, so each one lands in the lineage
An environment can only bring out what the agent was trained to do. Design for native behavior.
Athanor · 36 · counts: whole campaign, from the transcripts · command counts include status and log queries, and one shell call can run several
What we learned · the lesson
Newer agents need less from us.
Only constraints, feedback and history stayed with us
the environment decides
the agent decides
v0.1
a menu of 5 tools
v0.2
forms the agent filled in
v0.3
scaffolding around the agent
v1.0
starter code and a list of known pitfalls on top
next
starter code goes back to the agent
So human effort goes into the environment, not into the research procedure: keep it stable, and keep it thin.
Athanor · 37 · schematic: widths show the direction, not a measurement
What we learned · why post-training first
Post-training has a known target, known moves and a first step far above noise. Algorithm research is closer to a random walk
Research and engineering are one loop of trying; they differ in how defined the problem is and how noisy the feedback.
Athanor · 38 · left: our campaign’s best score; middle and right: schematic
Limits
Single runs are noisy, and the headroom is not reached yet
One long run per base model
• Qwen3-4B: a team of four, 104 h
• three more bases: one agent, 72 h each; Qwen3-8B also with Codex
Runs vary a lot
• one 24-hour setting, run twice: up to 9 points apart
• so one run cannot rank two agents
Public data, partly distilled
• no teacher model in the loop
• but much public data was distilled from stronger models
Not yet at the ceiling
• reasoning releases of the same size score far higher
• no 24-hour run reaches Qwen’s instruct model
Athanor · 39
What comes next
More research in this environment, then the same kind of environment for other fields
Arena: compare agents
• same environment, other agents and base models
• compare how efficiently they do research
• repeats turn one result into a measurement
Automated science of post-training
• many campaigns
• turned into tested knowledge
• each rule kept with the experiments behind it
A world model of research
• predict a result from the record
• spend GPUs only where unsure
• learned from every campaign’s full record
Autoresearch for other fields
• wherever the feedback is objective
• same parts: a frozen evaluator, services, the record
• the pace: seconds for kernels, days to months for lab science
Athanor · 40
Toward a larger loop
The loop stays;
the pieces around it can be handed to agents, one at a time
autoresearch
loop
propose
try
feedback
select
Recipes
today: agents pick and combine methods
next: agents propose new methods
Data
today: a public pool we chose, plus what they fetch or generate
next: agents generate, verify, clean
Benchmarks
today: dev and sealed suites we chose
next: agents build the suites
Researchers
today: an off-the-shelf agent
next: RL-train agents in this environment
Environment
today: Athanor, built by us with coding agents
next: agents improve it from the researchers’ trajectories
Base model and goal
today: Qwen3-4B-Base and a score to beat
next: agents choose what to improve
Athanor · 41 · what is given today, and whether agents can take it over · orange: already handed to the agents · blue: this talk’s environment, built with coding agents · dashed: next to hand over
★ Takeaways
Agents can already do post-training research end to end.
Build the feedback and the records; hand them the rest
It can be done
Off-the-shelf agents took a raw base model to its maker’s instruct level in four days.
sealed held-out: 48.2 vs Qwen’s 47.9
They research like researchers
Known methods, reading failures, building on teammates, doubting their own scores. And they follow the fastest feedback.
they re-ran 23 apparent winners before trusting them; 19 fell back
Set the goal, build the environment
The run is bounded by GPU-hours and the clock more than by the agent, and teaching it our procedure did not scale.
one 24-hour run: 22 of 24 hours waiting · 0 uses of our plugins and skills
Where feedback is objective and each step clears the noise, research becomes a search agents can run; people choose what gets scored.
Athanor · 42 · cf. Felix Rieseberg, “How to be more ambitious”: use an outside judge; don’t let the agent grade its own work · z.ai, “Toward RSI” (2026)
Backup
Design philosophy and details
for questions: design ideas · our choices · earlier versions · stepping back
Design idea · research as decisions
Research is a chain of decisions. Code gets the information right; whoever reads it decides
Three design questions: who makes each decision · what it needs to know · how it gets that, losslessly and cheaply
Our answer: research made programmable. Mechanism in code, policy in the agent.
Athanor · B1
Design idea · native primitives
Native primitives: context on demand, commands that compose in code, and only the measurement fixed
MCP tools + Agent Skills
the agent’s context
tool schemas, loaded up front
MCP server
Agent Skills
~ context: schemas loaded up front
✓ evaluator: a server tool, fixed
research actions become a fixed menu again, as in v0.1
tried in v0.1 and v0.3
From scratch, a frozen evaluator
agent
trainer
agent writes
evaluator
frozen, one command
data cleaning
agent writes
✓ context: only what the agent reads
✓ evaluator: frozen, behind one command
no structured record: only git and transcripts, so an audit rebuilds the lineage by hand
our control arm, no Athanor services, October
CLI + Markdown + free code
Markdown manuals, read on demand
AGENTS.md
always read, short
INTERFACE.md, PITFALLS.md
when needed
starters/*/README.md
when used
athanor CMD --help
per command
trainer
agent writes, free
evaluator
athanor eval
data cleaning
athanor data
✓ context: AGENTS.md, then --help
✓ evaluator: frozen, behind one command
the agent writes all research code; scoring and decontamination stay frozen
Native: the shell, files and code the agent was trained on; nothing is mandatory except the measurement.
Athanor · B2 · cf. Anthropic, “When to fold subagents into your main agent loop”, 2026 · “Code execution with MCP”, 2025
Design idea · who decides, who sees what
Three parties: the operator runs the lab, the agent researches, code keeps the rules
judges
can touch
sees
operator
where and when to run; the final score
everything; each action logged with its reason
operations facts of every node; the sealed suite
agents
all research
its own nodes: its own code, or our commands by choice
its own research and the team’s record
code
nothing
runs and records everything
declarations and records
The operator never steers the research, and the code adds no research limits or approvals of its own: each would be a hidden research decision.
Athanor · B3 · the operator is a person or an agent acting for them; none of the three acts for another
Our choice · one decision maker per line of research
One decision maker per line of research holds the full context. Everything else is a service
Evidence
records, every graded answer, papers, teammates’ notes
externalized: the same for everyone who reads it
Decision
what to try next, how to read a result, when to stop
why one agent with the full context
• it rests on every detail at once
• a reviewer lacks the full context, so judges by a general prior
• each hand-off loses some detail
Execution
jobs on GPUs, frozen evaluation, checkpoints, recovery
in code: the same every time, reusable, its result returned in full
Feedback comes from the world: a job finishes or fails, a frozen evaluator returns a score; not opinion.
Athanor · B4 · many agents pay off when work splits into pieces with cheap checks; post-training experiments are costly and coupled
Our choice · why not a team of specialists
Splitting research by role turns it into a relay; one agent with services keeps every detail
Split off only work that can be checked on its own, a job or a score; keep every decision with the one agent that sees everything.
Athanor · B5 · cf. Anthropic, “When to fold subagents into your main agent loop”, 2026
Our choice · command, code or Markdown
Fixed steps go in a command, research in the agent’s own code, judgments and lessons in Markdown
CLI command
The steps are fixed and must run the same way every time, for trust or an unattended deadline.
athanor job: start a training job
get GPUs
→
start
→
watch
→
save
if something unexpected happens, it stops and reports what it saw
Who decides between steps: nobody; the steps are written in code
The agent’s own code
The research itself: what to try, and how to change the training.
its training code
+ every 4th answer: greedy
Agent 3’s own fix: looping answers now lose reward
Who decides: the agent, by writing the code
Markdown
A judgment between steps, or a lesson not to repeat.
Runbook · a machine was lost
◆ old one dead?
→
new one
→
resume
◆ if the old one may still run, stop it first: two machines must never write one record
PITFALLS.md · a trap hit once, written down
#2 vLLM re-imports your script → add an if __name__ == "__main__" guard
Who decides: whoever reads it, the agent or the operator
Code detects and records; the decider chooses. A judgment that keeps getting the same answer graduates into code.
Athanor · B6
Earlier versions · v0.1, June–July
v0.1 fixed the agent’s action space: a menu of five tools, and every recipe had to fit it
Why we fenced it in: agents had cheated on PostTrainBench, training on test items and handing in the maker’s instruct weights.
Lesson: do not design the agent’s actions: a designed set misses moves the agent needs and builds in human bias that does not scale. Give it the goal and feedback it cannot fake; it writes the rest.
Athanor · B7 · the five tools as named in the v0.1 code; the agent’s recipes had to fit a fixed set of steps
Earlier versions · v0.2, July–August
A typed form for every recipe: the agent could try only the ideas the form had fields for
TrainSpec
method
SFT | GRPO | DPO
dataset
from the catalog
lr, epochs
numbers in a range
rollout
no such field
→ checks → queue → GPUs
“decode a quarter of the rollouts greedily”
an agent’s idea to stop repetition loops; found in v1.0, it has no field here
The natural way: write code
$ python my_run.py
running…
A form or a tool call limits what can be said; in its own code, an agent can express any idea.
A second mistake: our own agent harness
• we built it instead of using Claude Code
• every bug ours: context overflow, drift, tool design
• none of Claude Code’s upgrades; dropped 5 August
×
A designed interface can never list every idea, and it changes slower than the agent iterates; just let the agent write its own code.
Athanor · B8 · form simplified; the idea in the bubble is the one an agent later found in v1.0
Stepping back · research as optimization
Research is optimization by trial. The agent, the base model and the data determine the ceiling; the environment sets the speed
The same parts as any optimization
optimization
autoresearch
objective
the loss
the dev score; held-out stays sealed
one step
an update
one try: train, then evaluate
direction
the gradient
feedback in words: answers, logs, curves
noise
minibatch noise
evaluation noise, about half a point
speed
steps per second
tries per hour
The ceiling: how high it can go
fixed at setup; better feedback cannot move it
• the agent: its intelligence, and the prior it tries first
• the base model: the physical limit of what training can reach
• the data: the public pool on offer, with no teacher model
The climb: how fast it gets there
what the environment controls, within the given time and GPUs
• feedback: each score clear, and honest about its noise
• speed: more tries per hour, with fast jobs and fast scores
Automating research swaps the researcher; the environment’s job is more tries per hour, each scored clearly.
Athanor · B9 · cf. Jiayi Weng, “Learning Beyond Gradients”; Lilian Weng, “Harness Engineering for Self-Improvement”, 2026
Stepping back · verification at scale
AI first solves what it can verify. The longer one try takes to verify, the more the system around the agent matters
Post-training is verifiable, but each try takes whole nodes and hours: at that scale, verifying is infrastructure.
Athanor · B10 · cf. Jason Wei, “Asymmetry of verification and verifier’s law”, 2025 · bars: order of magnitude
Stepping back · a person or an agent
An agent reads everything and works around the clock, but keeps only what is in its context. So its memory lives outside it
a smart person
an agent
reading
a section at a time
a whole paper or codebase in one read
keeps going
days, with sleep
around the clock, 104 hours
judgment
taste from years of work
a strong prior, shared by every copy
memory
carries over
only its context and what it writes down
The agent holds the full context and decides; the person sets the goal; the world returns the scores; the record remembers.
Athanor · B11 · agent column: the four agents of this campaign · the record: journal, lineage, transcripts, workspaces