arbor
Category: research-automation · License: MIT · Version: 1.2 · Includes scripts
Tags: autonomous optimization, hypothesis tree, research automation, AI agents, experiments
When to use it
Autonomously improve a real artifact (code, training recipe, agent harness, data pipeline, prompt) against an objective and an evaluator, using Hypothesis Tree Refinement (HTR) from the Arbor paper. Use this whenever someone wants to iteratively optimize something over many experiments without overfitting — e.g. “get my model’s eval score up”, “improve this agent/harness”, “tune this pipeline”, “beat the baseline on this benchmark”, “run a search over approaches and keep the best”, “do an MLE-bench / Kaggle-style optimization”, or any long-horizon “make this artifact better and don’t just memorize the dev set” task. Trigger it even when the user doesn’t say “Arbor” or “hypothesis tree” but describes repeated experiment-and-evaluate loops, branching exploration of competing ideas, or worries about a dev/test gap. Runs Claude itself as the coordinator with subagent executors in isolated git worktrees; for the standalonearbor CLI tool see references/arbor-upstream.md.
Example requests
Ask your agent in plain language, for example:- “I have a training script for a small transformer and I want to push validation perplexity down as far as possible. Can you run a long autonomous search over optimizer and architecture tweaks and keep only what really generalizes?”
- “improve my agent harness pass rate”
- “My repo at ./mle_tabular is a Kaggle-style tabular competition solution (LightGBM, features in features.py, 5-fold CV in cv.py, MAE currently 0.412). I’d like to try many different approaches in parallel, record what each one taught us, drop the dead ends, and make sure the final winner isn’t just overfit to the CV folds. Budget is about 20 rounds.”
Install
/plugin marketplace add KalarisLabs/research-agent-skills and install the whole research-automation category with /plugin install research-automation@research-agent-skills.