# AIBuildAI-2.5 claims 73.3% medal rate at top of MLE-Bench

> AIBuildAI-2.5 tops MLE-Bench with an author-reported 73.3% medal rate, built around LLM-judged tree search, resource-aware scheduling, and cheaper-model routing.

- **Topic**: Models
- **Published**: 2026-09-24T05:31:12.620Z
- **Canonical URL**: https://highsignal.sh/stories/aibuildai-2-5-claims-73-3-medal-rate-at-top-of-mle-bench-01e73a0e

## Why It Matters

Autonomous agents that build AI models are inching toward the capability of experienced AI engineers on realistic benchmarks. The authors here say the bottleneck has shifted from raw capability to efficiency: too few candidates executed per budget, unplanned training-job scheduling, and a single expensive model serving every agent call. AIBuildAI-2.5 targets all three at once, which is the practical constraint on running these agents at scale rather than in demos.

## Key Findings & Analysis

### The efficiency fix: judge, schedule, route

An arXiv preprint (cross-listed in CS.CL and CS.AI) from an unnamed author team proposes AIBuildAI-2.5, an agentic system that builds AI models by framing model development as a code search problem solved through tree search with LLM agents. The authors identify three efficiency weaknesses in existing agents of this type: few candidates can be executed within a realistic budget, so reward rankings rest on sparse, noisy scores; no resource-aware strategy schedules training jobs, hurting hardware utilization; and every agent call is served by one powerful model, inflating inference cost. The system addresses these with an LLM-guided tree search in which a judge scores each candidate on expected improvement, grounding, and feasibility, and a selector ranks candidates from those scores plus search state; a scheduler that launches training jobs according to hardware resource status; and a router that assigns lower-cost LLMs to less demanding tasks while reserving the most capable model for the hardest sub-tasks.

### Reported results

The preprints state AIBuildAI-2.5 ranks first on MLE-Bench with a medal rate of 73.3% and outperforms a strong baseline on six autonomous AI research tasks from AIRS-Bench. These are the authors' own reported figures from the preprint, not independent evaluations, and the supplied evidence includes no third-party replication, methodology detail, or baseline naming. Treat the ranking and medal rate as claims pending external verification.

## Primary Sources & Citations

- [AIBuildAI-2.5: Efficient Autonomous AI Model Development Through LLM-Guided Tree Search](https://arxiv.org/abs/2609.25047) — *ArXiv CS.CL (Computation and Language)* (Reporting)
- [AIBuildAI-2.5: Efficient Autonomous AI Model Development Through LLM-Guided Tree Search](https://arxiv.org/abs/2609.25047) — *ArXiv CS.AI* (Reporting)

---

[← Back to front page](https://highsignal.sh/) | [Daily Brief](https://highsignal.sh/brief) | [All stories](https://highsignal.sh/latest)