Andrej Karpathy • March 2026

Karpathy's
Autoresearch

AI Agents Running Autonomous ML Research on a Single GPU

Open Source Repository

github.com/karpathy/autoresearch

Agent Active: Autonomous Iteration

The Big Idea

Give an AI agent a small but real LLM training setup and let it experiment autonomously overnight.

Agent modifies the training code
Trains for exactly 5 minutes
Checks if validation metric improved
Keeps or discards the change
Repeats — all night, no human needed

Experiment Log

Experiment #104 PASSED

$ python train.py --run_id 105

> Patching: transformer.py (Attention Head 12)

> Training: [###########.........] 55%

> Val Loss: 1.42 → 1.38 (IMPROVEMENT)

_

"Better model by morning"

The Shift in Research

"One day, frontier AI research used to be done by meat computers in between eating, sleeping, and having other fun..."

That era is long gone. Research is now entirely the domain of autonomous swarms of AI agents running across compute cluster megastructures in the skies.

@karpathy — March 2026

AI compute cluster megastructure in the sky

Autonomous Swarm Active

Research Velocity

+10,000%

vs Human Baseline

Paradigm Shift

Agentic
Engineering

Applying autonomous loops to the core of machine learning research.

Autopilot Research

No manual hyperparameter tuning or babysitting runs. The agent manages the stack.

High Velocity

~12 experiments per hour. Roughly 100 iterations every single night while you sleep.

Instruction Based

Humans write program.md; AI iterates on train.py (models, optimizers, architecture).

Throughput Comparison

Source: github.com

Metrics: Exp/8hr Window +1,150% Increase

Shift: Code -> Instructions

System Architecture

The
Architecture:
3 Key Files

The human programs the strategy in program.md.

The agent programs the code in train.py.

File Name

Owner

Purpose & Scope

prepare.py

Fixed

Data prep, tokenizer, dataloader, eval utilities. Not modified by agent.

train.py

Agent

GPT model architecture, optimizer (Muon + AdamW), and full training loop logic.

program.md

Human

High-level strategy, research goals, and constraints for the AI agent.

5m

Per Experiment

∞

Autonomous Loops

"Every experiment is a data point.
Every run is directly comparable."

Autonomous Research Flow

THE
EXPERIMENT
LOOP

A closed-loop system where AI agents evolve code within fixed time constraints, ensuring every run is directly comparable.

01

Ingest Context

Agent reads program.md to understand the human researcher's constraints and goals.

02

Agentic Modification

The agent edits train.py, refactoring architecture, optimizers, or hyperparameters.

03

5-Minute Budget

Training runs for a strict wall-clock window. Normalizes performance metrics across all architecture changes.

04

Metric Validation

Evaluate val_bpb (bits per byte). Only lower scores advance the research lineage.

05

Version Control

Success triggers a git commit. Failure leads to immediate discarding of the branch.

06

Infinite Loop

Repeat indefinitely. Every run is a comparable data point in the autonomous search space.

The Autoresearch Framework

Core Design Principles

Manageable Scope

Single File Modification

Agent only modifies train.py (~630 lines). Keeps diffs reviewable and architectural scope tightly constrained.

Hardware Specific

Fixed 5-Min Budget

Experiments are directly comparable regardless of model size or batch changes. Optimizes for your specific GPU performance.

Simplicity First

Self-Contained Design

No distributed training or complex configs. One GPU, one file, and a single metric (val_bpb) to rule them all.

* Applied principles ensure that every experiment is a clean data point in the research loop.

Case Studies & Growth

Results &
Early Traction

Val Loss Reduction

1.0 0.97

Bits Per Byte (BPB) achieved via autonomous iteration.

Viral Adoption

23k+

Stars in just days. Leading the agentic research trend.

Optimization Lead: Agent-optimized smaller models consistently outperformed larger manual configurations.

19%

Tobi Lütke

CEO, Shopify

"Adapted the framework internally... reported a 19% improvement in validation scores using agentic iteration."

Source: github.com

Timeline of Automation

The Evolution
of Coding

01

Feb 2025

Vibe Coding

Human describes what to build in English. AI acts as a high-speed translator of intent to syntax.

Human Role

The Narrator

02

Feb 2026

Agentic Engineering

Human orchestrates multiple agents. Oversight and architectural guardrails are the primary focus.

Human Role

The Architect

03

Mar 2026

Autoresearch

Human writes a strategy Markdown file. Agent experiments autonomously and indefinitely.

Human Role

The Strategist

The Key Insight

The End of
Coding.

You're no longer writing code or even telling AI to write it. You are writing the research strategy document.

The best labs will be defined by the quality of their program.md, not just their compute.

program.md — Autoresearch

# Research Strategy v2.4

- Goal: Optimize validation loss

- Method: Iterative LR pruning

- Compute: 1x H100

Agent Running Experiments...

The New Unit of Output

Compute is a commodity. Strategy is the edge.

Paradigm Shift 2026 // Autoresearch

Future Roadmap

What's
Next

The New Paradigm

program.md

The program.md is evolving from a config file into the primary unit of researcher output. We are no longer optimizing weights; we are optimizing instructions for autonomous discovery.

Multi-Agent Swarms

Scaling from single agents to massive parallel experiment swarms running 24/7 without intervention.

8xH100 Clusters

Expansion to production-grade nanochat clusters, enabling research at the frontier of commercial compute scales.

Community Forks

Democratizing agentic research across platforms.

MLX (macOS)

RTX (Windows)

Edge GPU

FUTURE

Deployment

Quick Start
Guide

Deploy the autonomous research stack in minutes. Iterate on program.md using Claude Code.

NVIDIA GPU (H100)

Python 3.10+

uv manager

bash — autoresearch

The Future /
ML Research

The Human Baseline

1 Experiment

Per day. Limited by sleep, lab politics, and biological latency. High cost, high friction.

Autonomous Agent

100+ Experiments

Overnight. No breaks, no forgetting, no ego. Just raw search through the idea space.

Computing... 100x Efficiency

Autoresearch is the beginning of AI research eating itself.

Get the code

karpathy/autoresearch

Karpathy'sAutoresearch

The Big Idea

AgenticEngineering

Autopilot Research

High Velocity

Instruction Based

Throughput Comparison

The Architecture: 3 Key Files

Ingest Context

Agentic Modification

5-Minute Budget

Metric Validation

Version Control

Infinite Loop

Core Design Principles

Manageable Scope

Single File Modification

Hardware Specific

Fixed 5-Min Budget

Simplicity First

Self-Contained Design

Tobi Lütke

Feb 2025

Vibe Coding

Feb 2026

Agentic Engineering

Mar 2026

Autoresearch

The New Unit of Output

program.md

Multi-Agent Swarms

8xH100 Clusters

Community Forks

The Future /ML Research

1 Experiment

100+ Experiments

Karpathy's
Autoresearch

Agentic
Engineering

The
Architecture:
3 Key Files

The Future /
ML Research