John Yang Portfolio
PhD student at Stanford
PhD student at Stanford (work began at Princeton); created SWE-bench and SWE-agent, which established "agent-computer interfaces" — a structured set of tools (file viewer, search, edit commands) purpose-built for how a language-model agent, not a human, interacts with a codebase.
Location
Somewhere on planet Earth, most likely near a laptop.
Companies
Credentials
- Princeton University
- Stanford University
Skills
Notable
- SWE-bench: foundational benchmark for code agents
- SWE-agent: agent-computer interface design
What makes it work
A breakdown of the choices that make this portfolio stand out.
A named interface layer designed specifically for agents, not humans
SWE-agent's "agent-computer interface" concept treats the tools an agent uses (not just its reasoning) as a first-class design problem, recognizing that interfaces built for human developers are often the wrong shape for a language model to use reliably.
A benchmark built to resist gaming, then hardened further
SWE-bench Verified and the multimodal/multilingual variants show iterating on a benchmark's rigor over time as agents got better at exploiting the original version's weaknesses, rather than treating the first release as final.
A tight suite of complementary tools around one research question
InterCode, SWE-bench, SWE-agent, and SWE-smith each address a different piece of the same problem — how coding agents should be evaluated and interfaced with — rather than being unrelated one-off projects.
What AI Agent Architects can take from this
Specific, actionable tips to apply to your own portfolio — no generic advice.
Treat the interface between your agent and its environment as its own design problem, separate from the agent's reasoning — the wrong tool shape can break an otherwise good agent.
If you publish a benchmark, expect it to get gamed, and plan to release harder or corrected versions rather than treating v1 as permanent.
Build a small suite of complementary tools around one specific research question instead of one-off unrelated projects — it makes your body of work read as a coherent research program.
Projects
Almost everything here is about measuring and training coding agents: a benchmark from real GitHub issues, an agent built for it, and a pipeline that produces training data for the next agent. Each is released as data, code and a paper together.
SWE-bench: Can Language Models Resolve Real-World GitHub Issues?
An evaluation set of 2,294 real software-engineering problems taken from GitHub issues and their merged pull requests across 12 popular Python repositories. The model gets a codebase and an issue and must edit the code so the repository's tests pass, which often means coordinated changes across several files.
It set the standard way to measure coding agents: real repositories, real tests, no multiple choice. Keeping a benchmark useful as models improved shows lasting ownership.
2,294 task instances from 12 Python repos
Pass/fail graded by the repository's own tests
Best model at launch: 1.96% resolved
SWE-agent
An agent that takes a GitHub issue and tries to fix it on its own. Its key idea is the agent-computer interface (ACI): file viewing, search, editing and test commands designed for how a language model works rather than a human, and the paper shows that interface design alone changes agent performance.
It showed that the tool layer the model sees matters as much as the model, and then the team replaced its own system with a smaller one when the evidence supported it.
Agent-computer interface: tools designed for LMs, not humans
12.5% on SWE-bench and 87.7% on HumanEvalFix at publication
Superseded by mini-swe-agent: 65% on Verified in ~100 lines
SWE-smith: Scaling Data for Software Engineering Agents
A pipeline that generates hundreds to thousands of software-engineering task instances for any GitHub repository by creating bugs that can be checked against tests. The team used it to build a training set and fine-tune their own model for SWE-agent.
Having built the benchmark, he then built the data supply to train agents for it, a path from evaluation to training that few researchers complete.
Generates task instances for any GitHub repo, with a tutorial claiming 100 in 10 minutes
50k+ instances across 128 repos
Trained SWE-agent-LM-32B: 40% on SWE-bench Verified
More AI Agent Architect portfolios
Ready to build your portfolio?
Follow the AI Agent Architect roadmap — skills, projects, and timeline to get hired.