AI Agent Architect

Shunyu Yao Portfolio

Researcher at OpenAI who authored ReAct (reasoning + acting, ICLR 2023) and Tree of Thoughts (NeurIPS 2023), two of the most widely adopted frameworks for how language-model agents plan and use tools; also co-created SWE-agent/SWE-bench and the tau-bench benchmark for evaluating tool-agent-user interactions.

Agent ReasoningTool UseReActBenchmarking

Location

Not listed — probably too busy shipping to update the bio.

Skills

Agent ReasoningTool UseReActBenchmarking

What makes it work

A breakdown of the choices that make this portfolio stand out.

1

A named reasoning pattern that became a default agent architecture

ReAct's interleaved "thought, action, observation" loop is now the default scaffold underneath most production agent frameworks, not just an academic curiosity — a rare case of a research paper's architecture becoming industry-standard implementation.

2

Benchmarks built alongside the architectures they test

WebShop, SWE-bench, and tau-bench each give the field a way to actually measure whether an agent design works, rather than judging it anecdotally — pairing new agent patterns with the yardstick to evaluate them is more useful than either alone.

3

A body of work that moves in step with agent capability itself

The progression from ReAct through Tree of Thoughts to Computer-Using Agent tracks agents evolving from single tool calls to multi-step deliberation to full computer control, showing sustained focus on one research question as it developed.

What AI Agent Architects can take from this

Specific, actionable tips to apply to your own portfolio — no generic advice.

  • If you design a reasoning or planning pattern for agents, name it clearly and explain the loop precisely — a well-named pattern is what gets adopted into other people's frameworks.

  • Build the benchmark alongside the architecture, not after — a way to measure whether your design actually works is as valuable as the design itself.

  • Let your public work show a single research question evolving over time rather than a scattered set of unrelated demos.

Projects

Four papers, each with a public repo: a reasoning pattern (ReAct), a search pattern (Tree of Thoughts) and two benchmarks for measuring agents like them. He shipped the method and the yardstick together.

ResearchadvancedarXiv Oct 2022; ICLR 2023

ReAct: Synergizing Reasoning and Acting in Language Models

Few-shot promptingPaLM-540BGPT-3Wikipedia APIHotpotQAFEVERALFWorldWebShop

A prompting method where the model interleaves reasoning traces with actions. The reasoning keeps track of and updates the plan, and the actions pull in outside information such as a Wikipedia API lookup. It was tested on question answering, fact checking and two interactive decision-making benchmarks.

The thought, action, observation loop from this paper is the basic structure of most tool-using agents. The repo's own README sends readers to LangChain's zero-shot ReAct agent for further use.

  • Interleaves 'thought' and 'action' steps so the plan updates after each observation

  • Grounding in a simple Wikipedia API reduces the hallucination and error propagation seen with chain-of-thought alone

  • +34 points on ALFWorld and +10 on WebShop over trained baselines, using 1–2 examples

ResearchadvancedarXiv May 2023; NeurIPS 2023

Tree of Thoughts: Deliberate Problem Solving with Large Language Models

GPT-4Tree search (BFS/DFS)Self-evaluation promptingPythonPyPI package

An inference framework that generalizes chain-of-thought. Instead of a single left-to-right pass, the model explores several reasoning paths made of coherent 'thoughts', scores its own choices, and can look ahead or backtrack. It was tested on three new planning-heavy tasks: Game of 24, Creative Writing and Mini Crosswords.

It treats LLM reasoning as a search problem with a value function, which is how planning agents are designed today. The repo ships every prompt and model output, so the headline result can be reproduced.

  • Game of 24: 4% → 74% success versus GPT-4 chain-of-thought

  • The model proposes candidate thoughts, then scores them itself to pick which branch to expand

  • Code, prompts and model outputs are all public; installable with pip

ResearchadvancedarXiv Jun 2024; ICLR 2025

τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains

LLM-simulated usersFunction callingDomain policy documentsDatabase-state evaluationPython

A benchmark in which a language agent talks with an LLM-simulated user while using domain API tools and following written policy guidelines. Grading compares the database state at the end of the conversation with an annotated goal state. It also introduces pass^k, a metric for how reliably an agent succeeds across repeated trials.

It measures what production teams actually need to know: whether an agent follows business rules consistently, not whether it succeeds once. The pass^k metric makes that inconsistency show up in the numbers.

  • Airline and retail domains, each with tools and a written policy

  • Outcome grading on final database state instead of the transcript

  • pass^k metric: gpt-4o's pass^8 in retail was below 25%

ResearchadvancedarXiv Jul 2022; NeurIPS 2022

WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents

Simulated e-commerce siteReinforcement learningImitation learningPythonFlask

A simulated shopping website with 1.18 million real products and 12,087 crowd-sourced instructions. From a text request, an agent has to search, move between pages, pick product options and buy the right item. The team collected over 1,600 human demonstrations to train and compare agents.

He built the environment before building agents for it, which is the unglamorous work that makes agent research measurable. The 29% vs 59% human gap gave the field a concrete target.

  • 1.18M real products, 12,087 instructions, 1,600+ human demos

  • Best agent 29% vs human experts 59%

  • Became a test environment for later agent papers, including ReAct

Ready to build your portfolio?

Follow the AI Agent Architect roadmap — skills, projects, and timeline to get hired.

AI Agent Architect Roadmap