AI Agent Architect

Noah Shinn Portfolio

Researcher

Researcher (previously at Sierra and Northeastern University) who created Reflexion, a NeurIPS 2023 paper on giving language-model agents verbal self-reflection and reinforcement without updating model weights; also co-created tau-bench for evaluating tool-agent-user interactions in realistic multi-turn settings.

Agent MemorySelf-ReflectionReinforcement LearningBenchmarking

Location

Coordinates unknown. Vibes: international.

Companies

SierraNortheastern University

Credentials

  • Northeastern University

Skills

Agent MemorySelf-ReflectionReinforcement LearningBenchmarking

Notable

  • Reflexion: NeurIPS 2023 paper on agent self-reflection

What makes it work

A breakdown of the choices that make this portfolio stand out.

1

A named architectural pattern for agent self-improvement

Reflexion gives agents a specific mechanism — reflect on a failed attempt in natural language, store that reflection, and use it on the next attempt — turning "agents that learn from mistakes" from a vague goal into an implementable pattern.

2

Weight-free learning as a deliberate architectural constraint

Choosing to improve agent behavior through stored verbal feedback rather than fine-tuning or RL weight updates is a specific design decision, one that made the pattern usable with any off-the-shelf model rather than requiring custom training.

3

Small, runnable projects that test the underlying idea narrowly

Repositories that apply the reflection-and-memory idea to narrow, checkable tasks let the architecture be evaluated in isolation rather than only inside one large, hard-to-debug system.

What AI Agent Architects can take from this

Specific, actionable tips to apply to your own portfolio — no generic advice.

  • Give your agent's self-improvement mechanism a specific, describable form (what gets stored, when it's used) rather than describing it only as "the agent learns."

  • Consider whether your architecture needs to retrain a model at all — a well-designed memory or feedback mechanism can improve behavior without touching model weights.

  • Test a new architectural idea on small, narrow, checkable tasks before deploying it inside one large agent — isolating the mechanism makes it easier to verify it actually works.

Projects

Agent research with a builder's streak: a self-improvement method, a benchmark for tool-using agents that must follow real policies, and the gym that made hard-coding evaluation practical. His site also lists a dozen small tools for search, git and the terminal built along the way.

ResearchadvancedarXiv Mar 2023; NeurIPS 2023

Reflexion: Language Agents with Verbal Reinforcement Learning

GPT-4Episodic memoryHumanEvalHotpotQAALFWorldPython

A method for improving language agents without updating their weights. After a failed attempt, the agent writes a verbal reflection on the feedback, stores it in an episodic memory buffer, and reads it on the next try. It works with scalar or free-form feedback from outside sources or generated internally.

Self-critique stored in memory is now a standard agent pattern, and this paper named it and measured it. The released logs let others check the results.

  • Learning from trial and error without fine-tuning: reflections live in memory

  • HumanEval pass@1: 91% vs GPT-4's 80%

  • Tested across coding, reasoning (HotpotQA) and decision-making tasks

ResearchadvancedarXiv Jun 2024; ICLR 2025

τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains

LLM-simulated usersFunction callingPolicy documentsPython

A benchmark from Sierra, where he says he was among the first employees. An agent with domain API tools and a written policy handles a conversation with an LLM-simulated user, and success is graded by comparing the final database state with a goal state. It introduced pass^k to measure consistency over repeated trials.

He built it inside a company deploying customer-facing agents, and it tests what that setting demands: following rules reliably, not one-off wins.

  • Tool use + simulated user + domain policy in one benchmark

  • Graded on end state, not the transcript

  • pass^k reliability metric

ToolintermediateReleased with Reflexion (2023)

LeetcodeHardGym

PythonOpenAI GymLeetCode submission API

An RL-style environment that sends code-generation agents' solutions to LeetCode's own submission server, built on OpenAI Gym. It supports about fifteen languages, from C and Rust to Racket. A dataset script builds an uncontaminated, HumanEval-style set of free LeetCode Hard problems.

Evaluation infrastructure against a real external grader, with care taken over test-set contamination, which is the less visible work that agent benchmarks rely on.

  • env.step(submission) returns LeetCode's real verdict

  • Dataset builder filters out class-dependent and void problems

  • Multi-language: C, C#, Java, Python, Rust, Racket and more

Ready to build your portfolio?

Follow the AI Agent Architect roadmap — skills, projects, and timeline to get hired.

AI Agent Architect Roadmap