Greg Kamradt Portfolio
President, ARC Prize Foundation
President and board member of the ARC Prize Foundation. Created the "Needle in a Haystack" long-context test that Google's Gemini 1.5 technical report modeled its recall evals on. Also runs the @DataIndependent YouTube channel (69.9K subscribers) and ships small agent tools (InterAuth, agnts.sh, RunReq) on a projects page listing 11 of them.
Location
Not listed — probably too busy shipping to update the bio.
Companies
Skills
Notable
- President and Board Member, ARC Prize Foundation (listed on arcprize.org)
- Created the Needle In A Haystack long-context test, cited as the model for recall tests in Google's Gemini 1.5 technical report
- Speaker, OpenAI DevDay (Oct 2025): "Measuring Agents With Interactive Evaluations"
- YouTube channel @DataIndependent: 69.9K subscribers, 206 videos
What makes it work
A breakdown of the choices that make this portfolio stand out.
Created a benchmark that became an industry reference point
Google's Gemini 1.5 technical report built its needle-in-a-haystack recall tests on Kamradt (2023), and the repo has 2.4k GitHub stars and now ships as a v2 pip package. An evaluation method others adopt is a durable technical contribution beyond any single product.
A portfolio of small, shipped tools rather than one flagship product
Listing multiple distinct, named tools (InterAuth, agnts.sh, RunReq) shows range and shipping velocity across the agent-tooling space rather than betting everything on one product.
Speaking engagements tied to a specific technical contribution
Presenting at OpenAI DevDay and MIT specifically on agent evaluation methods ties his speaking history to the concrete technical work he's known for, not general AI commentary.
What AI Engineers can take from this
Specific, actionable tips to apply to your own portfolio — no generic advice.
If you design an evaluation method or benchmark, publish and name it clearly — a well-designed benchmark can become a lasting reference point others cite.
Ship several small, named tools rather than waiting for one big flagship product — a portfolio of shipped work demonstrates range.
Seek speaking opportunities tied to your specific technical contribution, not generic AI commentary — it reinforces exactly what you're known for.
Projects
Greg Kamradt runs a benchmark foundation and dates his side projects like a changelog: eleven of them, from bronze-cast mountain maps to a link service for agents. The benchmark that made his name sits in the same list, one line long.
A pressure test for LLM long-context retrieval. A fact (the "needle") is hidden at varying depths in contexts of varying length, and the test checks whether the model can recall it. The v2 package runs a sweep of context-length by needle-depth cells, scores each response, and writes one JSONL row per cell. Built-in tasks cover single-fact lookup, multi-fact recall, UUID retrieval and multi-hop UUID chains.
It is a simple, visual, reproducible eval that the industry adopted. Designing tests that clearly expose a model weakness is exactly what AI evaluation roles need.
Sweeps context length against needle depth to map where recall breaks down
A --fake demo mode runs with no API key, which proves the install in about a second
Provider plugins for OpenAI, Anthropic and Cohere, extensible to others
InterAuth
A pass-through broker that gives an AI agent a disposable, read-only share address for a Google Doc: no OAuth, no install, no signup. You share the Doc with the address like you would with a colleague, and revoke it from Drive anytime. Access mirrors the share, so Viewer means read and Editor lets the agent edit a Sheet.
It takes on a real agent problem, scoped and revocable data access, by reusing a permission model users already understand instead of building a new one.
Scoped to exactly what you share; revoke from Google Drive like any collaborator
No OAuth consent flow or signup
Works with any agent that can make HTTP requests
agnts.sh
A link and message-board service built for agents. Every URL serves three response shapes from the same endpoint, based on the Accept header: an HTML stub for humans, raw markdown for agents, and full JSON. Boards let agents post, reply, search and check for new notes, with optional passwords and expiration.
It shows API design for agents as first-class clients: content negotiation and plain-text-first responses instead of making agents scrape HTML.
One URL, three representations (HTML, markdown, JSON) chosen by the Accept header
Shared boards for agent-to-agent notes, no signup
Ships its own agent-readable skill guide
More AI Engineer portfolios
Ready to build your portfolio?
Follow the AI Engineer roadmap — skills, projects, and timeline to get hired.