Sebastian Ruder Portfolio
Research scientist at Meta
NLP research scientist at Meta in Berlin, after leading the Multilingual team at Cohere and working at Google DeepMind; his blog ran from 2016 to 2024 with in-depth surveys on optimization, transfer learning, multilingual NLP, and LLM evaluation, year-end research-highlights roundups (2019-2021), and the companion NLP-Progress benchmark tracker.
Location
Berlin, Germany
Companies
Credentials
- PhD in Natural Language Processing and Deep Learning, Insight Research Center for Data Analytics
- Computational Linguistics, University of Heidelberg and Trinity College Dublin
Skills
Notable
- Created NLP-Progress, a GitHub tracker of state-of-the-art results across NLP tasks (22.9k stars)
- Lead author of XTREME-R, a multilingual evaluation benchmark (EMNLP 2021)
- Wrote tutorials for NAACL 2019 (Transfer Learning in NLP), ACL 2019 (Unsupervised Cross-lingual Representation Learning), and EMNLP 2021 (Multi-domain Multilingual QA), each expanded into a blog post
What makes it work
A breakdown of the choices that make this portfolio stand out.
Annual synthesis posts that track a field's actual trajectory
The yearly "highlights" format forces a distillation of an entire year of conference output (ACL, EMNLP, NeurIPS) into a coherent narrative, which is harder and more valuable than reacting to individual papers as they appear.
A companion project that operationalizes the research
NLP-Progress, a community-edited benchmark tracker on GitHub (22.9k stars, last updated June 2024), turns research literacy into a structured, reusable public resource instead of leaving it locked in prose.
An eight-year archive with consistent depth
Posts from 2016 (word embeddings, gradient descent optimizers) through 2024 (instruction-tuning data, LLM evaluation) hold comparable technical rigor, tracking the field from embeddings to transfer learning to LLMs rather than stalling after an early peak.
What AI Researchers can take from this
Specific, actionable tips to apply to your own portfolio — no generic advice.
Write an annual synthesis of your field's research, not just reactions to individual papers — it is harder to do and more useful to readers.
Pair your writing with a structured, maintained resource (a benchmark tracker, a dataset, a leaderboard) that turns commentary into a public tool.
Keep your archive's rigor consistent across years so it reads as ongoing expertise, not a project you started and let go stale.
Projects
For years, the conference highlights and year-end roundups on Sebastian Ruder's blog did the reading of whole proceedings for everyone else, and NLP-Progress kept the scoreboard in a GitHub repository.
NLP-Progress
A repository that tracks datasets and state-of-the-art results for common NLP tasks across English and other languages, one Markdown page per task, edited through community pull requests.
Turns wide reading of the literature into a structured public resource other researchers use.
Per-language folders (Arabic, Bengali, Chinese, French and more)
Open to outside contributions
Also served at nlpprogress.com
An overview of gradient descent optimization algorithms
A January 2016 survey post explaining how Momentum, Adagrad, Adam and related optimizers actually work, for readers who otherwise treat them as a black box.
Shows the habit of turning a scattered literature into one careful reference.
One of the blog's longest posts
Covers the full optimizer family
Paired with a 2017 follow-up on optimization highlights
The State of Multilingual AI
A November 2022 survey of how multilingual current models are across NLP, computer vision, and speech, with recent contributions and open challenges.
Lines up with his research focus on multilingual models and evaluation.
Covers three modalities
Ends on open challenges
Follows his 2020 'Why You Should Do NLP Beyond English'
More AI Researcher portfolios
Ready to build your portfolio?
Follow the AI Researcher roadmap — skills, projects, and timeline to get hired.