AI Researcher

Sebastian Ruder Portfolio

Research scientist at Meta

NLP research scientist at Meta in Berlin, after leading the Multilingual team at Cohere and working at Google DeepMind; his blog ran from 2016 to 2024 with in-depth surveys on optimization, transfer learning, multilingual NLP, and LLM evaluation, year-end research-highlights roundups (2019-2021), and the companion NLP-Progress benchmark tracker.

NLPTransfer LearningMultilingual ModelsLLM Evaluation

Location

Berlin, Germany

Companies

MetaCohereGoogle DeepMindAYLIEN

Credentials

  • PhD in Natural Language Processing and Deep Learning, Insight Research Center for Data Analytics
  • Computational Linguistics, University of Heidelberg and Trinity College Dublin

Skills

NLPTransfer LearningMultilingual ModelsLLM Evaluation

Notable

  • Created NLP-Progress, a GitHub tracker of state-of-the-art results across NLP tasks (22.9k stars)
  • Lead author of XTREME-R, a multilingual evaluation benchmark (EMNLP 2021)
  • Wrote tutorials for NAACL 2019 (Transfer Learning in NLP), ACL 2019 (Unsupervised Cross-lingual Representation Learning), and EMNLP 2021 (Multi-domain Multilingual QA), each expanded into a blog post

What makes it work

A breakdown of the choices that make this portfolio stand out.

1

Annual synthesis posts that track a field's actual trajectory

The yearly "highlights" format forces a distillation of an entire year of conference output (ACL, EMNLP, NeurIPS) into a coherent narrative, which is harder and more valuable than reacting to individual papers as they appear.

2

A companion project that operationalizes the research

NLP-Progress, a community-edited benchmark tracker on GitHub (22.9k stars, last updated June 2024), turns research literacy into a structured, reusable public resource instead of leaving it locked in prose.

3

An eight-year archive with consistent depth

Posts from 2016 (word embeddings, gradient descent optimizers) through 2024 (instruction-tuning data, LLM evaluation) hold comparable technical rigor, tracking the field from embeddings to transfer learning to LLMs rather than stalling after an early peak.

What AI Researchers can take from this

Specific, actionable tips to apply to your own portfolio — no generic advice.

  • Write an annual synthesis of your field's research, not just reactions to individual papers — it is harder to do and more useful to readers.

  • Pair your writing with a structured, maintained resource (a benchmark tracker, a dataset, a leaderboard) that turns commentary into a public tool.

  • Keep your archive's rigor consistent across years so it reads as ongoing expertise, not a project you started and let go stale.

Projects

For years, the conference highlights and year-end roundups on Sebastian Ruder's blog did the reading of whole proceedings for everyone else, and NLP-Progress kept the scoreboard in a GitHub repository.

Open SourceadvancedMulti-year community project

NLP-Progress

MarkdownGitHubBenchmarks

A repository that tracks datasets and state-of-the-art results for common NLP tasks across English and other languages, one Markdown page per task, edited through community pull requests.

Turns wide reading of the literature into a structured public resource other researchers use.

  • Per-language folders (Arabic, Bengali, Chinese, French and more)

  • Open to outside contributions

  • Also served at nlpprogress.com

ResearchadvancedSingle long-form survey

An overview of gradient descent optimization algorithms

OptimizationDeep Learning

A January 2016 survey post explaining how Momentum, Adagrad, Adam and related optimizers actually work, for readers who otherwise treat them as a black box.

Shows the habit of turning a scattered literature into one careful reference.

  • One of the blog's longest posts

  • Covers the full optimizer family

  • Paired with a 2017 follow-up on optimization highlights

ResearchadvancedSingle long-form survey

The State of Multilingual AI

Multilingual NLPSpeechComputer Vision

A November 2022 survey of how multilingual current models are across NLP, computer vision, and speech, with recent contributions and open challenges.

Lines up with his research focus on multilingual models and evaluation.

  • Covers three modalities

  • Ends on open challenges

  • Follows his 2020 'Why You Should Do NLP Beyond English'

Ready to build your portfolio?

Follow the AI Researcher roadmap — skills, projects, and timeline to get hired.

AI Researcher Roadmap