AI Researcher

Jonathan Lorraine Portfolio

Senior Research Scientist, NVIDIA Research (Learning and Perception Research lab)

Senior Research Scientist at NVIDIA Research (PhD, University of Toronto, advised by David Duvenaud), working on agentic AI for science and engineering with recent work spanning provenance in video generation (MOTIVE), real-time world models (OmniDreams), and multimodal generation (LLaMA-Mesh).

Generative AIWorld ModelsReward ModelingNVIDIA Research

Location

Toronto, Canada

Companies

NVIDIAGoogleFacebook AI Research

Credentials

  • PhD Computer Science, University of Toronto (2018–2024)
  • M.Sc. Applied Computing, University of Toronto (2016–2018)
  • H.B.Sc., University of Toronto (2012–2016)

Skills

Generative AIWorld ModelsReward ModelingNVIDIA Research

Notable

  • ICML 2026 Oral with Outstanding Paper Honorable Mention (MOTIVE)
  • Best Paper, ICML 2026 F2S Workshop (WorldTrace)
  • ICLR 2024 spotlight (Graph Metanetworks)

What makes it work

A breakdown of the choices that make this portfolio stand out.

1

A stated diagnosis of the field, not just a topic list

Framing the site around "what limits modern AI now is direction, not capability" gives a specific, arguable position on where the field's bottleneck actually is, rather than a neutral summary of interests.

2

Named projects spanning distinct modalities under one research bet

MOTIVE (video provenance), OmniDreams (world models), and LLaMA-Mesh (3D generation) are different outputs, listed together as recent work directly beneath one stated agenda (the rewards, verifiers and evaluation that let models improve themselves), so the modality changes while the framing stays fixed.

3

Academic teaching listed alongside industry research

Ten University of Toronto course offerings, from neural networks and probabilistic learning to NLP and imitation learning for robotics, taught as a (lead) teaching assistant per the CV, signal the research is grounded in fundamentals solid enough to teach, not just applied industry output.

What AI Researchers can take from this

Specific, actionable tips to apply to your own portfolio — no generic advice.

  • State a specific, arguable diagnosis of what actually limits your field right now — it is more memorable than a neutral list of research interests.

  • If your projects span different modalities or applications, name the single underlying bet that unifies them.

  • If you teach alongside your research, list the specific courses — it signals the work is grounded in fundamentals, not just the latest result.

Projects

Jonathan Lorraine's papers arrive with a kit: webpage, slides, poster, bibtex, and often a 1-minute video beside the 5-minute one, so each result is pre-cut for a reader with a minute or an afternoon. LATTE3D's row runs to thirteen links and ends at Two Minute Papers.

ResearchadvancedICML 2026 (Oral)

MOTIVE: Motion Attribution for Video Generation

Video diffusion modelsGradient-based data attributionMotion-weighted loss masksVBench

Which fine-tuning clips make a video model's motion better or worse? MOTIVE is a gradient-based data attribution framework built to scale to large video datasets and models. Motion-weighted loss masks separate temporal dynamics from static appearance, so the influence scores measure motion specifically. It's positioned as the first framework to attribute motion rather than appearance in video generative models, and to use that attribution to curate fine-tuning data.

It connects data curation directly to a measurable model behavior. Teams training generative models need exactly that, since the usual answer to bad motion is simply 'more data'.

  • 74.1% human-preference win rate vs. the pretrained base model

  • ICML 2026 Oral, Outstanding Paper Honorable Mention

  • Project page shows positive and negative influential clips per query motion

ResearchadvancedPre-print, 2024 (NVIDIA and Tsinghua University)

LLaMA-Mesh: Unifying 3D Mesh Generation with Language Models

LLaMASupervised fine-tuning3D mesh generationHugging Face TransformersBlender

Instead of building a new tokenizer for 3D data, LLaMA-Mesh writes a mesh's vertex coordinates and face definitions as plain text. That lets a pretrained LLM read and generate meshes without expanding its vocabulary. A supervised fine-tuning dataset teaches the model to generate meshes from text prompts, mix text and meshes in one reply, and interpret meshes it's given.

The central idea is simple and it works: represent the new modality in the format the model already knows. It shipped as working tools people can use, not just a PDF.

  • Meshes as plain text: no vocabulary expansion needed

  • Conversational 3D generation and mesh understanding in one model

  • 1.2k GitHub stars, public weights, demo and Blender add-on

ResearchadvancedICCV 2023

ATT3D: Amortized Text-to-3D Object Synthesis

Text-to-3DNeural radiance fieldsScore distillationAmortized optimization

DreamFusion-style text-to-3D produces good objects but runs a long optimization for every single prompt. ATT3D trains one model across many prompts at once, sharing computation across the prompt set. That takes less time than optimizing each prompt separately.

It applies an optimization idea from the PhD work, amortization, to a production-relevant bottleneck. The result is a line of work (ATT3D, then LATTE3D), not a one-off paper.

  • First-author ICCV 2023 paper

  • One model for many prompts instead of one optimization per prompt

  • Led to LATTE3D: text-to-mesh in 400ms

ResearchadvancedAISTATS 2020 (PhD, University of Toronto)

Optimizing Millions of Hyperparameters by Implicit Differentiation

Bilevel optimizationImplicit function theoremInverse Hessian approximationHyperparameter optimization

Gradient-based hyperparameter optimization, made cheap enough for modern networks. The implicit function theorem is combined with efficient inverse-Hessian approximations. The paper also works out how this relates to differentiating through the optimization steps directly. In one experiment, a whole data-augmentation network is treated as hyperparameters, with every weight tuned for validation performance.

Nested optimization of this kind sits behind the site's stated agenda of making reward and evaluation signals trainable at scale. It shows the current work rests on a mathematical foundation built over years.

  • Millions of hyperparameters tuned alongside millions of weights

  • Only a few times the cost of standard training

  • Core chapter of the 2024 PhD thesis advised by David Duvenaud

Ready to build your portfolio?

Follow the AI Researcher roadmap — skills, projects, and timeline to get hired.

AI Researcher Roadmap