Jonathan Lorraine Portfolio
Senior Research Scientist, NVIDIA Research (Learning and Perception Research lab)
Senior Research Scientist at NVIDIA Research (PhD, University of Toronto, advised by David Duvenaud), working on agentic AI for science and engineering with recent work spanning provenance in video generation (MOTIVE), real-time world models (OmniDreams), and multimodal generation (LLaMA-Mesh).
Location
Toronto, Canada
Companies
Credentials
- PhD Computer Science, University of Toronto (2018–2024)
- M.Sc. Applied Computing, University of Toronto (2016–2018)
- H.B.Sc., University of Toronto (2012–2016)
Skills
Notable
- ICML 2026 Oral with Outstanding Paper Honorable Mention (MOTIVE)
- Best Paper, ICML 2026 F2S Workshop (WorldTrace)
- ICLR 2024 spotlight (Graph Metanetworks)
What makes it work
A breakdown of the choices that make this portfolio stand out.
A stated diagnosis of the field, not just a topic list
Framing the site around "what limits modern AI now is direction, not capability" gives a specific, arguable position on where the field's bottleneck actually is, rather than a neutral summary of interests.
Named projects spanning distinct modalities under one research bet
MOTIVE (video provenance), OmniDreams (world models), and LLaMA-Mesh (3D generation) are different outputs, listed together as recent work directly beneath one stated agenda (the rewards, verifiers and evaluation that let models improve themselves), so the modality changes while the framing stays fixed.
Academic teaching listed alongside industry research
Ten University of Toronto course offerings, from neural networks and probabilistic learning to NLP and imitation learning for robotics, taught as a (lead) teaching assistant per the CV, signal the research is grounded in fundamentals solid enough to teach, not just applied industry output.
What AI Researchers can take from this
Specific, actionable tips to apply to your own portfolio — no generic advice.
State a specific, arguable diagnosis of what actually limits your field right now — it is more memorable than a neutral list of research interests.
If your projects span different modalities or applications, name the single underlying bet that unifies them.
If you teach alongside your research, list the specific courses — it signals the work is grounded in fundamentals, not just the latest result.
Projects
Jonathan Lorraine's papers arrive with a kit: webpage, slides, poster, bibtex, and often a 1-minute video beside the 5-minute one, so each result is pre-cut for a reader with a minute or an afternoon. LATTE3D's row runs to thirteen links and ends at Two Minute Papers.
MOTIVE: Motion Attribution for Video Generation
Which fine-tuning clips make a video model's motion better or worse? MOTIVE is a gradient-based data attribution framework built to scale to large video datasets and models. Motion-weighted loss masks separate temporal dynamics from static appearance, so the influence scores measure motion specifically. It's positioned as the first framework to attribute motion rather than appearance in video generative models, and to use that attribution to curate fine-tuning data.
It connects data curation directly to a measurable model behavior. Teams training generative models need exactly that, since the usual answer to bad motion is simply 'more data'.
74.1% human-preference win rate vs. the pretrained base model
ICML 2026 Oral, Outstanding Paper Honorable Mention
Project page shows positive and negative influential clips per query motion
LLaMA-Mesh: Unifying 3D Mesh Generation with Language Models
Instead of building a new tokenizer for 3D data, LLaMA-Mesh writes a mesh's vertex coordinates and face definitions as plain text. That lets a pretrained LLM read and generate meshes without expanding its vocabulary. A supervised fine-tuning dataset teaches the model to generate meshes from text prompts, mix text and meshes in one reply, and interpret meshes it's given.
The central idea is simple and it works: represent the new modality in the format the model already knows. It shipped as working tools people can use, not just a PDF.
Meshes as plain text: no vocabulary expansion needed
Conversational 3D generation and mesh understanding in one model
1.2k GitHub stars, public weights, demo and Blender add-on
ATT3D: Amortized Text-to-3D Object Synthesis
DreamFusion-style text-to-3D produces good objects but runs a long optimization for every single prompt. ATT3D trains one model across many prompts at once, sharing computation across the prompt set. That takes less time than optimizing each prompt separately.
It applies an optimization idea from the PhD work, amortization, to a production-relevant bottleneck. The result is a line of work (ATT3D, then LATTE3D), not a one-off paper.
First-author ICCV 2023 paper
One model for many prompts instead of one optimization per prompt
Led to LATTE3D: text-to-mesh in 400ms
Optimizing Millions of Hyperparameters by Implicit Differentiation
Gradient-based hyperparameter optimization, made cheap enough for modern networks. The implicit function theorem is combined with efficient inverse-Hessian approximations. The paper also works out how this relates to differentiating through the optimization steps directly. In one experiment, a whole data-augmentation network is treated as hyperparameters, with every weight tuned for validation performance.
Nested optimization of this kind sits behind the site's stated agenda of making reward and evaluation signals trainable at scale. It shows the current work rests on a mathematical foundation built over years.
Millions of hyperparameters tuned alongside millions of weights
Only a few times the cost of standard training
Core chapter of the 2024 PhD thesis advised by David Duvenaud
More AI Researcher portfolios
Ready to build your portfolio?
Follow the AI Researcher roadmap — skills, projects, and timeline to get hired.