Books › AI Infrastructure Engineer roadmap
Best books for AI Infrastructure Engineers
The systems underneath AI. The opening books cover Linux internals, container orchestration and a deep-learning framework. The build set covers GPU programming, high-performance computing, system performance, ML system design and production Kubernetes. The lead set covers computer architecture, foundation-model engineering and senior technical influence.
11 books in 3 stages, about 6,160 pages (154 hours of reading). The notes are ours, with no ratings or sales claims.
How to choose your books
A short method, so you read the right book for where you are instead of the longest list.
Find your stage
Getting started · 3 books
The base layers: Linux internals, running workloads on Kubernetes and the framework that runs on the hardware.
Jump to this stage ↓Doing the job · 5 books
Performance and scale: GPU programming, high-performance computing, system performance, ML system design and production Kubernetes.
Jump to this stage ↓Senior and leading · 3 books
Hardware to organisation: computer architecture, foundation-model engineering and technical leadership.
Jump to this stage ↓
If you are unsure, start one stage lower than you think. Books in a later stage assume the working knowledge the earlier ones give you.
Begin with one book
Our starting pick
How Linux Works
by Brian Ward · 2004
Training and serving jobs run on Linux machines. Understanding the operating system makes every performance problem easier to locate.
See the book ↓Then pick by your situation
Choose the next book by the problem in front of you, not by its place in the list.
If you need to run workloads on GPUs and understand why they are slow
read Programming Massively Parallel Processors (doing the job)
If a job is slow and you do not know where the time goes
read Systems Performance (doing the job)
If you are running Kubernetes clusters for production workloads
read Production Kubernetes (doing the job)
Check it against what AI Infrastructure Engineer postings ask for
These skills come from our analysis of live AI Infrastructure Engineer job postings. If one is new to you, its book list sits one click away.
Each step on the scale reflects how often live job postings name the skill, tool or credential, based on our analysis of those postings. With fewer than 100 postings behind this page, treat the scale as directional: a sample this size can shift a skill by a step or more.
Set a time budget
| Stage | Books | Pages | Reading | At 3 hrs a week |
|---|---|---|---|---|
| Getting started | 3 | 1,240 | 31 hrs | 10 wks |
| Doing the job | 5 | 3,110 | 78 hrs | 26 wks |
| Senior and leading | 3 | 1,810 | 45 hrs | 15 wks |
| All 3 stages | 11 | 6,160 | 154 hrs | 51 wks |
Treat this as a ceiling, not a target. Read one book at a time, apply it, then move on; read reference-style books in sections.
Judge any book before you buy it
Age
This list runs from 1990 (Computer Architecture) to 2024 (AI Engineering). Older books earn their place by teaching principles that do not change. For tools and platforms, buy the latest edition.
Level
Check the stage before the title. A book from a later stage will make sense only once you have faced the problems it solves.
Format
The longest book here, Computer Architecture, runs about 940 pages. Read very long books as references: pick the chapter that matches the problem you have.
Fit
Ask what you will do differently the week after you finish. If you cannot answer, the book is for a different moment. Come back to it when the situation arrives.
Getting started
The base layers: Linux internals, running workloads on Kubernetes and the framework that runs on the hardware.
Start hereHow Linux Works
by Brian Ward · 2004
~390 pages~10 hrs to read
Explains the kernel, devices, processes, storage and networking underneath every GPU node. A foundation for debugging driver, boot and networking problems on cluster machines.
View book →
Kubernetes: Up and Running
by Brendan Burns, Joe Beda, Kelsey Hightower, Lachlan Evenson · 2017
~330 pages~8 hrs to read
Covers pods, services, deployments, config and storage. Read it first; most GPU scheduling for training and inference sits on Kubernetes.
View book →
Deep Learning with PyTorch
by Eli Stevens, Luca Antiga, Thomas Viehmann · 2020
~520 pages~13 hrs to read
Walks through tensors, training loops and model building in PyTorch. Infrastructure engineers should read it to understand what the workloads they host actually do.
View book →
Doing the job
Performance and scale: GPU programming, high-performance computing, system performance, ML system design and production Kubernetes.
Programming Massively Parallel Processors
by David B. Kirk, Wen-mei W. Hwu · 2010
~580 pages~15 hrs to read
Teaches GPU architecture and CUDA programming through memory hierarchy, parallel patterns and performance tuning. Read it to reason about why a job is bound by compute, memory or interconnect.
View book →
Parallel and High Performance Computing
by Robert Robey, Yuliana Zamora · 2021
~700 pages~18 hrs to read
Covers MPI, OpenMP, vectorization and GPU programming along with how to plan and measure parallel work. Relevant to cluster engineers who support distributed jobs.
View book →
Systems Performance
by Brendan Gregg · 2013
~930 pages~23 hrs to read
A methodical guide to analysing CPU, memory, disk and network performance on Linux, with tracing tools. Read it when a cluster underperforms and you have to find the bottleneck.
View book →
Designing Machine Learning Systems
by Chip Huyen · 2022
~390 pages~10 hrs to read
Describes the full lifecycle of production ML: data, training, deployment and monitoring. Helps infrastructure engineers understand what their users need from the platform.
View book →
Production Kubernetes
by Josh Rosso, Rich Lander, Alex Brand, John Harris · 2021
~510 pages~13 hrs to read
Covers the decisions around running Kubernetes at scale: networking, storage, security and multi-cluster operation. Read it when you move from using a cluster to building one.
View book →
Senior and leading
Hardware to organisation: computer architecture, foundation-model engineering and technical leadership.
Computer Architecture: A Quantitative Approach
by John L. Hennessy, David A. Patterson · 1990
~940 pages~24 hrs to read
The standard text on processor, memory and parallel hardware design, including domain-specific accelerators. Read it when you make hardware and capacity decisions.
View book →
AI Engineering: Building Applications with Foundation Models
by Chip Huyen · 2024
~350 pages~9 hrs to read
Covers how foundation-model applications are built, evaluated and served, including inference cost and latency. Useful for planning the capacity those workloads need.
View book →
The Staff Engineer's Path
by Tanya Reilly · 2022
~520 pages~13 hrs to read
Covers technical leadership without a management title: scoping work, influencing across teams and guiding large projects. Read it as your role expands from operating a cluster to setting direction.
View book →
Book links go to Amazon and are affiliate links. As an Amazon Associate I earn from qualifying purchases.
Frequently asked questions
Which book should a AI Infrastructure Engineer read first?
How Linux Works by Brian Ward. Training and serving jobs run on Linux machines. Understanding the operating system makes every performance problem easier to locate.
Do I need to read all 11 books?
No. The list is a menu, not a syllabus. Find your stage (getting started: 3, doing the job: 5, senior and leading: 3), begin with one book (we suggest How Linux Works), then pick the next book by the problem you are facing at work. Come back to the others when the situation they cover arrives.
How long does it take to read these books?
All 11 books add up to about 6,160 pages, roughly 154 hours at 1.5 minutes a page. At three hours a week that is about 51 weeks; at six hours a week, about 26. The list is meant to be read over time, one book at a time, not in one run.
Are these books up to date?
They range from 1990 (Computer Architecture) to 2024 (AI Engineering). Older titles are included because they teach principles that have not changed; for tools and platforms, we favour recent editions. Check the edition before you buy.
Related reading
Some links are affiliate links: we earn a small commission if you buy, at no cost to you. It does not change what we list.