Data Engineer

Joseph Machado Portfolio

Educational site by data engineer Joseph Machado, publishing hands-on tutorials and project write-ups on Airflow, dbt, Spark, and data-quality testing aimed specifically at practicing data engineers rather than generic "learn data engineering" content.

Apache AirflowdbtData PipelinesData Quality

Location

Remote-first, in the most literal sense — no idea where.

Skills

Apache AirflowdbtData PipelinesData Quality

Notable

  • Says 15+ years building distributed data platforms that processed multiple exabytes of data (per the About page)
  • beginner_de_project on GitHub: 595 stars, 206 forks
  • Author of the book Efficient Data Processing in SQL (Gumroad, $39)
  • Site says over 20,000 engineers have used its guides

What makes it work

A breakdown of the choices that make this portfolio stand out.

1

Tutorials built around runnable, complete pipeline projects

Posts like the Airflow 3.0 tutorial and fact/dimension table design walk through a full working pipeline rather than isolated code snippets, so a reader can actually reproduce the pattern in their own stack.

2

Explicit interview-preparation content alongside technical tutorials

Running a three-part Data Engineering Interview Preparation Series (data structures and algorithms, system design, SQL) alongside the hands-on project write-ups acknowledges that a data engineering portfolio needs to serve both learning and job-seeking, and structures the site around both.

3

Consistent focus on tools actually used in production data teams

The archive keeps circling back to dbt, Airflow, Spark, Iceberg, SQL and Python — dbt and Airflow tutorials refreshed in 2026, Spark-and-Iceberg posts on MERGE INTO and the small-files problem — rather than covering every tool in the ecosystem shallowly.

What Data Engineers can take from this

Specific, actionable tips to apply to your own portfolio — no generic advice.

  • Write tutorials around a complete, runnable pipeline rather than isolated code snippets — readers learn more from something they can reproduce end to end.

  • If you're building a portfolio site, pair technical project write-ups with practical interview-prep content; both serve a job-seeking reader.

  • Go deep on the small number of tools your target audience actually uses in production instead of covering the whole ecosystem shallowly.

Case studies

Almost every tutorial in Joseph Machado's archive comes with a repo to clone and a single command to start it. The free DE101 book's code has 940 forks against 132 stars, which suggests readers are doing the exercises rather than bookmarking them.

Case StudybeginnerSelf-paced, 14 chapters

Data Engineering for Beginners (DE101)

SQLPythondbt-coreApache AirflowDockerJupyter

A free online book that walks from SQL joins, CTEs and window functions through Python for pipelines, Kimball fact/dimension modeling and the multi-hop architecture, to Docker, dbt-core and Airflow, ending in a capstone project. Code for every chapter lives in a separate GitHub repo.

The chapter order mirrors how a junior data engineer actually gets productive: transform with SQL, glue with Python, model for analytics, then schedule and orchestrate.

  • Chapters are titled as claims ("Most companies use the multi-hop architecture") rather than topics

  • Includes a "To LLMs or Not" section on how to use the book alongside AI tools

  • Capstone project closes the book

Projects

Open Sourcebeginner1–2 weekends

Beginner DE Project — Batch Edition

Apache AirflowApache SparkDuckDBMinIODocker ComposeMakeGitHub Codespaces

The companion repo to the site's beginner data engineering project post: a containerized batch pipeline orchestrated by Airflow, with Spark and a MinIO-backed local object store. It runs either in GitHub Codespaces or locally with `make up`, and the Airflow UI comes up on port 8080.

It shows the unglamorous parts a hiring manager looks for beyond a DAG that runs once: a reproducible Docker environment, a Makefile entry point, and a `make ci` step that runs checks and tests against the DAGs.

  • One-command start (`make up`), then `make ci` to run checks and tests

  • Runs in GitHub Codespaces as well as locally with Docker and 4GB of RAM

  • A tests/dags folder ships with the pipeline, not as an afterthought

ResearchintermediateSelf-paced book

Efficient Data Processing in SQL

SQLOLAP enginesData modeling

A paid book on how analytical databases store and process data — distributed storage and processing, analytical (window) functions, query optimization, partitioning and data modeling — aimed at engineers who can write SQL but want to write it efficiently.

Explaining why a query is slow (how an OLAP engine lays out and scans data) is the SQL skill interviewers probe beyond syntax.

  • Frames SQL proficiency around engine internals, not syntax drills

  • Covers partitioning, parallelism and trade-offs between approaches

Ready to build your portfolio?

Follow the Data Engineer roadmap — skills, projects, and timeline to get hired.

Data Engineer Roadmap