My #1 Piece of Advice for Learning Data Engineering
Here's my number one piece of advice to anyone trying to learn Data Engineering or become a Data Engineer:
Build a data engineering project.
Building a data engineering project was the single most impactful thing I did in grad school. The project I built got me interviews. It gave me something to talk about in those interviews.
And it got me hired as a full time Data Engineer just a few weeks after I graduated.
Companies want to hire people who can hit the ground running on day one. Employers want to know: can you handle this job?
There's no better way to prove you've got exactly what it takes than to show them the proof. And a project does exactly that.
Here's what every good data engineering project should have:
1. Data ingestion from a source like an API
2. Landing data in a data lake/cloud storage
3. Transforming data through a data pipeline
4. Scheduling and workflow orchestration of the pipeline
5. Data quality checks
6. Visualizations that give meaning to the data
7. A public GitHub project page to showcase everything
Want to learn how to build a project like this?
I created a course to help people learn core Data Engineering skills by building a portfolio project. It’s called Build Your First Serverless Data Engineering Project, and enrollment is now open until April 7th.
It's a live cohort course to teach you everything you need to know to build a fully serverless data engineering project in AWS.
The course includes:
- Building a fully functional data engineering project from start to finish
- Live sessions to get help when you need it and stay motivated
- Lessons on major AWS services like Kinesis, Lambda, S3, Athena, and Glue
- Advanced data engineering SQL strategies
- Advice on showcasing your project on your resume and in interviews
I’ve made this course as interactive and hands-on as possible, so you get to do as you learn. By the end, you’ll have built a complete Data Engineering project you can showcase as part of a portfolio. You'll also expand your network with a solid group of peers.
In just 2 weeks, you’ll learn how to:
Build a fully functional data engineering project from start to finish
Explore a variety of AWS services for serverless data engineering
Showcase your new knowledge with a project portfolio
Here’s the schedule:
Live session dates (8:00 PM Eastern): 4/7, 4/9, 4/10, 4/14, 4/16
Optional office hours on Sunday 4/13
1-2 hours/week outside of class for additional project work
This will be my 4th time teaching the course. Students in my prior cohorts rated the course 4.8/5.0 stars on average! Here’s what past students have to say:
“Build Your First Serverless Data Engineering Project course with David was awesome. David's clear guidance on each sessions, guided time to have hands on experience and multiple resources to work on individual project at the end with feedback and recommendation. Definitely a great teacher, obviously! All in all, enjoyed the course and would definitely recommend.” - Alex, Front-End Developer, Software Test Engineer
“I am glad that I signed up for this course. All technical details and nuances were well explained. Further, this course touched upon real world use cases and practical examples and David responded to every query in detail with examples.” - Rushit, Analytics Lead
“I recently completed a course on leveraging serverless architecture using AWS, and it was an exceptional learning experience. I want to give special credit to David, our instructor, who was detail-oriented and always prompt in responding to any questions. This course not only taught me how to utilize serverless architecture with AWS but also enhanced my skills in visualizing and presenting my final project on GitHub. It's an excellent course for anyone looking to transition from on-premises solutions to the cloud or for data engineers interested in learning about serverless data pipelines. Highly recommended!” - Divya, Senior Data Engineer at Comcast
Interested? Enrollment is open until April 7th. Check out the course syllabus, and see what my past students have said about it here: