https://producthunt.programnotes.cn/en/tags/data-engineer/ Recent content in Data Engineer on Producthunt daily Hugo -- gohugo.io en Thu, 21 Aug 2025 15:31:15 +0800 https://producthunt.programnotes.cn/en/p/data-engineer-handbook/ Thu, 21 Aug 2025 15:31:15 +0800 https://producthunt.programnotes.cn/en/p/data-engineer-handbook/ <img src="https://images.unsplash.com/photo-1648730665635-4f2b670cd921?ixid=M3w0NjAwMjJ8MHwxfHJhbmRvbXx8fHx8fHx8fDE3NTU3NjE0MDF8&ixlib=rb-4.1.0" alt="Featured image of post data-engineer-handbook" /><h1 id="dataexpert-iodata-engineer-handbook"><a class="link" href="https://github.com/DataExpert-io/data-engineer-handbook" target="_blank" rel="noopener" >DataExpert-io/data-engineer-handbook</a> </h1><h1 id="the-data-engineering-handbook">The Data Engineering Handbook </h1><p><a href="https://trendshift.io/repositories/8755" target="_blank"><img src="https://trendshift.io/api/badge/repositories/8755" alt="DataExpert-io%2Fdata-engineer-handbook | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a></p> <p>This repo has all the resources you need to become an amazing data engineer!</p> <h2 id="getting-started">Getting started </h2><p>If you are new to data engineering, start by following this <a class="link" href="https://blog.dataengineer.io/p/the-2024-breaking-into-data-engineering" target="_blank" rel="noopener" >2024 breaking into data engineering roadmap</a></p> <p>If you are here for the <a class="link" href="https://learn.dataexpert.io/program/the-absolute-beginner-data-engineering-boot-camp-starting-august-7th-6453/details" target="_blank" rel="noopener" >4-week free beginner boot camp</a> you can check out:</p> <ul> <li><a class="link" href="beginner-bootcamp/introduction.md" >introduction</a></li> <li><a class="link" href="beginner-bootcamp/software.md" >software needed</a></li> </ul> <p>If you are here for the <a class="link" href="https://learn.dataexpert.io/program/free-community-boot-camp/details" target="_blank" rel="noopener" >6-week free intermediate boot camp</a> you can check out</p> <ul> <li><a class="link" href="intermediate-bootcamp/introduction.md" >introduction</a></li> <li><a class="link" href="intermediate-bootcamp/software.md" >software needed</a></li> </ul> <p>For more applied learning:</p> <ul> <li>Check out the <a class="link" href="projects.md" >projects</a> section for more hands-on examples!</li> <li>Check out the <a class="link" href="interviews.md" >interviews</a> section for more advice on how to pass data engineering interviews!</li> <li>Check out the <a class="link" href="books.md" >books</a> section for a list of high quality data engineering books</li> <li>Check out the <a class="link" href="communities.md" >communities</a> section for a list of high quality data engineering communities to join</li> <li>Check out the <a class="link" href="newsletters.md" >newsletter</a> section to learn via email</li> </ul> <h2 id="resources">Resources </h2><h3 id="great-list-of-over-25-books">Great <a class="link" href="books.md" >list of over 25 books</a> </h3><p>Top 3 must read books are:</p> <ul> <li><a class="link" href="https://www.amazon.com/Fundamentals-Data-Engineering-Robust-Systems/dp/1098108302/" target="_blank" rel="noopener" >Fundamentals of Data Engineering</a></li> <li><a class="link" href="https://www.amazon.com/Designing-Data-Intensive-Applications-Reliable-Maintainable/dp/1449373321/" target="_blank" rel="noopener" >Designing Data-Intensive Applications</a></li> <li><a class="link" href="https://www.amazon.com/Designing-Machine-Learning-Systems-Production-Ready/dp/1098107969" target="_blank" rel="noopener" >Designing Machine Learning Systems</a></li> </ul> <h3 id="great-list-of-over-10-communities-to-join">Great <a class="link" href="communities.md" >list of over 10 communities to join</a>: </h3><p>Top must-join communities for DE:</p> <ul> <li><a class="link" href="https://discord.gg/JGumAXncAK" target="_blank" rel="noopener" >DataExpert.io Community Discord</a></li> <li><a class="link" href="https://datatalks.club/slack" target="_blank" rel="noopener" >Data Talks Club Slack</a></li> <li><a class="link" href="https://www.dataengineerthings.org/" target="_blank" rel="noopener" >Data Engineer Things Community</a></li> </ul> <p>Top must-join communities for ML:</p> <ul> <li><a class="link" href="https://discord.com/invite/ezzszrRZvT" target="_blank" rel="noopener" >AdalFlow Discord</a></li> <li><a class="link" href="https://discord.gg/dzh728c5t3" target="_blank" rel="noopener" >Chip Huyen MLOps Discord</a></li> </ul> <h3 id="companies">Companies: </h3><ul> <li>Orchestration <ul> <li><a class="link" href="https://www.mage.ai" target="_blank" rel="noopener" >Mage</a></li> <li><a class="link" href="https://www.astronomer.io" target="_blank" rel="noopener" >Astronomer</a></li> <li><a class="link" href="https://www.prefect.io" target="_blank" rel="noopener" >Prefect</a></li> <li><a class="link" href="https://www.dagster.io" target="_blank" rel="noopener" >Dagster</a></li> <li><a class="link" href="https://airflow.apache.org/" target="_blank" rel="noopener" >Airflow</a></li> <li><a class="link" href="https://kestra.io/" target="_blank" rel="noopener" >Kestra</a></li> <li><a class="link" href="https://www.shipyardapp.com/" target="_blank" rel="noopener" >Shipyard</a></li> <li><a class="link" href="https://github.com/dagworks-inc/hamilton" target="_blank" rel="noopener" >Hamilton</a></li> </ul> </li> <li>Data Lake / Cloud <ul> <li><a class="link" href="https://www.tabular.io" target="_blank" rel="noopener" >Tabular</a></li> <li><a class="link" href="https://www.microsoft.com" target="_blank" rel="noopener" >Microsoft</a></li> <li><a class="link" href="https://www.databricks.com/company/about-us" target="_blank" rel="noopener" >Databricks</a></li> <li><a class="link" href="https://www.onehouse.ai" target="_blank" rel="noopener" >Onehouse</a></li> <li><a class="link" href="https://delta.io/" target="_blank" rel="noopener" >Delta Lake</a></li> <li><a class="link" href="https://ilum.cloud/" target="_blank" rel="noopener" >Ilum</a></li> <li><a class="link" href="https://ducklake.select/" target="_blank" rel="noopener" >DuckLake</a></li> <li><a class="link" href="https://iceberg.apache.org/" target="_blank" rel="noopener" >Apache Iceberg</a></li> <li><a class="link" href="https://polaris.apache.org/" target="_blank" rel="noopener" >Apache Polaris</a></li> <li><a class="link" href="https://lakekeeper.io/" target="_blank" rel="noopener" >Lakekeeper</a></li> </ul> </li> <li>Data Warehouse <ul> <li><a class="link" href="https://www.snowflake.com/en/" target="_blank" rel="noopener" >Snowflake</a></li> <li><a class="link" href="https://www.firebolt.io/" target="_blank" rel="noopener" >Firebolt</a></li> <li><a class="link" href="https://www.databend.com/" target="_blank" rel="noopener" >Databend</a></li> </ul> </li> <li>Data Quality <ul> <li><a class="link" href="https://www.getdbt.com/" target="_blank" rel="noopener" >dbt</a></li> <li><a class="link" href="https://www.metaplane.dev/" target="_blank" rel="noopener" >Metaplane</a></li> <li><a class="link" href="https://www.gable.ai" target="_blank" rel="noopener" >Gable</a></li> <li><a class="link" href="https://www.greatexpectations.io" target="_blank" rel="noopener" >Great Expectations</a></li> <li><a class="link" href="https://streamdal.com" target="_blank" rel="noopener" >Streamdal</a></li> <li><a class="link" href="https://coalesce.io/" target="_blank" rel="noopener" >Coalesce</a></li> <li><a class="link" href="https://www.soda.io/" target="_blank" rel="noopener" >Soda</a></li> <li><a class="link" href="https://dqops.com/" target="_blank" rel="noopener" >DQOps</a></li> <li><a class="link" href="https://hedda.io" target="_blank" rel="noopener" >HEDDA.IO</a></li> <li><a class="link" href="https://github.com/MigoXLab/dingo" target="_blank" rel="noopener" >Dingo</a></li> </ul> </li> <li>Education Companies <ul> <li><a class="link" href="https://www.dataexpert.io" target="_blank" rel="noopener" >DataExpert.io</a></li> <li><a class="link" href="https://www.learndataengineering.com" target="_blank" rel="noopener" >LearnDataEngineering.com</a></li> <li><a class="link" href="https://www.algoexpert.io" target="_blank" rel="noopener" >AlgoExpert</a></li> <li><a class="link" href="https://www.bytebytego.com" target="_blank" rel="noopener" >ByteByteGo</a></li> </ul> </li> <li>Analytics / Visualization <ul> <li><a class="link" href="https://www.preset.io" target="_blank" rel="noopener" >Preset</a></li> <li><a class="link" href="https://www.starburst.io" target="_blank" rel="noopener" >Starburst</a></li> <li><a class="link" href="https://www.metabase.com/" target="_blank" rel="noopener" >Metabase</a></li> <li><a class="link" href="https://lookerstudio.google.com/overview" target="_blank" rel="noopener" >Looker Studio</a></li> <li><a class="link" href="https://www.tableau.com/" target="_blank" rel="noopener" >Tableau</a></li> <li><a class="link" href="https://powerbi.microsoft.com/" target="_blank" rel="noopener" >Power BI</a></li> <li><a class="link" href="https://hex.ai/" target="_blank" rel="noopener" >Hex</a></li> <li><a class="link" href="https://superset.apache.org/" target="_blank" rel="noopener" >Apache Superset</a></li> <li><a class="link" href="https://evidence.dev" target="_blank" rel="noopener" >Evidence</a></li> <li><a class="link" href="https://redash.io/" target="_blank" rel="noopener" >Redash</a></li> <li><a class="link" href="https://lightdash.com/" target="_blank" rel="noopener" >Lightdash</a></li> </ul> </li> <li>Data Integration <ul> <li><a class="link" href="https://cube.dev" target="_blank" rel="noopener" >Cube</a></li> <li><a class="link" href="https://www.fivetran.com" target="_blank" rel="noopener" >Fivetran</a></li> <li><a class="link" href="https://airbyte.io" target="_blank" rel="noopener" >Airbyte</a></li> <li><a class="link" href="https://dlthub.com/" target="_blank" rel="noopener" >dlt</a></li> <li><a class="link" href="https://slingdata.io/" target="_blank" rel="noopener" >Sling</a></li> <li><a class="link" href="https://meltano.com/" target="_blank" rel="noopener" >Meltano</a></li> <li><a class="link" href="https://estuary.dev/" target="_blank" rel="noopener" >Estuary</a></li> </ul> </li> <li>Semantic Layers <ul> <li><a class="link" href="https://cube.dev" target="_blank" rel="noopener" >Cube</a></li> <li><a class="link" href="https://www.getdbt.com/product/semantic-layer" target="_blank" rel="noopener" >dbt Semantic Layer</a></li> </ul> </li> <li>Modern OLAP <ul> <li><a class="link" href="https://druid.apache.org/" target="_blank" rel="noopener" >Apache Druid</a></li> <li><a class="link" href="https://clickhouse.com/" target="_blank" rel="noopener" >ClickHouse</a></li> <li><a class="link" href="https://pinot.apache.org/" target="_blank" rel="noopener" >Apache Pinot</a></li> <li><a class="link" href="https://kylin.apache.org/" target="_blank" rel="noopener" >Apache Kylin</a></li> <li><a class="link" href="https://duckdb.org/" target="_blank" rel="noopener" >DuckDB</a></li> <li><a class="link" href="https://questdb.io/" target="_blank" rel="noopener" >QuestDB</a></li> <li><a class="link" href="https://www.starrocks.io/" target="_blank" rel="noopener" >StarRocks</a></li> </ul> </li> <li>LLM application library <ul> <li><a class="link" href="https://github.com/SylphAI-Inc/AdalFlow" target="_blank" rel="noopener" >AdalFlow</a></li> <li><a class="link" href="https://github.com/langchain-ai/langchain" target="_blank" rel="noopener" >LangChain</a></li> <li><a class="link" href="https://github.com/run-llama/llama_index" target="_blank" rel="noopener" >LlamaIndex</a></li> </ul> </li> <li>Real-Time Data <ul> <li><a class="link" href="https://aggregations.io" target="_blank" rel="noopener" >Aggregations.io</a></li> <li><a class="link" href="https://www.responsive.dev/" target="_blank" rel="noopener" >Responsive</a></li> <li><a class="link" href="https://risingwave.com/" target="_blank" rel="noopener" >RisingWave</a></li> <li><a class="link" href="https://www.striim.com/" target="_blank" rel="noopener" >Striim</a></li> </ul> </li> <li>Data Lineage <ul> <li><a class="link" href="https://openlineage.io/" target="_blank" rel="noopener" >OpenLineage</a></li> </ul> </li> </ul> <h3 id="data-engineering-blogs-of-companies">Data Engineering blogs of companies: </h3><ul> <li><a class="link" href="https://netflixtechblog.com/tagged/big-data" target="_blank" rel="noopener" >Netflix</a></li> <li><a class="link" href="https://www.uber.com/blog/houston/data/?uclick_id=b2f43229-f3f4-4bae-bd5d-10a05db2f70c" target="_blank" rel="noopener" >Uber</a></li> <li><a class="link" href="https://www.databricks.com/blog/category/engineering/data-engineering" target="_blank" rel="noopener" >Databricks</a></li> <li><a class="link" href="https://medium.com/airbnb-engineering/data/home" target="_blank" rel="noopener" >Airbnb</a></li> <li><a class="link" href="https://aws.amazon.com/blogs/big-data/" target="_blank" rel="noopener" >Amazon AWS Blog</a></li> <li><a class="link" href="https://techcommunity.microsoft.com/t5/data-architecture-blog/bg-p/DataArchitectureBlog" target="_blank" rel="noopener" >Microsoft Data Architecture Blogs</a></li> <li><a class="link" href="https://blog.fabric.microsoft.com/" target="_blank" rel="noopener" >Microsoft Fabric Blog</a></li> <li><a class="link" href="https://blogs.oracle.com/datawarehousing/" target="_blank" rel="noopener" >Oracle</a></li> <li><a class="link" href="https://engineering.fb.com/category/data-infrastructure/" target="_blank" rel="noopener" >Meta</a></li> <li><a class="link" href="https://www.onehouse.ai/blog" target="_blank" rel="noopener" >Onehouse</a></li> <li><a class="link" href="https://estuary.dev/blog/" target="_blank" rel="noopener" >Estuary Blog</a></li> </ul> <h3 id="data-engineering-whitepapers">Data Engineering Whitepapers: </h3><ul> <li><a class="link" href="https://ibimapublishing.com/articles/CIBIMA/2011/695619/695619.pdf" target="_blank" rel="noopener" >A Five-Layered Business Intelligence Architecture</a></li> <li><a class="link" href="https://www.cidrdb.org/cidr2021/papers/cidr2021_paper17.pdf" target="_blank" rel="noopener" >Lakehouse:A New Generation of Open Platforms that Unify Data Warehousing and Advanced Analytics</a></li> <li><a class="link" href="https://link.springer.com/chapter/10.1007/978-3-030-23381-5_5" target="_blank" rel="noopener" >Big Data Quality: A Data Quality Profiling Model</a></li> <li><a class="link" href="https://arxiv.org/abs/2310.08697" target="_blank" rel="noopener" >The Data Lakehouse: Data Warehousing and More</a></li> <li><a class="link" href="https://dl.acm.org/doi/10.5555/1863103.1863113" target="_blank" rel="noopener" >Spark: Cluster Computing with Working Sets</a></li> <li><a class="link" href="https://research.google/pubs/the-google-file-system/" target="_blank" rel="noopener" >The Google File System</a></li> <li><a class="link" href="https://www.onehouse.ai/whitepaper/onehouse-universal-data-lakehouse-whitepaper" target="_blank" rel="noopener" >Building a Universal Data Lakehouse</a></li> <li><a class="link" href="https://arxiv.org/abs/2401.09621" target="_blank" rel="noopener" >XTable in Action: Seamless Interoperability in Data Lakes</a></li> <li><a class="link" href="https://research.google/pubs/mapreduce-simplified-data-processing-on-large-clusters/" target="_blank" rel="noopener" >MapReduce: Simplified Data Processing on Large Clusters</a></li> <li><a class="link" href="https://vita.had.co.nz/papers/tidy-data.pdf" target="_blank" rel="noopener" >Tidy Data</a></li> <li><a class="link" href="https://www.ssp.sh/brain/data-engineering-whitepapers/" target="_blank" rel="noopener" >Data Engineering Whitepapers</a></li> </ul> <h3 id="social-media-accounts">Social Media Accounts </h3><p>Here&rsquo;s the mostly comprehensive list of data engineering creators: <strong>(You have to have at least 5k followers somewhere to be added!)</strong></p> <h4 id="youtube">YouTube </h4><table> <thead> <tr> <th>Name</th> <th>YouTube Channel</th> <th style="text-align: right">Follower Count</th> </tr> </thead> <tbody> <tr> <td>ByteByteGo</td> <td><a class="link" href="https://www.youtube.com/c/ByteByteGo" target="_blank" rel="noopener" >ByteByteGo</a></td> <td style="text-align: right">1,000,000+</td> </tr> <tr> <td>Zach Wilson</td> <td><a class="link" href="https://www.youtube.com/@eczachly_" target="_blank" rel="noopener" >Data with Zach</a></td> <td style="text-align: right">150,000+</td> </tr> <tr> <td>Shashank Mishra</td> <td><a class="link" href="https://www.youtube.com/@shashank_mishra" target="_blank" rel="noopener" >E-learning Bridge</a></td> <td style="text-align: right">100,000+</td> </tr> <tr> <td>Seattle Data Guy</td> <td><a class="link" href="https://www.youtube.com/c/SeattleDataGuy" target="_blank" rel="noopener" >Seattle Data Guy</a></td> <td style="text-align: right">100,000+</td> </tr> <tr> <td>TrendyTech</td> <td><a class="link" href="https://www.youtube.com/c/TrendytechInsights" target="_blank" rel="noopener" >TrendyTech</a></td> <td style="text-align: right">100,000+</td> </tr> <tr> <td>Darshil Parmar</td> <td><a class="link" href="https://www.youtube.com/@DarshilParmar" target="_blank" rel="noopener" >Darshil Parmar</a></td> <td style="text-align: right">100,000+</td> </tr> <tr> <td>Andreas Kretz</td> <td><a class="link" href="https://www.youtube.com/c/andreaskayy" target="_blank" rel="noopener" >Andreas Kretz</a></td> <td style="text-align: right">100,000+</td> </tr> <tr> <td>The Ravit Show</td> <td><a class="link" href="https://youtube.com/@theravitshow" target="_blank" rel="noopener" >The Ravit Show</a></td> <td style="text-align: right">100,000+</td> </tr> <tr> <td>Guy in a Cube</td> <td><a class="link" href="https://www.youtube.com/@GuyInACube" target="_blank" rel="noopener" >Guy in a Cube</a></td> <td style="text-align: right">100,000+</td> </tr> <tr> <td>Adam Marczak</td> <td><a class="link" href="https://www.youtube.com/@AdamMarczakYT" target="_blank" rel="noopener" >Adam Marczak</a></td> <td style="text-align: right">100,000+</td> </tr> <tr> <td>nullQueries</td> <td><a class="link" href="https://www.youtube.com/@nullQueries" target="_blank" rel="noopener" >nullQueries</a></td> <td style="text-align: right">100,000+</td> </tr> <tr> <td>TECHTFQ by Thoufiq</td> <td><a class="link" href="https://www.youtube.com/@techTFQ" target="_blank" rel="noopener" >TECHTFQ by Thoufiq</a></td> <td style="text-align: right">100,000+</td> </tr> <tr> <td>SQLBI</td> <td><a class="link" href="https://www.youtube.com/@SQLBI" target="_blank" rel="noopener" >SQLBI</a></td> <td style="text-align: right">100,000+</td> </tr> <tr> <td>Alex Freberg</td> <td><a class="link" href="https://www.youtube.com/@AlexTheAnalyst" target="_blank" rel="noopener" >Alex The Analyst</a></td> <td style="text-align: right">100,000+</td> </tr> <tr> <td>Ankur Ranjan</td> <td><a class="link" href="https://www.youtube.com/@TheBigDataShow" target="_blank" rel="noopener" >Big Data Show</a></td> <td style="text-align: right">100,000+</td> </tr> <tr> <td>Prashanth Kumar Pandey</td> <td><a class="link" href="https://www.youtube.com/@ScholarNest" target="_blank" rel="noopener" >ScholarNest</a></td> <td style="text-align: right">77,000+</td> </tr> <tr> <td>ITVersity</td> <td><a class="link" href="https://www.youtube.com/@itversity" target="_blank" rel="noopener" >ITVersity</a></td> <td style="text-align: right">67,000+</td> </tr> <tr> <td>Soumil Shah</td> <td><a class="link" href="https://www.youtube.com/@SoumilShah" target="_blank" rel="noopener" >Soumil Shah</a></td> <td style="text-align: right">50,000</td> </tr> <tr> <td>Ansh Lamba</td> <td><a class="link" href="https://www.youtube.com/@AnshLambaJSR" target="_blank" rel="noopener" >Ansh Lamba</a></td> <td style="text-align: right">18,000+</td> </tr> <tr> <td>Azure Lib</td> <td><a class="link" href="https://www.youtube.com/@azurelib-academy" target="_blank" rel="noopener" >Azure Lib</a></td> <td style="text-align: right">10,000+</td> </tr> <tr> <td>Advancing Analytics</td> <td><a class="link" href="https://www.youtube.com/@AdvancingAnalytics" target="_blank" rel="noopener" >Advancing Analytics</a></td> <td style="text-align: right">10,000+</td> </tr> <tr> <td>Kahan Data Solutions</td> <td><a class="link" href="https://www.youtube.com/@KahanDataSolutions" target="_blank" rel="noopener" >Kahan Data Solutions</a></td> <td style="text-align: right">10,000+</td> </tr> <tr> <td>Ankit Bansal</td> <td><a class="link" href="https://youtube.com/@ankitbansal6" target="_blank" rel="noopener" >Ankit Bansal</a></td> <td style="text-align: right">10,000+</td> </tr> <tr> <td>Mr. K Talks Tech</td> <td><a class="link" href="https://www.youtube.com/channel/UCzdOan4AmF65PmLLks8Lmww" target="_blank" rel="noopener" >Mr. K Talks Tech</a></td> <td style="text-align: right">10,000+</td> </tr> <tr> <td>Samuel Focht</td> <td><a class="link" href="https://www.youtube.com/@PythonBasics" target="_blank" rel="noopener" >Python Basics</a></td> <td style="text-align: right">10,000+</td> </tr> <tr> <td>Mehdi Ouazza</td> <td><a class="link" href="https://www.youtube.com/@mehdio" target="_blank" rel="noopener" >Mehdio DataTV</a></td> <td style="text-align: right">3,000+</td> </tr> <tr> <td>Alex Merced</td> <td><a class="link" href="https://www.youtube.com/@alexmerceddata_" target="_blank" rel="noopener" >Alex Merced Data</a></td> <td style="text-align: right">N/A</td> </tr> <tr> <td>John Kutay</td> <td><a class="link" href="https://www.youtube.com/@striiminc" target="_blank" rel="noopener" >John Kutay</a></td> <td style="text-align: right">N/A</td> </tr> <tr> <td>Emil Kaminski</td> <td><a class="link" href="https://www.youtube.com/@DatabricksPro" target="_blank" rel="noopener" >Databricks For Professionals</a></td> <td style="text-align: right">5,000+</td> </tr> </tbody> </table> <h4 id="linkedin">LinkedIn </h4><table> <thead> <tr> <th>Name</th> <th>LinkedIn Profile</th> <th style="text-align: right">Follower Count</th> </tr> </thead> <tbody> <tr> <td>Zach Wilson</td> <td><a class="link" href="https://www.linkedin.com/in/eczachly" target="_blank" rel="noopener" >Zach Wilson</a></td> <td style="text-align: right">400,000+</td> </tr> <tr> <td>Chip Huyen</td> <td><a class="link" href="https://www.linkedin.com/in/chiphuyen/" target="_blank" rel="noopener" >Chip Huyen</a></td> <td style="text-align: right">250,000+</td> </tr> <tr> <td>Shashank Mishra</td> <td><a class="link" href="https://www.linkedin.com/in/shashank219/" target="_blank" rel="noopener" >Shashank Mishra</a></td> <td style="text-align: right">100,000+</td> </tr> <tr> <td>Seattle Data Guy</td> <td><a class="link" href="https://www.linkedin.com/in/benjaminrogojan" target="_blank" rel="noopener" >Ben Rogojan</a></td> <td style="text-align: right">100,000+</td> </tr> <tr> <td>TrendyTech</td> <td><a class="link" href="https://www.linkedin.com/in/bigdatabysumit/" target="_blank" rel="noopener" >Sumit Mittal</a></td> <td style="text-align: right">100,000+</td> </tr> <tr> <td>Darshil Parmar</td> <td><a class="link" href="https://www.linkedin.com/in/darshil-parmar/" target="_blank" rel="noopener" >Darshil Parmar</a></td> <td style="text-align: right">100,000+</td> </tr> <tr> <td>Andreas Kretz</td> <td><a class="link" href="https://www.linkedin.com/in/andreas-kretz" target="_blank" rel="noopener" >Andreas Kretz</a></td> <td style="text-align: right">100,000+</td> </tr> <tr> <td>ByteByteGo (Alex Xu)</td> <td><a class="link" href="https://www.linkedin.com/in/alexxubyte" target="_blank" rel="noopener" >Alex Xu</a></td> <td style="text-align: right">100,000+</td> </tr> <tr> <td>Azure Lib (Deepak Goyal)</td> <td><a class="link" href="https://www.linkedin.com/in/deepak-goyal-93805a17/" target="_blank" rel="noopener" >Deepak Goyal</a></td> <td style="text-align: right">100,000+</td> </tr> <tr> <td>Alex Freberg</td> <td><a class="link" href="https://www.linkedin.com/in/alex-freberg/" target="_blank" rel="noopener" >Alex Freberg</a></td> <td style="text-align: right">100,000+</td> </tr> <tr> <td>SQLBI (Marco Russo)</td> <td><a class="link" href="https://www.linkedin.com/in/sqlbi" target="_blank" rel="noopener" >Marco Russo</a></td> <td style="text-align: right">50,000+</td> </tr> <tr> <td>Ankit Bansal</td> <td><a class="link" href="https://www.linkedin.com/in/ankitbansal6/" target="_blank" rel="noopener" >Ankit Bansal</a></td> <td style="text-align: right">50,000+</td> </tr> <tr> <td>Marc Lamberti</td> <td><a class="link" href="https://www.linkedin.com/in/marclamberti" target="_blank" rel="noopener" >Marc Lamberti</a></td> <td style="text-align: right">50,000+</td> </tr> <tr> <td>Ankur Ranjan</td> <td><a class="link" href="https://www.linkedin.com/in/thebigdatashow/" target="_blank" rel="noopener" >Ankur Ranjan</a></td> <td style="text-align: right">48,000+</td> </tr> <tr> <td>ITVersity (Durga Gadiraju)</td> <td><a class="link" href="https://www.linkedin.com/in/durga0gadiraju/" target="_blank" rel="noopener" >Durga Gadiraju</a></td> <td style="text-align: right">48,000+</td> </tr> <tr> <td>Prashanth Kumar Pandey</td> <td><a class="link" href="https://www.linkedin.com/in/prashant-kumar-pandey/" target="_blank" rel="noopener" >Prashanth Kumar Pandey</a></td> <td style="text-align: right">37,000+</td> </tr> <tr> <td>Alex Merced</td> <td><a class="link" href="https://www.linkedin.com/in/alexmerced" target="_blank" rel="noopener" >Alex Merced</a></td> <td style="text-align: right">30,000+</td> </tr> <tr> <td>Ijaz Ali</td> <td><a class="link" href="https://www.linkedin.com/in/ijaz-ali-6aaa87122/" target="_blank" rel="noopener" >Ijaz Ali</a></td> <td style="text-align: right">24,000+</td> </tr> <tr> <td>Mehdi Ouazza</td> <td><a class="link" href="https://www.linkedin.com/in/mehd-io/" target="_blank" rel="noopener" >Mehdi Ouazza</a></td> <td style="text-align: right">20,000+</td> </tr> <tr> <td>Ananth Packkildurai</td> <td><a class="link" href="https://www.linkedin.com/in/ananthdurai/" target="_blank" rel="noopener" >Ananth Packkildurai</a></td> <td style="text-align: right">18,000+</td> </tr> <tr> <td>Ansh Lamba</td> <td><a class="link" href="https://www.linkedin.com/in/ansh-lamba-793681184/" target="_blank" rel="noopener" >Ansh Lamba</a></td> <td style="text-align: right">13,000+</td> </tr> <tr> <td>Manojkumar Vadivel</td> <td><a class="link" href="https://www.linkedin.com/in/manojvsj/" target="_blank" rel="noopener" >Manojkumar Vadivel</a></td> <td style="text-align: right">12,000+</td> </tr> <tr> <td>Advancing Analytics</td> <td><a class="link" href="https://www.linkedin.com/in/simon-whiteley-uk/" target="_blank" rel="noopener" >Simon Whiteley</a></td> <td style="text-align: right">10,000+</td> </tr> <tr> <td>Li Yin</td> <td><a class="link" href="https://www.linkedin.com/in/li-yin-ai/" target="_blank" rel="noopener" >Li Yin</a></td> <td style="text-align: right">10,000+</td> </tr> <tr> <td>Jaco van Gelder</td> <td><a class="link" href="https://www.linkedin.com/in/jwvangelder/" target="_blank" rel="noopener" >Jaco van Gelder</a></td> <td style="text-align: right">10,000+</td> </tr> <tr> <td>Joseph Machado</td> <td><a class="link" href="https://www.linkedin.com/in/josephmachado1991/" target="_blank" rel="noopener" >Joseph Machado</a></td> <td style="text-align: right">10,000+</td> </tr> <tr> <td>Eric Roby</td> <td><a class="link" href="https://www.linkedin.com/in/codingwithroby/" target="_blank" rel="noopener" >Eric Roby</a></td> <td style="text-align: right">10,000+</td> </tr> <tr> <td>Simon Späti</td> <td><a class="link" href="https://www.linkedin.com/in/sspaeti/" target="_blank" rel="noopener" >Simon Späti</a></td> <td style="text-align: right">10,000+</td> </tr> <tr> <td>Constantin Lungu</td> <td><a class="link" href="https://www.linkedin.com/in/constantin-lungu-668b8756" target="_blank" rel="noopener" >Constantin Lungu</a></td> <td style="text-align: right">10,000+</td> </tr> <tr> <td>Lakshmi Sontenam</td> <td><a class="link" href="https://www.linkedin.com/in/shivaga9esh" target="_blank" rel="noopener" >Lakshmi Sontenam</a></td> <td style="text-align: right">9,500+</td> </tr> <tr> <td>Dani Pálma</td> <td><a class="link" href="https://www.linkedin.com/in/danthelion/" target="_blank" rel="noopener" >Daniel Pálma</a></td> <td style="text-align: right">9,000+</td> </tr> <tr> <td>Soumil Shah</td> <td><a class="link" href="https://www.linkedin.com/in/shah-soumil/" target="_blank" rel="noopener" >Soumil Shah</a></td> <td style="text-align: right">8,000+</td> </tr> <tr> <td>Arnaud Milleker</td> <td><a class="link" href="https://www.linkedin.com/in/arnaudmilleker/" target="_blank" rel="noopener" >Arnaud Milleker</a></td> <td style="text-align: right">7,000+</td> </tr> <tr> <td>Dimitri Visnadi</td> <td><a class="link" href="https://www.linkedin.com/in/visnadi/" target="_blank" rel="noopener" >Dimitri Visnadi</a></td> <td style="text-align: right">7,000+</td> </tr> <tr> <td>Lenny</td> <td><a class="link" href="https://www.linkedin.com/in/lennyardiles/" target="_blank" rel="noopener" >Lenny A</a></td> <td style="text-align: right">6,000+</td> </tr> <tr> <td>Dipankar Mazumdar</td> <td><a class="link" href="https://www.linkedin.com/in/dipankar-mazumdar/" target="_blank" rel="noopener" >Dipankar Mazumdar</a></td> <td style="text-align: right">5,000+</td> </tr> <tr> <td>Daniel Ciocirlan</td> <td><a class="link" href="https://www.linkedin.com/in/danielciocirlan" target="_blank" rel="noopener" >Daniel Ciocirlan</a></td> <td style="text-align: right">5,000+</td> </tr> <tr> <td>Hugo Lu</td> <td><a class="link" href="https://www.linkedin.com/in/hugo-lu-confirmed/" target="_blank" rel="noopener" >Hugo Lu</a></td> <td style="text-align: right">5,000+</td> </tr> <tr> <td>Tobias Macey</td> <td><a class="link" href="https://www.linkedin.com/in/tmacey" target="_blank" rel="noopener" >Tobias Macey</a></td> <td style="text-align: right">5,000+</td> </tr> <tr> <td>Marcos Ortiz</td> <td><a class="link" href="https://www.linkedin.com/in/mlortiz" target="_blank" rel="noopener" >Marcos Ortiz</a></td> <td style="text-align: right">5,000+</td> </tr> <tr> <td>Julien Hurault</td> <td><a class="link" href="https://www.linkedin.com/in/julienhuraultanalytics/" target="_blank" rel="noopener" >Julien Hurault</a></td> <td style="text-align: right">5,000+</td> </tr> <tr> <td>John Kutay</td> <td><a class="link" href="https://www.linkedin.com/in/johnkutay/" target="_blank" rel="noopener" >John Kutay</a></td> <td style="text-align: right">5,000+</td> </tr> <tr> <td>Hassaan Akbar</td> <td><a class="link" href="https://www.linkedin.com/in/ehassaan" target="_blank" rel="noopener" >Hassaan Akbar</a></td> <td style="text-align: right">5,000+</td> </tr> <tr> <td>Subhankar</td> <td><a class="link" href="https://www.linkedin.com/in/subhankarumass/" target="_blank" rel="noopener" >Subhankar</a></td> <td style="text-align: right">5,000+</td> </tr> <tr> <td>Nitin</td> <td><a class="link" href="https://www.linkedin.com/in/tomernitin29/" target="_blank" rel="noopener" >Nitin</a></td> <td style="text-align: right">N/A</td> </tr> <tr> <td>Hassaan</td> <td><a class="link" href="https://www.linkedin.com/in/shassaan/" target="_blank" rel="noopener" >Hassaan</a></td> <td style="text-align: right">5000+</td> </tr> <tr> <td>Javier de la Torre</td> <td><a class="link" href="www.linkedin.com/in/javier-de-la-torre-medina" >Javier</a></td> <td style="text-align: right">5000+</td> </tr> </tbody> </table> <h4 id="xtwitter">X/Twitter </h4><table> <thead> <tr> <th>Name</th> <th>X/Twitter Profile</th> <th style="text-align: right">Follower Count</th> </tr> </thead> <tbody> <tr> <td>ByteByteGo</td> <td><a class="link" href="https://twitter.com/alexxubyte/" target="_blank" rel="noopener" >alexxubyte</a></td> <td style="text-align: right">100,000+</td> </tr> <tr> <td>Dan Kornas</td> <td><a class="link" href="https://www.twitter.com/dankornas" target="_blank" rel="noopener" >@dankornas</a></td> <td style="text-align: right">66,000+</td> </tr> <tr> <td>Zach Wilson</td> <td><a class="link" href="https://www.twitter.com/EcZachly" target="_blank" rel="noopener" >EcZachly</a></td> <td style="text-align: right">30,000+</td> </tr> <tr> <td>Seattle Data Guy</td> <td><a class="link" href="https://www.twitter.com/SeattleDataGuy" target="_blank" rel="noopener" >SeattleDataGuy</a></td> <td style="text-align: right">10,000+</td> </tr> <tr> <td>SQLBI</td> <td><a class="link" href="https://x.com/marcorus" target="_blank" rel="noopener" >marcorus</a></td> <td style="text-align: right">10,000+</td> </tr> <tr> <td>Joseph Machado</td> <td><a class="link" href="https://twitter.com/startdataeng" target="_blank" rel="noopener" >startdataeng</a></td> <td style="text-align: right">5,000+</td> </tr> <tr> <td>Alex Merced</td> <td><a class="link" href="https://www.twitter.com/amdatalakehouse" target="_blank" rel="noopener" >@amdatalakehouse</a></td> <td style="text-align: right">N/A</td> </tr> <tr> <td>John Kutay</td> <td><a class="link" href="https://x.com/JohnKutay" target="_blank" rel="noopener" >@JohnKutay</a></td> <td style="text-align: right">N/A</td> </tr> <tr> <td>Mehdi Ouazza</td> <td><a class="link" href="https://x.com/mehd_io" target="_blank" rel="noopener" >mehd_io</a></td> <td style="text-align: right">N/A</td> </tr> </tbody> </table> <h4 id="instagram">Instagram </h4><table> <thead> <tr> <th>Name</th> <th>Instagram Profile</th> <th style="text-align: right">Follower Count</th> </tr> </thead> <tbody> <tr> <td>Sundas Khalid</td> <td><a class="link" href="https://www.instagram.com/sundaskhalidd" target="_blank" rel="noopener" >sundaskhalidd</a></td> <td style="text-align: right">300,000+</td> </tr> <tr> <td>Zach Wilson</td> <td><a class="link" href="https://www.instagram.com/eczachly" target="_blank" rel="noopener" >eczachly</a></td> <td style="text-align: right">150,000+</td> </tr> <tr> <td>Andreas Kretz</td> <td><a class="link" href="https://www.instagram.com/learndataengineering" target="_blank" rel="noopener" >learndataengineering</a></td> <td style="text-align: right">5,000+</td> </tr> <tr> <td>Alex Merced</td> <td><a class="link" href="https://www.instagram.com/alexmercedcoder" target="_blank" rel="noopener" >@alexmercedcoder</a></td> <td style="text-align: right">N/A</td> </tr> </tbody> </table> <h4 id="tiktok">TikTok </h4><table> <thead> <tr> <th>Name</th> <th>TikTok Profile</th> <th style="text-align: right">Follower Count</th> </tr> </thead> <tbody> <tr> <td>Zach Wilson</td> <td><a class="link" href="https://www.tiktok.com/@eczachly" target="_blank" rel="noopener" >@eczachly</a></td> <td style="text-align: right">70,000+</td> </tr> <tr> <td>Alex Freberg</td> <td><a class="link" href="https://www.tiktok.com/@alex_the_analyst" target="_blank" rel="noopener" >@alex_the_analyst</a></td> <td style="text-align: right">10,000+</td> </tr> <tr> <td>Mehdi Ouazza</td> <td><a class="link" href="https://www.tiktok.com/@mehdio_datatv" target="_blank" rel="noopener" >@mehdio_datatv</a></td> <td style="text-align: right">N/A</td> </tr> </tbody> </table> <h3 id="great-podcasts">Great Podcasts </h3><ul> <li><a class="link" href="https://www.dataengineeringshow.com/" target="_blank" rel="noopener" >The Data Engineering Show</a></li> <li><a class="link" href="https://www.dataengineeringpodcast.com/" target="_blank" rel="noopener" >Data Engineering Podcast</a></li> <li><a class="link" href="https://www.datatopics.io/" target="_blank" rel="noopener" >DataTopics</a></li> <li><a class="link" href="https://podcasts.apple.com/us/podcast/the-engineering-side-of-data/id1566999533" target="_blank" rel="noopener" >The Data Engineering Side Of Data</a></li> <li><a class="link" href="https://www.ascend.io/dataaware-podcast/" target="_blank" rel="noopener" >DataWare</a></li> <li><a class="link" href="https://www.deezer.com/us/show/5293247" target="_blank" rel="noopener" >The Data Coffee Break Podcast</a></li> <li><a class="link" href="https://datastackshow.com/" target="_blank" rel="noopener" >The Datastack show</a></li> <li><a class="link" href="https://www.intricity.com/learningcenter/podcast" target="_blank" rel="noopener" >Intricity101 Data Sharks Podcast</a></li> <li><a class="link" href="https://www.rittmananalytics.com/drilltodetail/" target="_blank" rel="noopener" >Drill to Detail with Mark Rittman</a></li> <li><a class="link" href="https://analyticshour.io/" target="_blank" rel="noopener" >Analytics Power Hour</a></li> <li><a class="link" href="https://listen.casted.us/public/127/Catalog-%26-Cocktails-2fcf8728" target="_blank" rel="noopener" >Catalog &amp; cocktails</a></li> <li><a class="link" href="https://datatalks.club/podcast.html" target="_blank" rel="noopener" >Datatalks</a></li> <li><a class="link" href="https://www.databricks.com/discover/data-brew" target="_blank" rel="noopener" >Data Brew by Databricks</a></li> <li><a class="link" href="https://rise-of-the-data-cloud.simplecast.com/" target="_blank" rel="noopener" >The Data Cloud Podcast by Snowflake</a></li> <li><a class="link" href="https://www.striim.com/podcast/" target="_blank" rel="noopener" >What&rsquo;s New in Data</a></li> <li><a class="link" href="https://www.datastax.com/resources/podcast/open-source-data" target="_blank" rel="noopener" >Open||Source||Data by Datastax</a></li> <li><a class="link" href="https://developer.confluent.io/podcast/" target="_blank" rel="noopener" >Streaming Audio by confluent</a></li> <li><a class="link" href="https://podcasts.apple.com/us/podcast/the-data-scientist-show/id1584430381" target="_blank" rel="noopener" >The Data Scientist Show</a></li> <li><a class="link" href="https://podcast.mlops.community/" target="_blank" rel="noopener" >MLOps.community</a></li> <li><a class="link" href="https://open.spotify.com/show/3Km3lBNzJpc1nOTJUtbtMh" target="_blank" rel="noopener" >Monday Morning Data Chat</a></li> <li><a class="link" href="https://www.thoughtspot.com/data-chief/podcast" target="_blank" rel="noopener" >The Data Chief</a></li> </ul> <h3 id="great-list-of-20-newsletters">Great <a class="link" href="newsletters.md" >list of 20+ newsletters</a> </h3><p>Top must follow newsletters for data engineering:</p> <ul> <li><a class="link" href="https://blog.dataengineer.io" target="_blank" rel="noopener" >DataEngineer.io Newsletter</a></li> <li><a class="link" href="https://joereis.substack.com" target="_blank" rel="noopener" >Joe Reis</a></li> <li><a class="link" href="https://www.startdataengineering.com" target="_blank" rel="noopener" >Start Data Engineering</a></li> <li><a class="link" href="https://www.dataengineeringweekly.com" target="_blank" rel="noopener" >Data Engineering Weekly</a></li> <li><a class="link" href="https://dataengineerthings.substack.com/" target="_blank" rel="noopener" >Data Engineer Things</a></li> </ul> <h3 id="glossaries">Glossaries: </h3><ul> <li><a class="link" href="https://www.ssp.sh/brain/data-engineering/" target="_blank" rel="noopener" >Data Engineering Vault</a></li> <li><a class="link" href="https://glossary.airbyte.com/" target="_blank" rel="noopener" >Airbyte Data Glossary</a></li> <li><a class="link" href="https://dataengineering.wiki/Index" target="_blank" rel="noopener" >Data Engineering Wiki by Reddit</a></li> <li><a class="link" href="https://www.secoda.co/glossary/" target="_blank" rel="noopener" >Seconda Glossary</a></li> <li><a class="link" href="https://www.databricks.com/glossary" target="_blank" rel="noopener" >Glossary Databricks</a></li> <li><a class="link" href="https://airtable.com/shrGh8BqZbkfkbrfk/tbluZ3ayLHC3CKsDb" target="_blank" rel="noopener" >Airtable Glossary</a></li> <li><a class="link" href="https://dagster.io/glossary" target="_blank" rel="noopener" >Data Engineering Glossary by Dagster</a></li> </ul> <h3 id="design-patterns">Design Patterns </h3><ul> <li><a class="link" href="https://www.github.com/DataExpert-io/cumulative-table-design" target="_blank" rel="noopener" >Cumulative Table Design</a></li> <li><a class="link" href="https://www.github.com/EcZachly/microbatch-hourly-deduped-tutorial" target="_blank" rel="noopener" >Microbatch Deduplication</a></li> <li><a class="link" href="https://www.github.com/EcZachly/little-book-of-pipelines" target="_blank" rel="noopener" >The Little Book of Pipelines</a></li> <li><a class="link" href="https://datadeveloperplatform.org/architecture/" target="_blank" rel="noopener" >Data Developer Platform</a></li> </ul> <h3 id="courses--academies">Courses / Academies </h3><ul> <li><a class="link" href="https://www.dataexpert.io" target="_blank" rel="noopener" >DataExpert.io course</a> use code <strong>HANDBOOK10</strong> for a discount!</li> <li><a class="link" href="https://www.learndataengineering.com" target="_blank" rel="noopener" >LearnDataEngineering.com</a></li> <li><a class="link" href="https://www.technicalfreelanceracademy.com/" target="_blank" rel="noopener" >Technical Freelancer Academy</a> Use code <strong>zwtech</strong> for a discount!</li> <li><a class="link" href="https://www.edx.org/learn/data-engineering/ibm-data-engineering-basics-for-everyone" target="_blank" rel="noopener" >IBM Data Engineering for Everyone</a></li> <li><a class="link" href="https://www.qwiklabs.com/" target="_blank" rel="noopener" >Qwiklabs</a></li> <li><a class="link" href="https://www.datacamp.com/" target="_blank" rel="noopener" >DataCamp</a></li> <li><a class="link" href="https://www.udemy.com/user/shruti-mantri-5/" target="_blank" rel="noopener" >Udemy Courses from Shruti Mantri</a></li> <li><a class="link" href="https://rockthejvm.com/" target="_blank" rel="noopener" >Rock the JVM</a> teaches Spark (in Scala), Flink and others</li> <li><a class="link" href="https://datatalks.club/" target="_blank" rel="noopener" >Data Engineering Zoomcamp by DataTalksClub</a></li> <li><a class="link" href="https://josephmachado.podia.com/efficient-data-processing-in-spark" target="_blank" rel="noopener" >Efficient Data Processing in Spark</a></li> <li><a class="link" href="https://www.scaler.com/" target="_blank" rel="noopener" >Scaler</a></li> <li><a class="link" href="https://www.datateams.ai/" target="_blank" rel="noopener" >DataTeams - Data Engingeer hiring platform</a></li> <li><a class="link" href="https://danielblanco.dev/links" target="_blank" rel="noopener" >Udemy Courses from Daniel Blanco</a></li> </ul> <h3 id="certifications-courses">Certifications Courses </h3><ul> <li><a class="link" href="https://cloud.google.com/certification/data-engineer" target="_blank" rel="noopener" >Google Cloud Certified - Professional Data Engineer</a></li> <li><a class="link" href="https://www.databricks.com/learn/certification/apache-spark-developer-associate" target="_blank" rel="noopener" >Databricks - Certified Associate Developer for Apache Spark</a></li> <li><a class="link" href="https://www.databricks.com/learn/certification/data-engineer-associate" target="_blank" rel="noopener" >Databricks - Data Engineer Associate</a></li> <li><a class="link" href="https://www.databricks.com/learn/certification/data-engineer-professional" target="_blank" rel="noopener" >Databricks - Data Engineer Professional</a></li> <li><a class="link" href="https://learn.microsoft.com/en-us/credentials/certifications/exams/dp-203/?tab=tab-learning-paths" target="_blank" rel="noopener" >Microsoft DP-203: Data Engineering on Microsoft Azure</a></li> <li><a class="link" href="https://learn.microsoft.com/credentials/certifications/fabric-analytics-engineer-associate/" target="_blank" rel="noopener" >Microsoft DP-600: Fabric Analytics Engineer Associate</a></li> <li><a class="link" href="https://learn.microsoft.com/en-us/credentials/certifications/fabric-data-engineer-associate/?practice-assessment-type=certification" target="_blank" rel="noopener" >Microsoft DP-700: Fabric Data Engineer Associate</a></li> <li><a class="link" href="https://aws.amazon.com/certification/certified-data-engineer-associate/" target="_blank" rel="noopener" >AWS Certified Data Engineer - Associate</a></li> </ul>