My writing portfolio before 2024

Links to some of my older blog posts

Below is a list with links to some of my past writing. Happy 2024 and beyond! 🥂

Data Engineering

• Should You Hire a Software Engineer or a Data Engineer?

• Workflow Orchestration vs. Data Orchestration — Are Those Different?

• 10 Common Mistakes When Building Analytical Data Models

• Is Python Really a Bottleneck?

• 3 Tricks to Simplify Your Python Data Projects

• TaskFlow API in Apache Airflow 2.0 — Should You Use It?

• Are NoSQL Databases Relevant for Data Engineering?

• 7 Reasons Why You Should Consider a Data Lake (and Event-Driven ETL)

• Can Data Lakes Accelerate Building ML Data Pipelines?

• Here is What Happens If You Decouple Your BI Stack

• How a Shared Slack Channel Can Improve Your Data Quality

• “Don’t Repeat Yourself” is beneficial — Not Only in Software Engineering

• 10 Reasons Why Estimating Time For Data Projects is Hard

• Is Real-Time Processing Worth It For Your Analytical Use Cases?

• Why Kubernetes Has Become So Popular in Data Engineering

• Put a Stop to Data Swamps With Event-Driven Data Testing

• Scheduled vs. Event-driven Data Pipelines

• Event-driven Data Pipelines with AWS Lambda and GitHub Actions

• Programmable Data Infrastructure is Finally Within Reach

• DuckDB vs. MotherDuck — should you switch to the Cloud version?

• Apache Iceberg Crash Course for AWS users: Amazon S3, Athena & AWS Glue ❤️ Iceberg

• Polars, DuckDB, Pandas, Modin, Ponder, Fugue, Daft — which one is the best dataframe and SQL tool?

• Why data integration will never be fully solved, and what Fivetran, Airbyte, Singer, dlt and CloudQuery do about it

Serverless, Cloud & AI

• Why Many Engineers Don’t Understand Serverless

• Monitoring vs. Observability: Can You Tell The Difference?

• 10 Mistakes When Sizing Cloud Resources

• 10 Ways To Protect Your Mission-Critical Database

• How To Securely Parse GitHub Actions JSON Secrets for Azure CI/CD

• AI Tools and Autonomous Agents: Auto-GPT, BabyAGI, LangChain, AgentGPT, HeyGPT, and more

• How I Use OpenAI’s GPT-4 To Stay In Touch With My Mum More Consistently

• When GitHub Actions Get Painful to Troubleshoot, Try This Instead

AWS

• 10 Simple Hacks That Will Make You More Productive When Using AWS

• AWS Kinesis vs. SNS vs. SQS — A Comparison With Python Examples

• Demystifying boto3: How to Use Any AWS Service with Python

• 8 Must-Know Tricks to Use AWS S3 More Effectively in Python

• How I Manage Credentials in Python Using AWS Secrets Manager

• How I Built a Data Discovery API for AWS Data Lake

• How I Built CI/CD For Data Pipelines in Apache Airflow on AWS

• 8 Common Mistakes When Using AWS ECS to Manage Containers

• Deep Dive Into Amazon Timestream — Data Ingestion in Python

• Deep Dive Into Amazon Timestream — Building a Real-Time Dashboard

• How to Deploy a ChatGPT-powered Python FastAPI Microservice to AWS App Runner

Thinking in public

• All the Code You Will Ever Write Is Business Logic

• Should You Measure the Value of a Data Team?

• 10 Challenging Aspects of Software Engineering

• What I learned from NormConf 2022

• Giving and Receiving Negative Feedback

• 2023 State of Data Infrastructure — Key Trends from Matt Turck’s MAD Landscape

• Brevity means clarity