Data Engineer, Freelance Technical Writer, AWS Certified Solution Architect, HIIT, cloud & tech enthusiast living in Berlin.

Featured Posts:

Here is What Happens If You Decouple Your BI Stack

How to make Business Intelligence future-proof by applying software engineering principles

Read more...

TaskFlow API in Apache Airflow 2.0 — Should You Use It?

TaskFlow API is a feature that promises data sharing functionality and a simple interface for building data pipelines in Apache Airflow 2.0. It should allow the end-users to write Python code rather than Airflow code.

Read more...

Monitoring vs. Observability: Can You Tell The Difference?

Observabilit has gained a lot of popularity in recent years. To assess the health of IT systems, engineering teams typically use logs, metrics, and traces, which are used by various developer tools to facilitate observability.

Read more...

Manage Files and Database Connections in Python Like a Pro

How to manage external resources in Python with your custom context managers

Read more...

10 Data Engineering Practices to Ensure Data and Code Quality

What I learned from working with data at various companies

Read more...

Serverless Kubernetes Cluster on AWS with EKS on Fargate

In this blog post, I discuss EKS on Fargate, a service that lets us run a serverless Kubernetes cluster on AWS. I demonstrate the differences between ECS and EKS on Fargate and their implications.

Read more...

Why Many Engineers Are Missing The Point of Serverless

Recently, I saw a video that serverless doesn't make sense. Even though I really enjoyed it, I am not sure if the author’s points about serverless are valid.

Read more...

What to Consider When Migrating Data Warehouse to the Cloud

What to consider to make your data warehouse and data lake future-proof & how the separation of storage and compute was approached by Snowflake, Amazon, Google, SAP, and IBM

Read more...

Is Apache Airflow good enough for current data engineering needs?

The pros and cons of Apache Airflow as a workflow management platform for ETL & Data Science and deriving from that the use cases for which Airflow may be a good or a bad choice

Read more...

Managing dependencies between data pipelines in Apache Airflow & Prefect

A simple approach to managing dependencies between your workflows

Read more...

How to Get a Job as a Data Engineer

10 skills in demand and major factors determining career prospects in one of the fastest-growing profession of the century

Read more...

It Took Two Days and Seven Engineers to Move Data Between Two S3 Buckets

A team of engineers tried to quickly transfer 25TB of data from one S3 bucket to another. Their requirement was to do it within the next two hours. It worked, but it took 2 days, and 7 engineers.

Read more...