TaskFlow API is a feature that promises data sharing functionality and a simple interface for building data pipelines in Apache Airflow 2.0. It should allow the end-users to write Python code rather than Airflow code.

Observabilit has gained a lot of popularity in recent years. To assess the health of IT systems, engineering teams typically use logs, metrics, and traces, which are used by various developer tools to facilitate observability.

How to manage external resources in Python with your custom context managers

What I learned from working with data at various companies

In this blog post, I discuss EKS on Fargate, a service that lets us run a serverless Kubernetes cluster on AWS. I demonstrate the differences between ECS and EKS on Fargate and their implications.