Summary: The powder keg of the modern data stack by Benn Stancil

Life would be much easier if we had a single source of truth for what's happening across the data stack.

Original article: The powder keg of the modern data stack by Benn Stancil

Motivation: we would like our data to be organized and our tools to be well integrated together, but it’s difficult to accomplish in today’s fragmented data stack.

Single pane of glass

If your executives want to get a bird’s eye view of your data, its lineage, tools & people that use it, the key data assets in your cloud data warehouse, and the overall health of your system, there is no single dashboard you could offer. Everyone would love to see a management console for the data stack, but it doesn’t exist.

Who should offer a single pane of glass? Multiple vendor categories strive to become the central piece of the data stack:

“Data catalogs, data discovery platforms, metrics layers, observability tools, transformation and orchestration tools—all of these categories offer a sliver of centralization, from a single place to find data, to understand its lineage, to monitor its accuracy, or to define the business logic that’s applied to it. It’s easy to imagine a three-step master plan from each of these companies, starting with where they are today, and ending with being the control tower for the modern data stack.”

Core message & CTA

“The problem isn’t our technology or how teams use it; it’s market […]. Each new effort to make it better actually makes the fragmentation worse.” There will be many companies who will try to win that market, not just the small box they are in, but the author claims that “there can only be one single source of truth, one single pane of glass—and not everyone can be king.“