Why Your Current ETL Tools Are Just A House Of Cards

Friday, April 24, 2015

Picture of Andrew Griffin
Andrew Griffin
HouseOfCards_ETL

I’ve finally gotten around to watching the third season of “House of Cards,” and it’s as haunting and powerful as ever. It’s the type of show that makes me want to shower immediately after I watch it. I’m disgusted by the idea of a political system so choked by inflexibility, dysfunction, and widespread public deception. Is this how Washington DC really operates? If so, then it’s truly disillusioning.

What is a House of Cards, anyway? I typed this into Google, and the answer I got back was, “a structure built out of playing cards precariously balanced together.” For this Netflix show about Frank Underwood manipulating the dysfunction in government to maneuver his way into the highest seat in the land, it’s an apt name. “House of Cards” may also be an apt name for how ETL tools can impact a business without business leaders being aware of it. Let me explain.

Most business leaders don’t know what ETL tools are, or why they’re important, just like most of us don’t know who the Frank Underwoods in DC are (or why they matter). ETL tools (or “Extract, Transform, Load” tools) were designed to EXTRACT data out of one database, TRANSFORM it to suit another format, and LOAD that data into another database. For a long time, this was how data from disparate sources was aggregated and how business analytics were generated.

The truth is, most ETL tools in market today are a “House of Cards” because they consolidated power in the hands of the few who could use them — namely IT. Until recently, all data in a corporation was owned by the corporate IT function, and whenever any business users needed it, IT served as an increasingly inefficient gatekeeper (inefficient because as businesses have become more digital, technology systems and sources of data have exploded).

You could call that a downright tyrannical form of data “government” (which is what I would call Frank Underwood’s regime.) The fact is, ETL tools promote a “black box” — or inscrutable — approach because they “hide data” from the business, in a sense. They are keys that only certain people know how to use. ETL tools are mostly designed for Java developers, which leaves businesspeople with no coding skills in the dark and dependent upon their IT counterparts.

Meanwhile, modern business has evolved to make data accessible to everyone in an organization. In the very near future, access to data will be determined in real-time by using a “service gate” that opens and closes with the right key combination based on use case, role and identity. But ETL tools still haven’t evolved to match the new patterns of data ownership and accessibility. For example, they don’t enable business users to access data instantly when they need it. And they certainly don’t encourage collaboration between business and IT.

Like the power-hoarding Frank Underwood, CIOs who hoard data will get a nasty wake-up call one day soon.

My advice to such CIOs is, “Give power back to the people!” Don’t let dysfunctional ETL tools hold back your business. Empower your business users with self-service integration capabilities. Invest in ETL technologies that enable them to be self-sufficient data consumers. And finally, encourage shared data ownership, collaboration, and greater transparency between IT and the business. Or else it will all come falling down, just like a house of cards.