Silos, silos everywhere

The obstacles preventing data democratization

Rick Delgado

© Shutterstock / alphaspirit

Transparency is not something we’re typically very good at as a general rule — we’ve got a lot of work to do. Rick Delgado identifies the organizational and societal barriers to data democratization.

“Democratization,” is a term that has lost its zest of the past few years in tech-land — it sits right alongside disruption and innovation in terms of the amount of abuse it’s taken from startups and others describing how their idea or technology is going to introduce open principles and access within one vertical or another.

At its core, democratization is a hybridization of democratic principles (everyone has equal voice or representation) and open-source (everyone has access to the codebase). In the realm of data, this means radical transparency and access to information between groups of people or function types, often with highly varied interests.

Transparency is not something we’re typically very good at as a general rule. If we’re going to get their organizationally and then societally, we’ve got a lot of work to do.

Organizational barriers

Within the enterprise or government, the most critical barrier to data democratization relates to functional silos. Department “A” doesn’t want to share its information with Department “B,” because that might reflect poorly on “A” or somehow advantage “B” within the corporate power structure.

A great example of overcoming this has been shared repeatedly by well-known business philosopher Clayton Christensen. Soon after he released his theory of disruptive innovation, the Secretary of Defense took note and invited Christensen to visit and share his theory with them.

The end result was the creation of a more independent functional body to combat terrorism that had shared information at its core. Indeed, these same principles have led government—perhaps most especially the intelligence community—to democratize its data at high rate of speed.

SEE ALSO: Why is Big Data security so difficult?

The secret sauce? Identifying joint or mutually beneficial interests between silos. More often than not, and perhaps most especially in the enterprise, this likely has to go beyond business metrics or investor interests. The company mission statement alone won’t get you there.

Other notable barriers within organizations include infrastructure constraints (my system doesn’t speak your system’s language) and resources for analysis. The latter is particularly key—if an organization wants to lay all of its data on the table for everyone to see, that’s great. But without the ability to rapidly analyze and respond to insights from data, democratization is wasted.

On the upside, the former is rapidly being addressed by IT Transformation and machine learning-based infrastructures that will un-stop data flow.

Societal barriers

Outside of organizations, there are considerable societal barriers to data democratization as well.

The scientific community has had to cope with this for many years across many fields of study. Data is shared and published and peer-reviewed, but that doesn’t necessarily mean that data is intelligible or taken as fact. Constant controversy around topics like climate change, especially during times of political wind change, or the firewalling of studies and information that are of public concern are challenging to the idea of democratization.

At the root of this is our tendency toward being an emotionally driven rather than logically driven. All the data in the world might be shared, but if what is shared challenges our emotions or belief system, we’ll naturally resist.

Again, the key to overcoming this seems to be in finding unifying purpose to motivate data sharing. It’s a controversial example, but initially post 9/11, the Patriot Act wasn’t overly controversial. This attitude has shifted in the ensuing years, but it shows the power of a trigger moment. We in the U.S. were fine with data privacy being lessened to forward the common good.

SEE ALSO: Protect your data from unexpected cloud failure

Certainly, there are indications that societal wariness of data sharing is beginning to wane. Our willingness to allow our data to be used to provide better services from the likes of Amazon, Apple or Google is a clear anecdotal example of the other key to overcoming the societal data democratization barriers: giving people stuff that they want.

The future

It’s clear that the tools to mine and harness data at a global scale are well at hand. What remains to be seen is how long it will take organizations and societies to acclimate and leverage these tools. If our appetite for fast service and fast information remains as high as it is today, true, widespread data democratization may happen sooner than any of us think.

Rick Delgado
Rick Delgado is a freelance technology writer and commentator. Connect with him on Twitter @ricknotdelgaldo.

Inline Feedbacks
View all comments