Mind the gap

Have we bridged the gap between Data Science and DevOps?

Richard Gall
Data Science
© Shutterstock / PorqueNo Studios

How do Data Science and DevOps fit together? In this article, Richard Gall explains why integrating Data Science with your DevOps can lead to a better and smarter business.

Packt’s Skill Up 2016 survey revealed Data Science and DevOps to be big news for developers. JavaScript and Python, essential programming languages for data scientists, reigned as most popular, whilst DevOps made it into the biggest trends list. As we begin analyzing the data from our Skill Up 2017 survey, it will be interesting to see if this is still the case.

Data Science and DevOps

The big question, however, is how Data Science and DevOps fit together. Despite Data Science’s prevalence, Gianmario Spacagna, a data scientist for Barclays Bank in London, told Computing magazine at Spark Summit Europe in October 2015 that, in many instances, there’s not enough impact from Data Science teams. His suggested solution? Build a bridge between Data Science and DevOps:

If you’re a start-up, the smartest person you want to hire is your DevOps guy, not a data scientist. And you need engineers, machine learning specialists, mathematicians, statisticians, agile experts. You need to cover everything otherwise you have a very hard time to actually create proper applications that bring value.

Two years on, his idea still makes a lot of sense. It’s become clear over the past few years that ‘data’ itself isn’t enough; it might even be distracting for some organizations. Sometimes too much time is spent in spreadsheets and not enough time is spent actually having an impact. Real value comes from making decisions, building relationships, and building things.

Many organizations are still strategically flawed, in the sense they don’t know how to utilize Data Science. Instead of placing too much focus on what data they have and what they can get, there should be more emphasis on who can access it and what they can do with it. If Data Science isn’t joining these dots, DevOps can help by providing practical solutions, such as building dashboards and creating APIs.

SEE MORE: 6 best places to learn data science fast

Solutions such as this immediately give data additional value by making it more accessible and simply more usable. Even for SMEs, data scientists and analysts will have minimal impact if the organization fails to successfully integrate them into the business’s wider culture.

This approach also has an impact on how organizations think of customer experience. I always go back to this brilliant post from Airbnb. The type of ‘DevOps thinking’ that emerges is a relentless focus on customer experience. By this, I don’t simply mean that the work done by the Airbnb engineers is specifically informed by a desire to improve customer experiences. Instead, it’s the sense that tools through which internal collaboration and decision making take place should actually be similar to a customer experience; elegant, engaging, and intuitive. This doesn’t mean seeing every relationship as purely transactional, based on some perverse logic of self-interest, but rather having a deeper respect for how people interact and share ideas. If DevOps is an agile methodology that bridges the gap between development and operations, it can also help to bridge the gap between data and operations.

I can’t claim credit for inventing ‘DataOps’, and to be honest, it’s simply another buzzword for the managerial class. We should be careful not to create another gap between Data and Development. It simply doesn’t make sense in the world we’re building with software today. Even for web developers and designers, the products they are creating are so driven by data that separating the data from the development seems absurd.

SEE MORE: Flexibility – how to make data science in banking work

Are we asking enough about how Data Science can inform DevOps? It’s about opening up a dialogue between these different elements. While DevOps evangelists might argue that DevOps has already started that, the way forward is to push for more dialogue, more integration and more collaboration.

As the API economy becoming even more important to the success of both startups and huge corporations, the relationships between all these different areas are going to increase in complexity. Let’s build better and build smarter by talking more. DevOps is a good place to start the conversation, but it’s important to remember it’s just the start.

Richard Gall
Richard Gall is Communications Manager at Packt, utilising his background in writing and publishing to help Packt in their mission to help the world put software to work in new ways. Packt, one of the biggest tech publishers in the world, deliver effective learning and information services to IT professionals.

Inline Feedbacks
View all comments