AI and AIOps: Top 5 Technology Trends for 2020
What will 2020 look like in terms of AI and AIOps? I thought I’d break this down into three distinct areas: technology, market and users. For each of these I’ve come up with five major trends which I think will have significant impact on how AIOps, and AI in general, evolves over the next twelve months. At the outset I’d like to say that the most significant changes will be seen in user behaviors and user practices, this is the year where we will see substantial transformation. But, let’s start with the five technology trends…
I believe ITOps and security as disciplines, and sets of technology, are on the verge of converging into a single continuous technology set. That convergence is framed and driven by AI technologies and for me this can be defined by five algorithms: data selection, pattern discovery, automated inferencing, communication (taking the results from data selection, pattern discovery and automated inferencing, wrapping them up and communicating them to a human being) and finally robotics (again take the first three algorithms and instead of communicating to a human being you communicate to a machine). These algorithms are tying together the realms of IT Operations and security management.
I think many incidents will not emerge as security problems or IT Operations problems until late in the process, algorithms will be running but it won’t be until the final stages of execution that the problem will be assigned to a particular discipline. AI will finally enable the convergence of ITOps and security.
Humans will trust AI with more
The next technology trend is AIOps specific. Until now when we’ve looked at market ready AIOps technologies, enterprises have largely used them to enhance existing activities and processes. The assumption has always been that the AI system would always be serving the needs and responding to the requests of a human operator – acting as an extension of that human operator’s cognitive ability.
I think we will see a real shift away from an enhancement model to a delegation model, where humans will begin to trust AI systems to work independently. One of the main reasons why is the complexity of IT systems, but also the timescales in which things need to be done are becoming shorter and shorter. The amount of time available from when an incident begins to emerge and can have an impact on business activity or user experience, is getting smaller and smaller. The need for an observation and response cycle to move quickly is growing and organizations can no longer afford human latency. This trend is a response to necessity.
Now, what about neural networks? Today, neural networks dominate the landscape of AI and AIOps, the question I pose is whether this is justifiable and sustainable. Let’s look at this commercially. Within the context of AIOps, neural networks have peaked in their ability to deliver effective and meaningful results. There are a number of limiting issues that relate directly to neural network algorithms, and it is my belief that these cannot be changed. All of which, prevent the widespread deployment at scale of neural networks in IT Ops management environments. It is my conviction that even though there is a lot of investment currently being made in neural networks, both within enterprise and vendor communities, it’s bearing on AIOps and the real-time business community will ultimately be limited.
Machine learning algorithms
The fourth trend relates to Machine Learning algorithms, which are a subset of pattern discovery algorithms, and can be broken down into two parts. The first part is how you create a model of the environment you are interested in – this part of the algorithm describes the structure that you’re trying to discover. Then there are the processes that gets you from the available data to the model. If you have a dataset, you need to know what that dataset is describing, so you have a set of functions or algorithms that teaches and then takes you from the data to the object which it purports to describe, but you must also have a model of that object.
Until now the object which has been described has largely been some kind of probability distribution – the Machine Learning dilemma is which probability distribution does the data I have tell me what I’m looking at. So probability distribution has been the object which your various Machine Learning algorithms are trying to help you discover in the data.
Topological data analysis
There are two such objects I believe will receive a lot of attention in 2020. One comes from the field of topology, there is an increasingly important area called topological data analysis which is basically like Machine Learning but instead of having probability distributions as the objects of discovery it has topological shapes. Why the interest? Well, the topological shapes themselves can be used to model all sorts of things – not just shapes, but different types of events or maps or spatial relationships, which is more difficult if your using probabilistic models. This will be very important for AIOps.
Game theoretic analysis
The second object is when you look at how people interact with one another when they’re playing a game, we become interested in what strategies players deploy and just as you can go from data to probability distributions, or data to topological spaces, you can also go from data to games and the strategies that govern games. We call this game theoretic analysis – it allows you to infer strategies and structures from the data that’s presented to you. This could become very valuable in trying to apply AI to security use cases.
The fifth trend pertains to the world of DevOps. In DevOps there’s a lot of automation and a focus on moving code from one stage to the next, with the aim of injecting that code into a production environment. Most of the automation relates to automating the production line. What were likely to see emerge in 2020 is the automation of many aspects of code writing, this isn’t replicating existing lines of code but relates more to making decisions about flow and what operations to apply – effectively automating the coders task. I think this is a further stage in the evolution of DevOps.
Those are the five technology trends, next time we will have a look at the market…