A tour of cloud computing: “Cloud-neutral adds a large amount of complexity and risk to a migration, without really solving the issue”
Cloud computing is worth exploring; this is what we think but of course, we’re no experts. Therefore, we decided to invite 12 experts to weigh in on the present and future of cloud computing. Our next guest is Peter Meulbroek, Global Head of DevOps Solutions at Risk Focus.
Cloud computing is worth exploring
In last year’s JAXenter Technology Trends Survey, we asked readers about their interest in different technologies and, according to the results, the cloud was a very relevant topic for developers. As you can see in the figure below, cloud computing was the runner-up in the “General IT topics” section, after software architecture.
If you want to read more about respondents’ favorite and least favorite cloud platforms, have a look at the results. Sure, cloud computing was already very popular but these results put things into perspective for us; in 2017, respondents were more interested in cloud computing than in microservices, DevOps, machine learning, blockchain and the list goes on. That may or may not still be the case, but these results opened our appetite for everything cloud-related.
Despite cloud computing’s popularity, there are still a lot of unknowns, misunderstandings and gaps. For example, earlier this year, we learned from Sumo Logic’s 2018 Global Security Trends in the Cloud report that almost half of their respondents reported that current tools do not work in the cloud. Furthermore, a whopping 97% out of the 300+ respondents felt that they lacked the tools for proper cloud security. Read more about the report here.
The bottom line is that cloud computing is worth exploring and the benefits definitely outweigh the risks. This is what we think but of course, we’re no experts. Therefore, we decided to invite 12 experts to weigh in on the present and future of cloud computing.
A tour of cloud computing will be published twice a week.
Here are the interviews published so far
Our next guest is Peter Meulbroek, Global Head of DevOps Solutions at Risk Focus.
JAXenter: Everything is in the cloud these days, including our precious data. How can developers maintain an appropriate level of security in an increasingly insecure landscape?
Peter Meulbroek: Security is becoming increasingly important to the enterprise, regardless of data location. The key to understanding security in the cloud is to realize that the old, obsolete paradigm for security — the perimeter defense — has gone the way of the curtain wall and needs to be replaced with defense in depth. Nor is it enough to manage data security between applications. Data, at rest or in flight, needs to be protected at all levels within an application, and managing security for an application is largely managing access to decrypt narrowly-focused cohorts of data. The security of keys becomes of vital importance in this context.
One of the common transitional use cases in the Cloud adoption that we see frequently is using a public cloud for development and lower environments and running production on premise. Data security and privacy in lower environments is of key concern in this migration. Several of our clients have opted to mask data in these lower environments to achieve both regulatory compliance and to reduce the risk of hacking. That in combination with Data Virtualization solutions is a safe, compliant way to start using the cloud and enjoy the acceleration that it provides without doing the full organizational lift.
JAXenter: What benefits does a cloud-based infrastructure bring? What are the drawbacks?
Peter Meulbroek: We are unabashed boosters of cloud-based infrastructure. Such an infrastructure does not need to be public-cloud based, though increasingly often it is public-cloud-based for many enterprises. A cloud-based infrastructure shares several fundamental characteristics.
- It is automated, so that basic actions in the cloud are scripted, and can be encapsulated. This allows for operational activities to be regularized, and for the enabling scripts to be tested and versioned, increasing reliability
- It provides a degree of self-service, so that consumption fits the needs of the end-user, not a facilitating group or organization. This can short-circuit long, laborious hand-offs, allowing for increased velocity.
- It presents abstract building blocks (low-level, such as storage, compute, and network; often high-level, such as container or function management) that can be deterministically assembled to rapidly create unique solutions. This allows a great deal of standardization, cross-training, and allows most applications groups to focus on business value
- It is elastic, so that solutions can grow over time. This allows parallelization in development / validation of applications and minimizes over-provisioning
- It is monitored, providing health, utilization, and business-case metrics. This allows better diagnostics and maintenance
- Finally, it is reactive, so that anticipated error conditions can be managed using developed and tested automations.
The downside of cloud-based infrastructure is that full-utilization of the above characteristics requires more than technical changes, the processes and culture of an organization need to be cloud-aware. Such changes are often more difficult than the technical changes.
JAXenter: What is your favorite cloud-based tool, service, or platform to use and why?
Peter Meulbroek: We hesitated whether to answer this question, not because we don’t have favorites, but because we find that picking tools frequently dominates the conversations with our clients, when we should be focusing on the business problem that motivates the movement to the cloud.
We believe that most large organizations will end up being multi-cloud, but not necessarily cloud-neutral.
We are reasonably agnostic about technology at Risk Focus; the number of excellent tools continues to grow. Though we maintain a wide expertise in tools, there are a few that we focus on, as much for the strong organization behind the tool as well as the quality of the tool. The listing of tools below is selected, because any cloud effort touches all these functional layers:
For configuration management, we often depend on Salt from Saltstack or Ansible from Red Hat. Salt allows a huge amount of flexibility, scaling, and validation, especially in autonomous environments. We find Ansible to be an excellent choice in environments that need to be fast-to-market but don’t require the complexity that Salt enables.
For orchestration, we prefer to be cloud-native. For environments where that isn’t optimal or possible, we love Hashicorp’s Terraform, as well as the rest of Hashicorp’s suite; it is tight, well-made, and becoming increasingly popular.
For application packaging and distribution, there is nothing better than containers. And there is no doubt that the best container orchestration tool out there is Kubernetes.
For data masking and data virtualization we like Delphix.
For monitoring and reaction, we often rely on InfluxData’s TICK stack. More than a time-series database, TICK allows us to accept either push or pull metrics, fine-grained control over storage and retention, and aggregation / alerting capabilities. This can be easily coupled with Kubernetes or with Salt/Ansible to create a reactive, self-healing or auto-scaling solution as needed.
JAXenter: Is Kubernetes becoming central to cloud adoption?
Peter Meulbroek: Kubernetes brings obvious and compelling advantages to development, validation, and application lifecycle management. We are heavy users of containers and embrace them for a wide variety of use cases. That being said, Kubernetes is complex, especially when managing non-purpose-built containerized apps. Cloud-native Kubernetes offerings can handle some of that complexity, but any organization taking the adopting containers as a deployment and management strategy needs to consider this complexity in their staffing and development plans.
One of the common use cases that Kubernetes is great at is the one we discussed earlier: lower environments in the Cloud, production on premises. This is frequently one of the best ways to start the Cloud journey. It also provides the only really interesting option for cloud-agnostic deployments that we have seen.
Peter Meulbroek: Almost every time we start an engagement with an organization, they want to be cloud neutral. We think this is very much the wrong place to start. There are times when this is the right position, but mostly it stems from a position of fear of picking the wrong cloud; or a fear of lock-in. Cloud-neutral adds a large amount of complexity and risk to a migration, without really solving the issue. It requires a great deal of sophistication that is premature for the organization just starting their cloud journey.
Analogous to multi-threaded code, serverless functions can be a real challenge to test, debug and monitor.
We believe that most large organizations will end up being multi-cloud, but not necessarily cloud-neutral. The goal is to adopt the cloud in a way that brings immediate value to an understood problem (e.g. increased velocity, lower costs, etc) and to do it securely. This involves addressing organizational challenges that allow cloud benefits to accrue; solving this problem across multiple clouds at the same time is close to impossible. Our advice is: do it once with one cloud. Then do it for another as needed (it will be easier as you will know more) and then look for generalization if required.
We have specifically developed an engagement model called the Player-Coach that is geared towards helping enterprises get started. We find that most of our clients know at a high level what they want to do but are challenged by the complexity of the implementation. We address this challenge by always providing a tangible delivery but help inform appropriate choices along the way.
JAXenter: If cloud technology wants to continue to grow, tools should grow and adapt as well. What are the most mature tools right now?
Peter Meulbroek: The public clouds are quite mature and have compelling offerings. The rate of innovation by all the major vendors challenges our customers’ abilities to absorb the technology. The challenge is not one of tooling but of problem definition: we see that most organizations are solving “yesterday’s problem” with the new tools.
The tools we listed above all fall into the mature category.
JAXenter: How can we capture the multi-cloud opportunity? What are the roadblocks to multi-cloud success?
Peter Meulbroek: As we discussed above, there is a difference between multi-cloud and cloud-agnostic. We think most big enterprises will leverage the former and few have been successful in the latter.
The challenge is not one of tooling but of problem definition: we see that most organizations are solving “yesterday’s problem” with the new tools.
It is quite complex to design a set of processes for an organization that does not use the cloud to begin its adoption. It is much more complex to design systems that allow clouds to be spanned. Though the representation of basic services and API abstractions are similar amongst the major cloud vendors, this similarity starts to break down for more complex offerings such as those around data management, streaming, and analysis. Though huge progress is being made, the ‘cloud-agnostic’ remains an unfulfilled promise.
On the other hand, we see many organizations that effectively leverage offerings from multiple clouds. This still requires common governance and some common tooling but is substantially simpler.
JAXenter: What do you think of serverless? Is it a “revolution of the cloud,” as Maciej Winnicki, Principal Software Engineer at Serverless Inc. told us last year?
Peter Meulbroek: Serverless functions are an extremely powerful abstraction, allowing for very lightweight microservice architectures without the overhead of managing a fleet of app servers. Development can be very rapid, as can the SDLC. However, serverless suffers from some very significant drawbacks. Analogous to multi-threaded code, serverless functions can be a real challenge to test, debug and monitor.
Serverless applications need careful thought around logging, tracing, and validation, and so serverless works best in a re-architecture, rather than refactor mode. As with most other Cloud technologies: let’s not look for the one that will rule them all, but rather enjoy the world of plenty and look for the technology that best solves a specific problem. Then we move to the next.
Application developers have entered a new era with the advent of cloud technology. If you want to meet the movers and shakers in the world of cloud computing, don’t miss JAX London, a four-day conference taking place October 8-11, 2018.