“Open source is where innovation happens today”
Cloud technologies are changing the IT landscape and companies that want to implement a cloud strategy must face the challenges that go hand in hand with this transformation. Apache jclouds, an open source multi-cloud toolkit for the Java platform, helps fill some of the gaps in the cloud landscape. We asked Ignasi Barrera, member of the Apache jclouds Project Management Committee to identify the benefits of jclouds and to clarify what’s under this project’s hood.
JAXenter: What is the idea behind jclouds? Which gap does it fill in the cloud landscape?
Ignasi Barrera: Apache jclouds is an open source multi-cloud toolkit for the Java platform that gives developers the freedom to create applications that are portable across clouds while giving them full control to use cloud-specific features.
Its main purpose is to avoid vendor lock-in by providing common interfaces and patterns that can be used to write code that works with all supported cloud providers, allowing developers to write it once, run it anywhere, and manage any cloud. This gives jclouds users the freedom they need to use whatever cloud better fits their needs (high availability, performance, cost, privacy, etc) without having to deal with the complexity behind them.
JAXenter: Paas, IaaS, SaaS and so on: Where would you place jclouds?
Ignasi Barrera: jclouds resides in the IaaS space. It provides some configuration management features, like the integration with Chef, that allows users not only to provision infrastructure but also configure application specific things such as lifecycle, integration with other components in distributed systems, etc. At its core, though, jclouds is an IaaS toolkit mainly focused on compute and storage.
JAXenter: Tell us more about what’s under this project’s hood: How does jclouds work?
Ignasi Barrera: jclouds is all about abstractions for cloud features, and abstractions are hard. Supporting many different cloud providers, with very heterogeneous offerings, is challenging. One tends to design interfaces with the minimum set of common functionality, but this approach is often too poor to meet basic use cases. In order to provide more flexibility while keeping code portability as a first class citizen, jclouds has a layered architecture that provides different levels of abstraction and detail.
Apart from running against most of the existing cloud providers, jclouds is also designed to run anywhere.
The provider and API layers provide implementations for all the supported cloud provider APIs with a set of fully featured interfaces. The abstraction layer consists of portable models and interfaces that hide the details and complexity of each provider and can be used to talk to any supported cloud. Users have the freedom to choose how portable they want their code to be and where to put the balance between highly portable code and control over cloud-specific features or configurations. They can choose to write code that uses the portable abstractions, the provider-specific APIs, or combine both.
Apart from running against most of the existing cloud providers, jclouds is also designed to run anywhere. It has a small dependency footprint and provides a driver mechanism that makes extremely easy to configure jclouds to use different HTTP drivers, logging frameworks, security and SSH libraries, etc, and even change the jclouds behavior at runtime by providing custom implementation classes for almost any jclouds feature.
JAXenter: Can you describe a typical use case where the benefits of jclouds shine through?
Ignasi Barrera: The real power of jclouds can be seen in multi-cloud environments. It provides a perfect foundation for PaaS or application orchestrators where the main entity is the application. Apache Brooklyn is a great example of such a use case. It uses jclouds as its IaaS layer to provision and configure the underlying infrastructure. This way it does not have to deal with most of the low-level details and can focus on application lifecycle, scheduling, monitoring, and so on.
There are also interesting use cases for hybrid cloud, where jclouds allows managing private cloud infrastructures, containerized environments or public clouds using the same code and portable interfaces, no matter where their infrastructure is or the technology behind them.
JAXenter: Tell us about the history of jclouds. How did the project begin?
Ignasi Barrera: Adrian Cole, the creator of jclouds, worked at Infinispan on 2009. Part of his job there was developing a connector for Amazon S3. During that development he realized that the existing libraries did not meet the needs of his use case, so he started working on a specialized cloud driver. That driver started as jclouds and quickly became a more generic interface to cloud storage. By the end of 2009, some compute features were added and a new portable interface was created. Over the years other cloud APIs such as DNS, load balancer or SDN came into the project, consolidating it as the cloud toolkit for Java.
JAXenter: What are your future plans for the project?
Ignasi Barrera: Well, there is no “me” in Apache projects. The community and the Project Management Committee (PMC) control the direction of the projects, and all decisions are made by consensus in public discussions. But speaking about my preferences, I’d like to see jclouds having more and better abstractions for things like SDN, or better support for containers and container orchestration frameworks.
The adoption of the cloud has brought to the masses technology that was only available to big companies.
JAXenter: In your view, how do cloud technologies change IT?
Ignasi Barrera: Nowadays any startup can build applications that process huge amounts of data, do deep learning, or execute computation intensive tasks. Anyone can build applications at scale. The adoption of the cloud has brought to the masses technology that was only available to big companies, and those companies have also played an important role in adoption by open sourcing internal software, publishing papers and research, and contributing and collaborating with others to create and evolve that technology.
Physical resources are no longer the limit, and this availability of resources, the amount of data to process, and the need for scalability and high availability have changed and shaped how applications are designed and developed today. We have been seeing a massive movement towards distributed architectures, microservices, and now we are also seeing serverless architectures. The entire industry is changing the way it produces and embraces technology.
JAXenter: Why is the cloud important for DevOps?
Ignasi Barrera: With the new way of developing applications, developers need to be more aware of where and how applications are deployed, and better understand how the different components of the application interact with other systems. In order to build applications at scale, resilience and fault tolerance must be taken into account since the early stages of development, and that not only requires programming knowledge, but also understanding the environment where applications will run, and how to monitor, profile and troubleshoot them.
JAXenter: What do you think is an example of a positive trend in cloud computing?
Ignasi Barrera: There are open source frameworks such as Spring Cloud, the Netflix OSS stack or Lagom that are consolidating as the way to build distributed applications. In the big and fast data world, projects like Apache Beam, Apache Spark or Apache Kafka have some of the biggest companies in the world contributing to them, and the same can be seen in other areas related to the cloud. If you look at the most relevant projects for the cloud today, you will realize that the common denominator is that most of them are open source projects.
One of the biggest challenges in moving to the cloud is the mental change to a completely different way of interacting with the infrastructure resources and applications.
Open source is where innovation happens today. Companies have understood the benefits of the open source model and have seen that it is a great way to create technology. They are adopting it and contributing more and more, and this gives a lot of potential to open innovation and to create the technology of the future.
JAXenter: What are the challenges for companies that want to implement a cloud strategy? What’s your take on that?
Ignasi Barrera: One of the biggest challenges in moving to the cloud is the mental change to a completely different way of interacting with the infrastructure resources and applications. It can give the perception of losing control for those that are used to manage on-premise environments because the only way to interact with them is through APIs or web consoles. Privacy is also one of the big challenges companies have to face if they want to move to the cloud. How cloud providers treat data, especially sensitive data, is crucial for business, and has to be evaluated very carefully before adopting a cloud strategy.
Thank you very much!