Full-stack experimentation adds missing elements to agile, DevOps, and CI/CD
We’ve already debunked the myth that DevOps is a magic bullet but we haven’t covered the missing elements in this movement. We invited Kavitha Mariappan, CMO at Split to talk about full-stack experimentation, why it’s a game-changer and how profound its impact is on a business.
JAXenter: What is Split Feature Experimentation Platform and what is its goal?
Kavitha Mariappan: The Split Feature Experimentation Platform is a unified solution for continuous delivery and full-stack experimentation. Split unifies DevOps and product management, helping agile engineering and product teams accelerate the pace of product delivery and make data-driven decisions. Split does this with robust feature flagging and extensive experimentation capabilities. With Split, organizations can now accelerate time to value, mitigate risk, and drive better outcomes, all in a unified platform.
JAXenter: What is the most important feature of this platform?
Kavitha Mariappan: At the foundation of the platform, Split uses feature flags to control the rollout state of actual product features, anywhere in the stack, via an SDK to turn the feature on or off, or to show multiple variations of the feature.
The Split Intelligent Results Engine is the most important feature that builds on the feature flag foundation. Split provides a robust statistical engine built from the ground up to ingest product and operational metrics helping teams drive to outcomes across the organization. Split’s results engine is metrics-first, automatically highlighting significant changes due to feature releases across the metrics that matter most.
Feature flags provide the enabling technology that makes it possible to run true feature experiments. The results engine delivers the functionality on top of feature flags needed to make better product decisions.
JAXenter: How can this platform help accelerate product delivery?
Kavitha Mariappan: Split unifies DevOps and product management, helping agile engineering and product teams accelerate the pace of product delivery and make data-driven decisions, through its robust feature flagging and extensive experimentation capabilities. With Split, organizations can now accelerate time to value, mitigate risk, and drive better outcomes, all in a unified platform.
Details on how?
- Accelerate time to value: Release fast and often, by unblocking dependencies that exist between feature releases.
- Mitigate risk: Minimize the blast radius of any feature release incident by gradually rolling out features across the customer base using feature flags.
- Make smarter decisions: Measure and understand the impact of every new feature on the product and operational metrics that matter.
JAXenter: Split Feature Experimentation Platform aims to help teams treat continuous delivery and experimentation as two sides of the same coin. What does that mean in terms of team achievements? Is it a game-changer?
Kavitha Mariappan: Continuous feature experimentation is essentially the completion of a journey that started with agile development and has grown through DevOps, continuous integration, and continuous delivery. Full-stack experimentation adds missing elements to Agile, DevOps, and CI/CD to allow those practices to move from the world of development to the world of data-driven product development. With agile methods, companies are able to add and evolve features to software faster.
With full-stack experimentation, it is possible to compare features and evaluate their impact on the business. Full-stack experimentation works by allowing code to be inserted in an application in chunks that can be turned on and off. This is a practice known as feature flagging. These changes can then be distributed throughout different parts of an application.
Engineer time is used more effectively since PMs do not need to bother engineers with the less value-added work of reconfiguring how features are exposed to customers for experimentation.
It definitely is a game-changer. The impact it has on a business is profound: companies don’t have to guess about whether a feature or new process works — they have clear metrics to assess effectiveness. At a team level, scrum teams can work more efficiently and effectively than ever. The reasoning behind decisions made by a product owner is made more clear than ever to engineers and PMs are empowered with clear data that helps them minimize randomization of the team’s efforts.
Engineer time is used more effectively since PMs do not need to bother engineers with the less value-added work of reconfiguring how features are exposed to customers for experimentation. Overall, the result is that the entire business can then test more ideas than ever before, quickly discard the ineffective ones, and embrace those that work the best. From this, the company embraces a culture of testing in which a greater number of ideas leads to greater innovation.
DevOps metrics and tools
JAXenter: How important are (well-defined) metrics to the success of a DevOps strategy?
Kavitha Mariappan: As organizations grow their software products and digital offerings beyond an initial feature set, it becomes increasingly challenging to show how recent feature investments impact the business. Enterprises need access to real-time analytics on how customers are interacting with these features. And these insights need to relate back to the full set of product KPIs.
It is not enough to simply report on website conversions. The impact of new features must be measured against low-level metrics, such as exceptions or page load times. Metrics that measure the overall customer experience are critical as well, such as the number of actions or time spent in the application. And, ultimately feature changes should tie all the way back to the key business metrics, such as revenue or units purchased.
To have confidence in product decisions, all the conclusions from the data must be statistically sound and trusted.
JAXenter: What tools should be part of a DevOps toolkit?
- Multi-language feature flagging: The rise of microservices as an application design pattern means that many different programming languages may be used to create one application.
- Open source SDKs: The modern DevOps toolkit must be able to be used in all popular open source language environments.
- Continuous delivery: The modern DevOps toolkit should support the automation systems needed for CI/CD.
- Metrics and analytics: In-depth metrics and analytics are a must, as they allow you to not only see the effectiveness of a given change but also how that change affects the performance of your larger network. The metrics must track the experiment but also the performance of the application at large.
- Unified platform and management console: A single place from which teams can manage all elements of experimentation and assess results. You should be able to control all of these elements within your full-stack experimentation platform from a single place so that the process flows smoothly.
- Sophisticated segmentation: Tracking user data, behavior, and other ways of segmenting users don’t happen in the full-stack experimentation platform. But the platform must be able to turn experiments on and off and track experimental data based on all available segments.