The New Stack Podcast

What LaunchDarkly Learned from 'Eating Its Own Dog Food'

Episode Summary

Feature flags — the on/off toggles, written in conditional statements, that allow organizations greater control over the user experience once code has been deployed — are proliferating and growing more complex, and demand robust feature management, said Karishma Irani, head of product at LaunchDarkly, in this episode of The New Stack Makers. In a November survey by LaunchDarkly, which queried more than 1,000 DevOps professionals, 69% of participants said that feature flags are “must-have, mission-critical and/or high priority” for their organizations. “Feature management, we believe, is a modern practice that's becoming more and more common with companies that want to deploy more frequently, innovate faster, and just keep a healthy engineering team,” Irani said. The idea of feature management, Irani said, is to “maximize value while minimizing risk.” LaunchDarkly uses its own software, she said, and eating its own dog food, as the saying goes, has paid off in gaining insights into user needs. As part of LaunchDarkly’s virtual conference Trajectory in November, Irani joined Heather Joslyn, features editor of The New Stack, for a wide-ranging conversation about the latest developments in feature management. This episode of Makers was sponsored by LaunchDarkly. Karishma Irani - @karishma_irani Heather Joslyn - @ha_joslyn The New Stack - @thenewstack

Episode Notes

Feature flags — the on/off toggles, written in conditional statements, that allow organizations greater control over the user experience once code has been deployed —  are proliferating and growing more complex, and demand robust feature management, said Karishma Irani, head of product at LaunchDarkly, in this episode of The New Stack Makers.

 

In a November survey by LaunchDarkly, which queried more than 1,000 DevOps professionals,  69% of participants said that feature flags are “must-have, mission-critical and/or high priority” for their organizations.

 

Feature management, we believe, is a modern practice that's becoming more and more common with companies that want to deploy more frequently, innovate faster, and just keep a healthy engineering team,” Irani said.

 

The idea of feature management, Irani said, is to “maximize value while minimizing risk.”

 

LaunchDarkly uses its own software, she said, and eating its own dog food, as the saying goes, has paid off in gaining insights into user needs.

 

As part of LaunchDarkly’s virtual conference Trajectory in November, Irani joined Heather Joslyn, features editor of The New Stack, for a wide-ranging conversation about the latest developments in feature management.

 

This episode of Makers was sponsored by LaunchDarkly.

Automating Approvals

As an example of the benefits of having first-hand knowledge of how their company's products are used, Irani pointed to an internal project in mid-2022.

 

When the company migrated from [sponsor_inline_mention slug="mongodb" ]MongoDB[/sponsor_inline_mention] to CockroachDB, it used new capabilities in its Feature Workflows product, which allow users to define a workflow that can schedule the gradual release of a feature flag for a future date and time, and automate approval requests.

 

“All of these async processes around approvals schedules, they're critical to releasing software, but they do slow you down and add more potential for manual error or human error,” Irani said. “And so our goal with Feature Workflows was to essentially automate the entire process of a feature release.”

Overhauling Experimentation

This past June, the company also revised its Experimentation offering, she said. Led by James Frost, LaunchDarkly’s head of experimentation, the team did “a complete overhaul of our stats engine, they enhanced the integration path of our customers’ existing data sets and metrics,” Irani said. “They redesigned our UX and the codified model and experimentation best practices into the product itself.”

 

For instance, a new metric import API helps prevent the problem of multiple teams or users within a company using different tools for A/B and other experiments. It “significantly cuts down on manual duplicate work when importing metrics for experimentation,” said Irani. “So you can get set up faster.”

 

Another addition to the Experimentation product is a sample ratio mismatch test, she said, so “you can be confident that all of your experiments are correctly allocating traffic to each variant.”

 

These innovations, along with new capabilities to the company’s Core Flagging Platform, are in general availability. On the horizon — and now available through LaunchDarkly’s early access program, is Accelerate, which lets users track and visualize key engineering metrics, such as deployment frequency, release frequency, lead time for code changes, and flag coverage.

 

“I'm sure you've caught on already,” Irani said, “but a few of these are Dora metrics, which obviously are extremely critical to our users.”

 

Check out the entire episode for more details on what’s new from LaunchDarkly and the problems that innovators in the feature management space still need to solve.

Episode Transcription

Alex Williams  0:08  

You're listening to the new stack makers, a podcast made for people who develop, deploy and manage at scale software. For more conversations that articles go to the new stack dot I O. All right now on with the show

 

Colleen Coll  0:29  

unleashed developer productivity for the software powered world by fundamentally changing how you deliver software to your customers. With LaunchDarkly ease feature management platform, empower developers can empower the business to release new features faster, and more efficiently than ever.

 

Heather Joslyn  0:49  

Hello, everyone, and welcome to another episode of the new stack makers podcast. I'm your host, Heather Jocelyn Features Editor of the new stack. And our topic today is what's new in feature management. We're joined today by Karishma Irani head of product at LaunchDarkly, the feature management platform, and thanks for joining us, Karishma.

 

Karishma Irani  1:07  

Thanks for having me.

 

Heather Joslyn  1:09  

Terrific. We'd like to thank our sponsor for today's conversation, which is LaunchDarkly. And we've got a lot ahead of us. So let's just jump right in. Before we go any further, just to make sure our listeners are on the same page as we as us as we start this conversation. What is feature management? How do you define it? Sure,

 

Karishma Irani  1:26  

it's a question I get, I find myself answering at least once a week. So feature management is a relatively new concept. So if you're not familiar with it, you're not alone. Feature management is powered by feature flags. So I'm going to start with the definition of feature flags first. For anyone who is not familiar with feature flags, it's a relatively common concept. And it is feature flags are essentially conditional statements in your code that control your code in production, and essentially also allow you to manage the user experience after your code has been deployed. So essentially, you're building a piece of software, you wrap it in a feature flag, you deploy it, release it to your users. And that piece of code the conditional statement, the flag helps you control what your end user sees. The most simple version of it is an on off switch, you can use feature flags, and a lot of our customers do and a lot of our prospects to use feature flags for a simple on off switch or toggle essentially. But over the years, feature flags have become more and more advanced and unlock more advanced use cases for our customers, where we find them using them as a lever in their code to separate deploys from releases. And what this allows our customers to do is essentially test their code after it's been deployed in production to make sure that it's bug free to make sure that it works as expected, it's performant. It's not interfering with any other releases from other teams going out at the same time. And even do things like create multivariate flags. So you can have six different versions of the same feature. And using advanced targeting control who sees what or based on a certain set of rules, release version A to a set of customers version B to a different set of customers. And the rules that define these customers can be literally anything, it can be location based, it can be plant based, it can be the version of the software, it could be any attribute that you pass. And so feature flags are essentially these very advanced toggles within your code that allow you to control your end user experience. Feature management, however, is the framework the process of practicing and using feature flags at scale. And it includes the process of not just using flags to launch features, but also to control those end user experiences in real time. And finally, also measure the impact of those experiences on your users. So feature management, we believe is foundational to modern software development. And it unlocks practices like trunk based deployment, DevOps, ci CD, as, as well as progressive delivery. And furthermore, teams that practice feature management at scale are known to increase the developer productivity ship, more bug free code, do fewer rollbacks, of their software, maintain higher system performance, etc. And so feature management, we believe is a modern practice that's becoming more and more common with companies that want to deploy more frequently, innovate faster, and just keep a healthy engineering team.

 

Heather Joslyn  4:41  

And it's less risky to correct. Absolutely,

 

Karishma Irani  4:44  

absolutely. And certainly our vision for feature management is to help our customers maximize value while minimizing risks, which a lot of companies think are contradictory,

 

Heather Joslyn  4:56  

because we're definitely you're asked more of developers faster and innovate faster and so on. And it's sort of a, I've heard it described as sort of a guardrail of sorts, you know, to ensure code quality as you are asking people to go faster.

 

Karishma Irani  5:10  

You hit the nail on the head with the term guardrail when I speak to our customers. And I'm like, So tell me, why do you think you need feature management? They describe it as almost needing a guardrail, which could be a metric, it could be an observability metric, it could be a business metric, or it could be a performance based operational metric, but having some sort of metric that determines whether they should proceed with the feature release or not. And feature flags essentially allow you to have that

 

Heather Joslyn  5:37  

what are some of the key pain points that developers and engineers might encounter with feature management?

 

Karishma Irani  5:43  

Right? By adopting feature management, it's actually really interesting. We recently released our state of feature management report, which is an independent and objective report, where learnings and insights that we put out there every year, and one of the insights that we learned from this year's survey of over 1000, DevOps practitioners was that adopting feature management can initially seem a little tedious and slow the team down, but over time, it actually speeds your team up and accelerate the rate of your innovation. And that can seem counterintuitive at first. So take, for example, a team that today is not using feature flags or not using a feature management platform like LaunchDarkly. In order to release their software, they simply deploy it to production, and it's been released to all of their customers, it's relatively straightforward. When using or when considering adopting a feature management platform, you have to put certain it's an additional tool, it's an extra step in the process, right or can seem at first where you have to define custom roles and permissions for who you want to have have access to release into certain environments, critical environments like production, you have to start flagging your code or pieces of your code in these conditional statements that are described as feature flags, you now need to work with your product manager or your release managers to let them know which flag controls what piece of software. And and essentially separating out or decoupling deploys from releases can seem like an unnecessary step at first. And so those are some challenges that our users run into when first adopting or considering adopting feature management. However, unanimously, the sentiment that we hear from our customers, once they've started using it and have been using it for a month or so is, I can't believe or I can remember how I used to ship software and a pre LaunchDarkly world. And it's it's kind of that sentiment, where when you're introduced to a new, modern practice of software delivery, when you're introduced to Ci CD, you can't go back to a pre CI CD world and LaunchDarkly is that component or feature management is that component, that kind of feels the same way where initially it might seem like breaking one step, which is deploying or releasing it from a single stage into two stages. But by decoupling deploys, from releases, you can gain so many benefits like testing and production, like kill switches, like running experiments, like delivering different experiences, etc.

 

Heather Joslyn  8:28  

Yeah, it's a new processes and new set of skills to learn. But that's kind of true about every, every new new way of working, let's talk about some of the new things that LaunchDarkly is rolling out this fall. And let's start with feature workflows. What are the key updates for that? And how do they solve problems for the user.

 

Karishma Irani  8:42  

So a really fun statement that I find myself saying pretty often is LaunchDarkly uses LaunchDarkly, to build and ship LaunchDarkly. And that's a mouthful, but it essentially means that we we eat our dog food, we dog food, our product constantly. And as a result of that a lot of our inspiration for our product roadmap actually comes from using the product ourselves. And so feature workflows is one such idea or one such concept where you have the set of toggles, right you have the set of flags, you have all of these rules, all of these different experiences you're controlling. And let's say your team's been building a feature for the last three weeks, and now it's time to turn it on. Using LaunchDarkly. It's as simple as adding a rule or just toggling the flag on and it's been released to your customers because the code already lives in production. But what's not immediately clear from that seemingly climactic moment is all of the offline work and all of the offline collaboration that goes into play to plan a release. I'm talking about things like waking up at 5am pacific time so you can release features at 8am to Eastern customers or even in things like getting an approval from your team lead or from your product manager, or just from the feature lead, when you're releasing something, all of these async processes around approvals schedules, they're critical to releasing software, but they do slow you down and add more potential for manual error or human error. And so our goal with feature workflows was to essentially automate the entire process of a feature release within LaunchDarkly. So we built these concepts like scheduling, we build concepts like approvals with a lot of thought and granularity. And what feature workflows essentially allows you to do is take these steps, stitch them together and define an end to end release workflow for your flag. So you can flag a feature, let's say, a new checkout page. If you're redesigning your checkout page, and you have it flagged, you're ready to release it to your customers. And essentially, what you want to do is roll back the release of the old checkout page, while simultaneously rolling out the release of the new checkout page, just do it safely just to make sure that things perform as expected, because there really isn't any replacement for testing and production with true infrastructure and your true users and a true scale. And so you can actually develop a workflow or define a workflow where you say, at 9am, three days from now, I want to get approval from these two team leads in order to start progressively rolling this flag out or this feature out by 5% over the next whatever. And launch shortly does the bat for you? And you can add lots of manual steps. Yeah, we have we have customers that are using feature workflows to do releases over three, four days. We actually use feature workflows to do an entire DB migration project about six months ago. Yeah, which database we move from MongoDB to CR dB. Cockroach dB. Okay. Yeah. And so there was a lot of ABB BB teachers involved. And so that's essentially where we use feature workflows. And the peace of mind that comes with automating these stages is kind of undescribable. In some ways,

 

Heather Joslyn  12:15  

are these new capabilities. Are they in general availability? Are they absolutely

 

Karishma Irani  12:18  

yeah, so we've had our scheduling capabilities and approvals functionality they've been in general availability for over a year. And as of earlier this year, we launched feature workflows, which allow you to stitch those together for a specific flag or multiple flags. And about three months ago, we released templates, which is essentially what allows you to save these workflows. So you can, let's say, you find your product team using the same release workflow, right, because you want to only have a standardized process for releases. You don't want to have people keep redefining it every time you want to release a feature. And so that's where templates really comes in, you can save a workflow as a template. And so you can have a template for feature releases for DB migrations for Canary rollouts for literally any kind of progressive rollout or staged rollout that you might have. And you can keep reapplying it and reusing it. And a lot of our enterprise features, enterprise customers to whom this feature is generally available, find this valuable because it helps them standardize their releases. So you have customers who have security and compliance requirements, right, they want to make sure that their teams are following certain processes and releasing features after getting approvals or releasing things at a certain date and time in a certain order. And so they can define a workflow, save it as a template, and then have the peace of mind that in order to release something to their customers, their teams need to use that template. And that's really where the standardization the consistency, but more importantly, the automation at scale helps.

 

Heather Joslyn  14:03  

Yeah, no definitely would help with the for an enterprise for any anyone who's going to be using these workflows over and over again, let's move on to the experimentation product. Can you tell us about with one benefit of feature management is that you can experiment you can a B test and so on. What is new in terms of experimentation capabilities?

 

Karishma Irani  14:21  

Yeah, so big shout out to our experimentation team and our head of experimentation, James frost, they launched an entire overhaul of our experimentation offering earlier this year in June of this year. And essentially what that new update involves is, it's a substantial update, calling it an update almost feels a little reductive or simplified, but they basically doubled down on our mission to enable any team to make informed decisions and ship confidently knowing that they're making a positive impact on the business metrics or the customer experience. And so they They're a complete overhaul of our stats engine, they enhance the integration path of our customers existing data sets and metrics, they redesigned our user interface. And they codified model and experimentation best practices into the product itself. And so two specific updates that I want to call out to our experimentation offering. The first is we noticed that with multiple Teams and Roles trying to make changes at the same time and production, collisions could happen, you could have Team A trying to ship something, Team B, trying to ship something else, which requires different audiences, or could, you know, just basically muddy the waters of your actual experiment and keeping the data sacred. And so parallel releases can overlap and interact with each other in unexpected ways. And there's just no way to prevent that unless you really want to slow your teams down or give them schedules or things like that. So to help with this, we've seen great success from teams where marketing teams, product teams, engineering teams, they all use the same tool to collaborate on releasing features releasing software to their end users. And so to help with this, we've integrated our experimentation offering into our already powerful capabilities like workflows, approvals, which make it easy and safe for product managers and marketers to work on the goal without having to worry about breaking things or using different tools. And so really just having the entire organization using a single tool is where we've seen a lot of success. And so to help with this, we have reduced friction when important metrics. So we have a new metric API metric import API, that significantly cuts down on manual duplicate work when importing metrics for experimentation. So you can get set up faster. And the second is what's known as a sample ratio mismatch test. So you can be confident that all of your experiments are correctly allocating traffic to each variant. And this is really important, because let's say you've designed and kicked off an experiment. That's great. The data you're collecting is precious with every day that goes by with more users that interact with your software with your tool with your product, you're collecting more data for that experiment. And we really want to make sure that this data is as accurate as possible. And so in order to avoid any potentially faulty results, we've made all of these updates so that you can be confident and really run experiments and ship more confident experiences, using the same tool that your developers are already using to ship software.

 

Heather Joslyn  17:53  

It's great. And again, another thing I'd like to ask you about is the core flagging platform, what are some of the new capabilities there?

 

Karishma Irani  18:01  

Yeah, so one major capability there is our team's functionality. So teams is a new generally available for enterprise customers functionality that we released earlier this year. And it essentially allows you to group members within LaunchDarkly into teams assign team names, theme descriptions, the maintainers, team roles and permissions. So you can create these customized experiences for different teams using LaunchDarkly. And this is important because if you want Team A to not see or interfere with teams these work, but you want to use one single account, this is really helpful. This is how teams work. This is how self software is built multiple teams working at the same time. But in order to make the adoption and setup of teams easier, we have integrated with IDPs specifically Octa it is now GA. And so if you're using Octa as an IDP, you can sync it with LaunchDarkly. And all of your teams will automatically be imported within LaunchDarkly. So you don't have to manually set it up and assign members to teams or assign names to teams. More importantly, it stays in sync. And then you can use these teams to assign them as flagman. Tinos or request approvals from them, etc.

 

Heather Joslyn  19:17  

So you can assign roles as well with

 

Karishma Irani  19:20  

exactly exactly so you can say, this is the team lead and I want to get approval for feature a from my team lead, but approval from for feature B from this entire team, the platform team. So you can actually specify teams as approvals. And the second major update to our core platform, which is also generally available is the flag defaults capability. And so creating flags that scale can be tedious, a little repetitive and also error prone. And so flag defaults allow you to set flag configurations that embody Best Practices things like setting up a flag default, setting up the type of flag whether it's a Boolean or multivariate, things like that, basically configuration across multiple projects to make it easier.

 

Heather Joslyn  20:07  

What's next on the roadmap? What's ahead at LaunchDarkly?

 

Karishma Irani  20:11  

Yeah, lots of exciting stuff. But the two big call outs that I want to make are custom contexts and accelerate. And so I'll start by describing custom contexts when you're so far up the scarf, feature flags and feature management and targeting experiences to your end users. And all of that makes sense when you're a b2c too. But let's say you're a b2b company, and your end users aren't really users, their devices, or tenants, or organizations, or whatever it might be. And so you want to use LaunchDarkly, to target or release things to devices, organization standards, again, any kind of custom context. And so this is a massive, massive update to our core technology that we're making for all of our customers in, which required a heavy lift and updating all of our SDKs and data models in a backward compatible way. So it's going to be seamless to anyone who doesn't need it or doesn't want to use it. But to customers, like customers like Hulu or ch Robinson who want to target based on licensing region, they now have a first class way to do that. And this is something that only LaunchDarkly supports and offers three. And in previously, in order to allow for this, you have to deploy workarounds, you had to create rules where you said, if user is device type, mobile, desktop tablet, but now you can say if device is tablet, mobile, desktop, etc. And so you can really using different attributes or different types of contexts to really target a single feature, it creates this first class experience for you and allows you to say, Hey, I wanted to feature a on for these three users and these five organizations and these six plan types, all within a single experience. Cool, yeah, then accelerate. That's the new one. So we recently announced accelerate as a new product offering soon to be an early access. And what accelerate is, is essentially a natural extension of our ability to measure the value that our customers get from feature management. So let me explain your let's say you're a customer, and you're using LaunchDarkly, to add feature flags to control to launch to essentially measure, measure different experiences, run different experiments, etc. And your team lead comes up to you and says, Hey, how do I know that LaunchDarkly is actually improving our engineering practices, or LaunchDarkly is actually benefiting our engineering team. Earlier in this discussion, we talked about how at first it can seem like separating deploys and releases is an extra step like why would someone do that? But there are benefits? What are those benefits? And how do you measure them. And that's really what accelerate is trying to provide our customers with, we know that engineering leaders want to measure software delivery, performance and reliability across their teams. So accelerate lets you track and visualize key engineering metrics, such as deployment frequency, release frequency, lead time for code changes, and flag coverage. And I'm sure you've caught on already. But a few of these are Dora metrics, which obviously are extremely critical to our users. But LaunchDarkly has a unique vantage point where not only do we have visibility into your lead time, or your number of deploys. But we can also provide you with release metrics. So we can tell you, how many times did you use a flag for kill switching a feature, aka how many times that you prevent an issue from happening and production, thus reducing your MTTR. So unlike other similar solutions on the market, accelerate treats deployments and releases separately. That is, it measures the rate of value delivery, which is your release frequency, and also shows you how many pull requests include feature flags, helping you correlate flag usage with software engineering performance. So it's great. We're really excited about this. What is

 

Heather Joslyn  24:28  

what is the timeline for those to be released? Are they available in any kind of preview?

 

Karishma Irani  24:33  

Yes. So accelerate is available in early access. And if you as a listener, or viewer are interested in getting access to accelerate, you can just go to launchdarkly.com backslash EAP for early access program and enter your details there.

 

Heather Joslyn  24:51  

One last question. What do you think are the big challenges in feature management that are still still to be solved? Or what do you see as the next big steps that the field and yeah growth leads to tackle.

 

Karishma Irani  25:01  

So that's a constant discussion that we have internally. And I'll share literally what we discussed last week and our product leadership team. And it's the idea of really supporting a continuous release while maximizing value and mitigating risk. And as simple as it may sound, it includes components like when I say, like making releases more continuous, more efficient, it includes components like automation, which are tricky. Today, workflows are at a specific environment level. But we're our teams are working on making these workflows, cross environments, so you can literally flag a piece of code and define its start to finish release all the way through production, and step away from your computer, knowing that LaunchDarkly is going to manage its rollout. But that's not all. That is the aspect around maximizing value rolling things out faster innovating. But the other side of the coin is mitigating risk. And that's really where you use the word guardrail. So I'm gonna, I'm gonna stick to that. That's really where those guardrail metrics come into play the idea of allowing a product manager to define which metrics and experiences that they want to have not negatively impacted, or which operational metrics or let's say, your CPU throughput, or some kind of threshold or latency, you want to have your performance metrics not be negatively impacted. You don't want to cause incidents by moving too fast. You only want to move with your progressive release, if you know that it's having a positive impact on your business metrics, whether that's a click rate, or view rate, revenue, whatever it might be. And so how do you combine those two together? Using experimentation, you can define these metrics, you can measure things and using our core feature management capabilities. You can automate rollouts. And we believe that the next step is resolving the challenge of bringing both of these together in a single seamless experience. And I'm really excited about the position that LaunchDarkly is in to be able to do that.

 

Heather Joslyn  27:13  

Excellent. Well, thank you very much. And that's all for this episode. We'd like to thank Krishna Irani for joining us today. Thank you. And just a bit of background, we recently interviewed Krishna, his colleague from LaunchDarkly, Ravi Thursday, and that episode of makers would make a good companion to this one as we talked in depth about feature management and the value it has for developers engineers in their organizations. So if that topic interests you, you should check that out next. And we'd like to thank the sponsor of today's episode LaunchDarkly. And thank you for joining us today. My name is Heather Joslin of the new stack, and we'll see you next time.

 

Colleen Coll  27:47  

Unleashed developer productivity for the software powered world by fundamentally changing how you deliver software to your customers. With LaunchDarkly ease feature management platform, empower developers can empower the business to release new features faster and more efficiently than ever.

 

Alex Williams  28:06  

Thanks for listening. If you'd like to show, please rate and review us on Apple podcast Spotify, or wherever you get your podcasts. That's one of the best ways you can help us grow this community and we really appreciate your feedback. You can find the full video version of this episode on YouTube. Search for the new stack and don't forget to subscribe so you never miss any new videos. Thanks for joining us and see you soon.

 

Transcribed by https://otter.ai