In this episode of The New Stack’s On the Road show at Open Source Summit in Austin, Webb Brown, CEO and co-founder of KubeCost, talked with The New Stack about opening up the black box on how much Kubernetes is really costing.
In this episode of The New Stack’s On the Road show at Open Source Summit in Austin, Webb Brown, CEO and co-founder of KubeCost, talked with The New Stack about opening up the black box on how much Kubernetes is really costing.
Whether we’re talking about cloud costs in general or the costs specifically associated with Kubernetes, the problem teams complain about is lack of visibility. This is a cliche complaint about AWS, but it gets even more complicated once Kubernetes enters the picture. “Now everything’s distributed, everything’s shared,” Brown said. “It becomes much harder to understand and break down these costs. And things just tend to be way more dynamic.” The ability of pods to spin up and down is a key advantage of Kubernetes and brings resilience, but it also makes it harder to understand how much it costs to run a specific feature.
And costs aren’t just about money, either. Even with unlimited money, looking at cost information can provide important information about performance issues, reliability or availability. “Our founding team was at Google working on infrastructure monitoring, we view costs as a really important part of this equation, but only one part of the equation, which is you’re really looking at the relationship between performance and cost,” Brown said. “Even with unlimited budged, you would still care about resourcing and configuration, because it can really impact reliability and availability of your services.”
SPEAKERS
Emily Omier, Webb Brown, Alex Williams, Colleen Coll
Colleen Coll
Welcome to this special edition of a new stack makers on the road. We're here at the Open Source summit in Austin, Texas. Discussions from the show floor were technologists giving you their expertise and insights to help you with your everyday work.
Emily Omier
Hi, everyone, I'm Emily O'Meara with the new stack. We're here at open source summit in Austin. Today, I'm here with Webb Brown, the co founder and CEO of cube cost. Thank you so much for joining us.
Webb Brown
Yeah, really excited to be here. Thank you so much for having me. And awesome to be on the floor here at the summit.
Emily Omier
So can we start out with having you introduce yourself? And let listeners know what you do?
Webb Brown
Yeah, absolutely. A co founder of coop cos and one of the CO creators of the open cost project, we build cost management cost optimization solutions purpose built for Kubernetes.
Emily Omier
When you say something like Kubernetes cost, what exactly does that mean? Seeing as how you don't get a bill from Kubernetes?
Webb Brown
Yeah, great question. I think that highlights some of the challenges here, there's this new level of abstraction, that is the Kubernetes scheduler, Kubernetes API. So kind of this old mechanism of using Cloud tags, and just allocating VMs really doesn't work anymore. We help teams kind of peer into that black box and really understand all the applications or teams or projects that are running in that environment, how much resources they're consuming, and what those resources actually cost. So you can then have a really accurate picture of okay, what teams, what departments, what cost centers are actually consuming resources, or even what customers that you're providing your services are consuming in terms of costs, resources, etc.
Emily Omier
And I know there's also companies that do like Cloud cost visibility. So how is it different to look specifically at Kubernetes? Versus this is just like Cloud cost visibility?
Webb Brown
Yeah, good question. I would say there's a range of differences. But really two standout is, one is Kubernetes itself is very common, where a lot of your resources are shared across all of your tenants. So in a traditional cloud cost environment, you would say, take a database and allocate it to this application that's consuming it. Now everything's distributed, everything shared. So it's much harder to like, understand and break down those costs. And then to is things just tend to be way more dynamic, right. So like, part of the amazing benefits of Kubernetes is like it's distributed, and things are resilient. Pods come up, jobs go down, nodes come up, nodes go down. And things are just very likely to be constantly in flux, or changing when you're at scale. So it's really those two dynamics that make it like applying a lot of the same fin ops principles. But how you do it is actually pretty different.
Emily Omier
So let's say hypothetically, you have an unlimited wallet, would you still care about this? Would it matter, even if you just have unlimited money?
Webb Brown
Great question. Yes, and no, I think when you have unlimited money, you're really just focused on your pace of innovation, and things that are kind of preventing you from delivering business value, delivering very reliable service, etc. Open costs, or coop costs can help, we're actually identifying performance bottlenecks and things from a resource constrained standpoint, that can actually cause either performance issues, or reliability or availability issues. As engineers, our founding team was at Google working on infrastructure monitoring, we view costs as a really important part of this equation, but only one part of the equation, which is like you're really looking at the relationship between performance and cost. If you were in a scenario, we you did have an unlimited budget, you still would really care about resourcing and configuration, etc. Because it can really impact reliability and availability of your services,
Emily Omier
I would imagine to well, in the real world, nobody has an absolutely unlimited budget. And usually you have to make some sort of choices, right? And sometimes the right choice is we want to throw money at this, because it's really important. So I would imagine there's also an element of this lets people make better choices.
Webb Brown
Absolutely. I think they're the way we think about and a lot of our users think about is like context really matters, not only at your organization level, but for each individual application, right? If you have a workload that is just kind of batch processing, and it doesn't really matter when it finishes, you may optimize for like the lowest costs, compute or storage, etc. That's available as long as the job gets done in this certain broad window. But if you have an H A or critical workload, yeah, you may be very inclined incentivized to spin whatever it takes to make sure you get say, five nines of uptime with the performance you expect to really can come down to the individual micro service or individual application where you're thinking about these really nuanced trade offs.
Emily Omier
And you're also really tying into the business like you can also think I'm imagining is this feature or is this application profitable?
Webb Brown
Absolutely, we think about like a more advanced spin offs practice. But really the unit economics of these different applications or you know, skews that you're selling, this is one area where interoperability becomes so important, where you're able to look at Kubernetes costs alongside costs outside of Kubernetes, like storage buckets, and external databases, but also alongside your business metrics. I think that's where the like open source piece and open standards that are being developed, like it's open costs project, thinking about open telemetry and how it can speak to Prometheus and cortex and these other cn CF projects, I think there's a real opportunity, and I'm seeing just a lot of like innovation, where we can just have a common language and that data can be transported wherever is like most effective for your company, whether it's engineers, Fin ops, or data scientists to like, make really powerful decisions on it.
Emily Omier
So speaking of fin ops, what does it mean to have a mature Finn ops organization? What are they doing that everyone else isn't?
Webb Brown
So then off to x conference was earlier this week, there's a lot of really interesting discussion there. We oftentimes talk about crawl, walk, run as like the evolution to getting there. I think it's having the visibility, you need to accurately monitor costs, across different applications, teams, etc, then it's feeling like you have the tools to actually optimize, again, based on your specific requirements around performance, reliability, etc. And then lastly, I think it's the ability to, like govern that on an ongoing basis. And I think your reference to kind of unit economic analysis to where you can really look on cost or profitability by customer or by API request, or by you know, search in your application is oftentimes evidence of being a pretty advanced practice.
Emily Omier
Is there anything that a very mature fun ops organization does that someone in an earlier level of maturity wouldn't even know that they need to think about?
Webb Brown
I think there's a lot one of the big ones is flipping from being reactive to proactive, right? I think it's really common for like early fin ops teams to kind of get spun up because wow, our expenditures were 2x, or 5x, or 10x, what we expected them to be last month or last quarter, whereas really advanced fin ops practices are really able to forecast accurately have all these guardrails in place to think about managing that risk. And to really be able to like dial that in, in proportion to kind of the growth of the business or usage of their
Emily Omier
products. What would you say is like the first step, say, you're at that early level, but you've just got a bill that was 10x. And you're thinking maybe, maybe you need to do something about it,
Webb Brown
I think first is stop and get visibility, really interesting cntf study from last year where 70% of teams running Kubernetes don't have accurate visibility into costs. So I think just getting eyes and ears, making that as transparent as possible throughout the organization. And using that to take a step back and say, Where do we go from here? What we've seen is like, actually, having accurate visibility can be a huge part of the battle. Because, you know, as engineers and ops practitioners, we all want to do the right thing and help manage these trade offs effectively. But without having the like data. It's nearly impossible to do that. So yeah, I think just starting with making sure you have fair, accurate, timely, and complete visibility of all of your computer or infrastructure spend,
Emily Omier
and whose responsibility is this? I mean, is it the developers responsibility? Or is it the CFOs responsibility,
Webb Brown
it really depends on the organization. But I would say we increasingly see just this, it is a joint effort. Engineering is contributing a real piece of getting the instrumentation in place, really thinking through the nuances of how Kubernetes is configured, or you know, applications are configured, things like auto scaling, or configured etc. But finance is playing this really valuable role finance, generally fin ops more broadly, of how to think about forecasting, how to think about unit economics and what we should be targeting. That's where So, ultimately, we see it really being kind of a joint team effort. When it's really successful.
Emily Omier
does that present any cultural challenges inside of organizations like collaboration?
Webb Brown
Yeah, I think it's really, you know, on an organization basis. I see this trend towards these teams are just working more and more effectively together somewhat out of necessity, because there's so much engineering complexity. A. And there's also a lot of like finance knowledge that may be broader scope than like the engineering efforts that are happening. I see it naturally happening where there's better and better alignment and relationships and collaboration. But I think that the nuances of how the organization is structured and even some of its origins can kind of shape some of those dynamics.
Emily Omier
To what extent are developers even part of the conversation around how much money this is costing? I mean, how much do they know about like the ultimate $50 million bill that their 5000 person organization is getting, maybe my my numbers are totally just pulled out of thin air,
Webb Brown
I would say an increasing amount are just aware. And that really starts from just like this data being at their fingertips, right? Whether it's in their cloud console on the like CLI, they use all the time, in their, like Grafana dashboards that they reference, but increasingly, we're spending at a level to where we need to be aware, because the lack of awareness can be very costly from a company's success, even. So I think it's like happening in different ways in different organizations. But I would say the other thing is, there's now all these really cool tools, especially in Kubernetes, where engineering teams can make a major difference on this, configuring things like cluster auto scaling, or taking advantage of spot, etc. These are deep integration decisions and problems because they can impact reliability and application performance. It's really like bringing engineers into these really cool cost optimization problems.
Emily Omier
Yeah, it seems like bringing engineers into like, really core business discussions. Exactly,
Webb Brown
exactly. And it's always coming at it from the lens of engineering performance, whether it's budgets or availability of resources to engineering groups, or again, application performance itself.
Emily Omier
Fabulous. Thank you so much for joining me. And thank you everyone who's watching and listening for joining us at open source summit in Austin.
Alex Williams
Thanks for listening. Subscribe on simple casts listen to more episodes on the new stack Baker crate and share your favorite audio RAM using our simple task player. More articles and great stories go to the new stack.io