Platform engineering “is the art of designing and binding all of the different tech and tools that you have inside of an organization into a golden path that enables self service for developers and reduces cognitive load,” said Kaspar Von Grünberg, founder and CEO of Humanitec, in this episode of The New Stack Makers podcast. This is structure is important for individual contributors, Grünberg said, as well as backend engineers: “if you look at the operation teams, it reduces their burden to do repetitive things. And so platform engineers build and design internal developer platforms, and help and serve users. “ This conversation, hosted by Heather Joslyn, TNS features editor, dove into platform engineering: what it is, how it works, the problems it is intended to solve, and how to get started in building a platform engineering operation in your organization. It also debunks some key fallacies around the concept. This episode was sponsored by Humanitec. Kaspar Von Grünberg - https://www.linkedin.com/in/kvgruenberg/ Heather Joslyn - @ha_joslyn The New Stack - @thenewstack
Platform engineering “is the art of designing and binding all of the different tech and tools that you have inside of an organization into a golden path that enables self service for developers and reduces cognitive load,” said Kaspar Von Grünberg, founder and CEO of Humanitec, in this episode of The New Stack Makers podcast.
This is structure is important for individual contributors, Grünberg said, as well as backend engineers: “if you look at the operation teams, it reduces their burden to do repetitive things. And so platform engineers build and design internal developer platforms, and help and serve users. “
This conversation, hosted by Heather Joslyn, TNS features editor, dove into platform engineering: what it is, how it works, the problems it is intended to solve, and how to get started in building a platform engineering operation in your organization. It also debunks some key fallacies around the concept.
This episode was sponsored by Humanitec.
The notion of “you build it, you run it” — first coined by Werner Vogels, chief technology officer of [sponsor_inline_mention slug="amazon-web-services-aws" ]Amazon,[/sponsor_inline_mention] in a 2006 interview — established that developers should “own” their applications throughout their entire lifecycle. But, Grünberg said, that may not be realistic in an age of rapidly proliferating microservices and multiple, distributed deployment environments.
“The scale that we're operating today is just totally different,” he said. “The applications are much more complex.” End-to-end ownership, he added, is “a noble dream, but unfair towards the individual contributor. We're asking developers to do so much at once. And then we're always complaining that the output isn't there or not delivering fast enough. But we're not making it easy for them to deliver.”
Creating a “golden path” — though the creation by platform teams of internal developer platforms (IDPs) — can not only free developers from unnecessary cognitive load, Grünberg said, but also help make their code more secure and standardized.
For Ops engineers, he said, the adoption of platform engineering can also help free them from doing the same tasks over and over.
“If you want to know whether it's a good idea to look at platform engineering, I recommend go to your service desk and look at the tickets that you're receiving,” Grünberg said. “And if you have things like, ‘Hey, can you debug that deployment?’ and ‘Can you spin up in a moment all these repetitive requests?’ that's probably a good time to take a step back and ask yourself, ‘Should the operations people actually spend time doing these manual things?’”
For organizations that are interested in adopting platform engineering, the Humanitec CEO attacked some of the biggest misconceptions about the practice. Chief among them: failing to treat their platform as a product, in the same way a company would begin creating any product, by starting with research into customer needs.
“If you think about how we would develop a software feature, we wouldn't be sitting in a room and taking some assumptions and then building something,” he said. “We would go out to the user, and then actually interview them and say, ‘Hey, what's your problem? What's the most pressing problem?’”
Other fallacies embraced by platform engineering newbies, he said, are “visualization” — the belief that all devs need is another snazzy new dashboard or portal to look at — and believing the platform team has to go all-in right from the start, scaling up a big effort immediately. Such an effort, he said is “doomed to fail.”
Instead, Grünberg said, “I'm always advocating for starting really small, come up with what's the most lowest common tech denominator. Is that containerization with EKS? Perfect, then focus on that."
And don’t forget to give special attention to those early adopters, so they can become evangelists for the product. “make them fans, prioritize the right way, and then show that to other teams as a, ‘Hey, you want to join in? OK, what's the next cool thing we could build?’”
Check out the entire episode for much more detail about platform engineering and how to get started with it.
Alex Williams 0:08
You're listening to the new stack makers, a podcast made for people who develop, deploy and manage at scale software. For more conversations and articles, go to the new stack dot I O. All right now on with the show
Colleen Coll 0:29
you manitech Is the platform orchestrator at the core of your internal developer platform. And let's platform teams remove bottlenecks by letting them build golden paths, or developers, developers self serve the tech they need to deploy and operate their apps driving productivity and velocity.
Heather Joslyn 0:49
Hello, and welcome to another edition of the new stack makers podcast. I'm your host Heather Jocelyn Features Editor of TNS and today we're going to talk about platform engineering, what it is the problems that it solves and the connection it has to DevOps platform engineering was a hot topic at cube con cloud native con this fall in Detroit and we're going to dig in and get you up to speed over the next several minutes. Our guests to help us do that today is Casper Vaughn Grunberg, founder and CEO of Humana tech. Welcome Casper.
Kaspar Von Grünberg 1:15
Heather, thank you so much for having me.
Heather Joslyn 1:17
Thank you for joining us, Casper, can you tell us a little bit about Humana tech and what it does drop. So
Kaspar Von Grünberg 1:21
if you're any tech is a platform orchestrator. That's a component that you plug in at the end of your CI pipelines, your registries, and it's basically streamlining all of the tech and tools that you have an efficient digital assembly line, if you want to call it like this, it is enabling dynamic configuration management. And that allows developers to express what they need. And then the basically, the response from the platform team will be automatically matched and then orchestrated. And it's used by a good amount of enterprises across the world to build what we refer to as dynamic internal developer platforms. Okay, thank you.
Heather Joslyn 1:59
Thank you. We'll be talking about internal developer platforms quite a bit in this conversation. We'd also like to acknowledge and thank humanity tech for sponsoring this episode of makers. There's a lot to talk about. So let's just jump in. As I mentioned, platform engineering was a big topic at cube con. This fall, let's get everyone who's listening to our conversation on the same page. How do you define platform engineering?
Kaspar Von Grünberg 2:21
Well, platform engineering, for me is the art of designing and binding all of the different tech and tools that you have inside of an organization into a golden path that enables self service for developers and reduces cognitive load that is very important, in my opinion for the individual contributor. And thus, if you look at the operation, teams reduce their burden to do repetitive things. And so platform engineers build and design internal developer platforms, and help and serve a user. So we're always big proponents of treating your platform as a product. And so your users, the users, that platform engineering team serves all the developers, it does
Heather Joslyn 3:08
seem like there's a lot of discussion in the field about or a realization that you know, a golden path, as you as you put it, is a solution that kind of is replacing the do it yourself, assemble your stack from your developer environment, from all all your choices that you have, what are some of the problems that platform engineering is intended to solve?
Kaspar Von Grünberg 3:27
Well, I think the I think we're really like in that in that whole conversation, especially around that little bit of tension that we had with the question, how does it relate to DevOps and all of these things, I think we need to look back at why do we need to do this? Why is this a problem we need to solve as an industry? If you go back to 2006, there was this famous talk by Vanna Fogle, the CTO of Amazon Web Services, and he proclaimed you build it, you run it, that idea that you should actually have end to end ownership. And that is conceptually a great idea. But you need to see it in the context of the time. 2006 was a different world. It was a, you know, low microservice world, we were faced with monoliths that were running on data centers, the digitisation, it wasn't at the level that we have right now globally distributed systems, massively complex systems, we have to serve millions of users. So the scale that we're operating today is just totally different. The applications are much more complex. And so that idea of self you build it, you run it in a cloud, native world end to end you build your own end to end ownership is to a certain degree, a noble dream, but unfair towards the individual contributor, we're asking developers to do so much at once, and then we're always complaining that hey, you know, the output isn't there or not delivering fast enough, but we're not making it easy for them to deliver. That's the underlying problem that complexity has As grown exponentially, but our response to taming that zoo has not actually adopted and platform engineering is an attempt to be respectful with the cognitive load of the individual contributed the individual user and binding things into a golden path to say, hey, you should have the freedom to do whatever you want, as long as it's compliant and secure. But if you want to focus on what you're really good at, you're an excellent front end engineer, you're great at QA, you're a great back end, you're great at infrastructures code, well, then you should also be, it should be okay for you to focus on these areas. And just to fold to other things, in the other areas that the platform engineering team actually supplies for you. You mentioned
Heather Joslyn 5:45
that it reduces cognitive load on the developers, are there other benefits as well, I mean, for example, in terms of security, in terms of just having standardization.
Kaspar Von Grünberg 5:54
Yeah. So the we need to look at there are different people that benefit in different ways, we have the individual contributor for them, it's definitely cognitive load, it's reducing waiting times in the enterprise, a huge problem is that you need a new environment, you're waiting, you need a database in a particular way, security is checking it, you're waiting, you know, very, very draining. So that's definitely it for the for the individual user. And then for the organization as a whole for the operation teams, it's standardization by design, if you have a lot of people doing things all over the place, you know, keeping track of things, maintaining things becomes increasingly complex, there are 100,000 ways to express the same state that your service should be in if you're using whatever, like a Helm chart. But if you're able to trim that down slightly, and drive standardizing behavior if you want, that is such a relief for the overall organization. And that's what platform engineering is about as well,
Heather Joslyn 6:58
for ops engineers as well. Does it? Does it help in terms of making it over time making it easier and faster for them to solve incidents or solve problems? Because the environment is the platform is the same for every developer? And so they're they're learning, you know, how to solve the same the same problems?
Kaspar Von Grünberg 7:17
Absolutely. So not only that, it's also about reducing pressure on ops. So if you want to know whether it's a good idea to look at Platform engineering, I recommend, go to your service desk and look at the tickets that you're receiving. And if you'd have things like hey, can you debug that deployment? Can you spin up an environment, all these repetitive requests? You know, that's probably like a, like, that's probably a good time to take a step back and ask yourself, should the operations people actually spend time doing these manual things? Which puts them under so much pressure and so draining and frustrating for them? Or should you actually look at automating that and build these golden paths and build these internal developer platforms?
Heather Joslyn 7:58
Are there any new challenges that platform engineering introduces to DevOps teams?
Kaspar Von Grünberg 8:01
Absolutely, it can if you do it the wrong way. So I think what we've seen in in some of the platforms in the 2010s, is that they've been introducing too much abstraction, that's probably the worst thing you can do, right? If, if you're saying well, everything's over the all over the place, we need to standardize completely. And then you build a CLI that you think works great in 90% of cases, but then it falls apart and 10%. But then the developer runs into that 10% case, that's incredibly frustrating. And they start mistrusting the platform. And then that can lead to a lot of cultural fighting and very draining situation. So I'm always advocating golden path over golden cages. I like to draw the parallel to natural language processing, if you're using Amazon, Alexa, or Siri or any of those assistants, and they work in nine out of 10 cases for us as you as humans, that's not enough, we still distrust, right? We all know that feeling. That's the same with platform engineering. And the respect, the thing you need to do is lay at abstractions, the individual user should choose how much cognitive load they want to consume, and then actually consume accordingly. And they should be able to follow the golden path or circumvent that. And if you want to get a more tangible example, they could decide to say, I'm staying on a on an abstract level, I'm just saying my workload needs a database of type Postgres, but they should also be able to go down and say, hey, you know what, I want to be the one tweaking the infrastructures code file that then results into me getting the database. So that's the idea of layered abstractions. That is a good response to that general problem.
Heather Joslyn 9:48
We may have covered this a little bit already talking about the impact on ops teams, but how is platform engineering related to SRE and DevOps and what kind of impact will it have on if you have a DevOps team? Have you have you have an SRE team.
Kaspar Von Grünberg 10:01
So the the term DevOps to certain degree has been abused, I would say, we've put that label on almost everything on operations and on sre. And like, you know, people that just held DevOps is to a certain degree of philosophy, it's a way of communicating, it's a way of aligning team that doesn't go away, that shouldn't go away. You know, DevOps and platform engineers, you know, platform engineering, and DevOps are actually, you know, friends, like, like 100%. Platform engineering is the practice of designing these paths that are respectful, like, it's actually like platform engineering is actually a product group that has a product owner, I very much recommend that team to have a product owner that has a user, the developer, that builds the product, the internal developer platform. And that's not what neither SRE nor DevOps is supposed to do. Sre is supposed to be build reliable systems that are secure, and that are, you know, up for the user. And that, you know, default gracefully, all of these things. DevOps is a philosophy it how do you design the relationship between teams? How do you structure that ownership if you want, and platform engineering as a practice, on how to build internal developer platforms? For a user? What are
Heather Joslyn 11:26
some of the most common fallacies about platform engineering,
Kaspar Von Grünberg 11:29
I think the thing that I see most often is that people try to start solving the most the thing that for them feels most obvious. So fallacy number one is definitely I called the prioritization fallacy, they start thinking about the lifecycle of an application, that could be when I'm starting an application, or let's say, I'm adding a new microservice, then I need to clone a template, or I want to clone a template. And then I want to configure CI, and I want to have that deployed somewhere very often, you know, that's sort of the key thing that new platform engineering teams start off with. Or another example could be, they're thinking about the lifecycle of a developer at that particular organization. And then they'll start at the beginning and say, well, at some point, the developer joins, and that experience of joining should be great. And that's an approach to take. But if you think about it, we're actually treating our platform as a product, then, if you think about how we would develop a software feature, we wouldn't be sitting in a room and taking some assumptions and building something, what we would do is we would go out to the user, and then actually interview them and say, Hey, what's your problem? Like? What's the most pressing problem? And the prioritization fallacy means I'm not treating my platform as a product, I'm just coming up with what I think is most obvious. But that doesn't doesn't need to be true. Like how often are you adding a new service? How often are you adding like in the lifetime of a developer is the onboarding really the critical thing, or the fact that they need to change environment variables, every 20 deployments, because that adds up, you know, on wanting doesn't, so that the recommendation, go out, interview your users, make sure you don't fall for that prioritization fallacy. Then another one is definitely the idea that you want to build that huge platform that works for everything, and then scale that to hundreds of users in the first three months, that's almost doomed to fail. The I always recommend like, there is a reason why there is no vendor out there that gives you a Heroku like experience across virtual machines, Kubernetes lambda and all the databases in the world? Well, because it's an almost unsolvable problem. And if no vendor can do it, it's unlikely that your platform engineering team, no matter how large it is, can actually pull that off, like I've never seen that really work. So I'm always advocating for start really small come up with what's the future, most lowest common tech denominator, if you want, you know, is that containerization with eks? Perfect, well, then focus on that. And make sure you actually build a great experience for that for a small group for a lighthouse app for a lighthouse team, make them fans prioritize the right way, and then show that to other teams as a hey, you want to join in? Okay, what's the next cool thing we could build? So that's definitely, definitely the second one. The third one is what I call the visualization fallacy. There's so many teams that say, well, by just visualizing stuff and showing nice dashboards and incredible portals that have our logo on the upper left corner, we are going to solve everything and developers will love us. And what I find very often is that if you look at the users 12 months later, developers don't actually look at yet another interface it needs to be deeply ingrained into their daily workflow. It needs to read really make a difference on the ground every day, every time you go beyond the simple update of an image, by definition, this is actually when you need a platform to do things. And so just putting out like a fancy potluck, probably won't cut it. I'm always saying you need to think, do you want to build the house first or the front door? You know, and having the front door is? Maybe not what's really moving the needle?
Heather Joslyn 15:28
That does bring me to a question about we all know that, you know, the saying like every technology challenges, actually, issue is actually a people issue. Have you talked a little bit about, you know, how do you get to get a sense of the cultural changes that need to happen to get developers to use the platform rather than them to use the IDP rather than, you know, oh, it's yet another dashboard with a logo in the upper left hand corner?
Kaspar Von Grünberg 15:52
Yeah, exactly. And I think the answer to that is lies in the way you're approaching platforming. Again, if you're doing this big bang, hey, there's the portal and why don't you use it developers, you'll not see use it like the the numbers will be catastrophic. So that is the pull versus push, if you push that to the user, not much effect. If you take a small Lighthouse application that's on your lowest tech denominator, it's already containerized. Already in Kubernetes, let's take that as an example. And you work with a lighthouse team, I always like to use the teams that are known to be open and receptive to innovation there, there's always this one team that is the front runner unit work with that front runner team and on that lighthouse application. And then you want to do that one thing really good. So that that lighthouse team says, Hey, this made a real difference. And this was developed with our input, and they really listened. And it was such a like such a healthy collaboration between platform teams, and that individual team, even if it's only like a tiny little thing, like a CI that helps you spin up something or whatever, like an optimized CI CD process, whatever it is, that small thing done really, really well. Then those Lighthouse teams will create a pool fact that they'll go to the rest of the organization say, hey, you know, we're one of you. And we've been working with them. They're the remarkable they're listening to us. And we are developing how you should develop in the 21st century. And you can join in, do you want to have this as well. Now that shows remarkable results much, much faster.
Heather Joslyn 17:30
And so turning to that lighthouse team are internal evangelists, they might call them are early adopters, or, you know that in starting small sounds like those are two of the takeaways there. What is he manitex approach to platform engineering.
Kaspar Von Grünberg 17:45
So our approach is that the platform engineers are the builders. And we are, you know, supplying some components that help you build amazing platforms. We believe in a slightly in my interpretation, more modern idea of configuration management we call dynamic configuration management. The idea is that you are basically describing your application in all possible environments in a almost like a baking recipe. And then you let that run through your existing version control pipelines, everything into the orchestrator. And that's us. You can think of as like a baking machine. So we're getting the baking recipe, we're we're baking the platform, we are baking the application with every single deployment new. And that allows the developers to now choose how deep they want to go in that design of layered abstractions. Basically, do they just want to describe how the cake looks if we take that analogy, and say, hey, it's a, you know, strawberry cake with clotted cream. And then they sent it against the platform orchestrator, the orchestrator says, well, what's the context AHA environment of type staging, this is what the how the platform team now wants me to add in the strawberries, compose everything, and then bake my cake. All the developer can say, hey, I want to go down to the TerraForm level, and tweak everything here, and then deliver and that allows for this great standardization by design, low cognitive load, developer self service, and helps you to craft these platforms. And we have a lot of different interfaces. We have an API CLI UI, fully get based version, so they never have to leave the version control system. Or you can just wire us up with your own, you know, service catalog developer portal. We have plugin plugins with backstage and other other portals out there. And yeah, allows you to develop reliable dynamic internal developer platforms.
Heather Joslyn 19:45
If you wanted to introduce platform engineering and an IDP engineered organization. We know that and a platform team. We know that we're headed into tricky economic times here possibly we're hearing about companies being more cautious about spending money higher So how do you make the case to the stakeholders who will decide whether the fund platform team?
Kaspar Von Grünberg 20:05
Yeah, so this is actually one of those cases, that's really straightforward. And this is exactly the same thing, or methodology that you would use to figure out where should you start with your platform. So the prioritization, you can use that to actually build your return on an investment case for your management, I recommend take a white piece of paper, write down all the things that go beyond the simple update of an image, I am going to like a developer would spin up a new environment, the developer requests, the database developer needs to rollback a developer needs to send me an audit log all of these things. Then ask yourself, How often do you do this against 100? deployments? How much time does that eat up from developers? How much time does that eat up from operations, and then you build that list, you multiply this times the number of total deployments, and the number of or the average salary in your organization. And you have a beautiful case right there, especially in times of crisis, making sure we automate making sure we are getting the most of the colleagues that we have around us, enabling them to really outmaneuver the crisis through great innovation. That's a proven recipe to help you leave the crisis stronger than before. Right? The other approach is fire, everybody, reduce investments, try to look away, that has never worked. So platform engineering and safety, the thing you want to do in situations like now and you know, look at Gartner trend for 23. Wherever you look, it's the thing to focus on.
Heather Joslyn 21:36
And it's as we know, it, like cutting innovation during a crisis doesn't, as you mentioned, doesn't doesn't pay off when the crisis is over. So I think that's all I have right now. Unless there's anything else you'd like people to know about IDPs about platform engineering.
Kaspar Von Grünberg 21:51
Well, I encourage you to participate in the global community of platform engineers. It's a very strong community, very healthy, fast growing. There's platform engineering.org, there are meetup groups, probably in your city, there's platform con. So definitely something that you should, you know, be part of learn. Use that opportunity to become a platform engineer and introduce that practice at your organization. I think that's a tremendous opportunity, both for you as an individual, as well as for your organization.
Heather Joslyn 22:23
And again, thank you Casper Vaughn Grunberg for joining us today. And we'd like to thank Humana tech as well or sponsor for this episode. And thank you to all of you listening for joining us. We'll see you next time.
Colleen Coll 22:34
You manitech is the platform orchestrator at the core of your internal developer platform. It lets platform teams remove bottlenecks by letting them build golden paths for developers, developers self serve the tech they need to deploy and operate their apps driving productivity and velocity.
Alex Williams 22:54
Thanks for listening. If you'd like to show, please rate and review us on Apple podcast Spotify, or wherever you get your podcasts. That's one of the best ways you can help us grow this community and we really appreciate your feedback. You can find the full video version of this episode on YouTube. Search for the new stack and don't forget to subscribe so you never miss any new videos. Thanks for joining us and see you soon.
Transcribed by https://otter.ai