EP46 - Why Developer Experience

Transcript

This transcript is unreviewed and may contain errors.

1. 0:02

   Now all we need to do is record something that doesn't suck. We have to start recording at like 7am, so we're a little less cynical. Funny is usually harder. Funny is way harder. Video works, audio works. It's a $3,500 chair. Woooah! Hello, and you are listening to the Think Big and Code Small podcast. My name is Dan Horton, and I am joined with my friend and co-host, Chris Westerhold. This episode is sponsored by CodeCrafters. Welcome to this week's podcast, where we delve into the critical aspects of enhancing developer experience. Imagine eliminating slow build times, navigating through well-documented processes, and experiencing a streamlined onboarding. Prioritizing developer experience isn't just beneficial, it's essential in today's market. We explore the importance of understanding developers' challenges to boost productivity, reduce tech debt, and spark innovation. Time is a precious commodity in software development, and prioritizing it will make a difference. We explore the balance between developer autonomy and the necessary boundaries required to maintain that order. Simplicity and clarity in the process and tools are critical in maintaining high productivity. And moreover, we'll examine why focusing on outcomes over processes is key and how measuring the right metrics can lead to significant improvements. Join us as we uncover strategies to enhance the developer's experience, drive better outcomes, and transform how you build software. It's going to be a good show. Let's get on it. You and I were playing around with writing a book on this topic, an e-book, and in that, some of the... some of the text or content as we start talking seems to be the same. And what I liked and maybe a good place to start and have some fun is it always talks about, you know, what does this look like? Why even, you know, like what is developer experience? It's a question I get from everybody all the time. But I like the idea of saying, what is a scenario even? What does a day in a life look like? And as we were talking about the e-book, you know, the idea was let's introduce into this. What does it look like? And the narrative was something like, you know, hey, somebody starting a new job. They're coming in. They're eager. They got ideas. They want to work on this. They want to contribute. And then they walk in and you got terrible outdated systems. You have unclear processes to follow. There's no support. I can't find anything. A week later, their enthusiasm kind of disappears. Now productivity goes down. That great... excited developer you have is now looking already online for the next job because the environment that they've been thrown into doesn't support their enthusiasm, nor is it allowing them to be productive. And now they want to go find a new location. So you've lost maybe one of your top developers within weeks of coming into a new environment. Well, you know, I think I've mentioned this stat before, but it's something 70 something percent of, you know, developers are burnt out yet that close to that same amount of people code on the weekends. And that that is a very clear thing. example of, you know, it's not, you know, developers love the act of coding. It's the rest of the garbage that goes around it. That is the problem. And, you know, that is a a great example of, you know, what is the what is the environment in which they live and what are the things that they have to go through to actually get their job done? And another another great and I actually heard this last week from somebody that we've been working with that we were doing an interview with people and one of them said there is a one percent chance that if I go looking for a looking for an answer inside of our documentation that I'm going to find it. Because the environment was so bad for finding information. Right. Yeah. And so what do they do in that scenario? You pick up the phone and call somebody. You send them a Slack message. You set up a meeting. And now all of a sudden you've interrupted that other person's day. Yeah. Maybe maybe they were, you know, in a really good flow state, working on getting something done. And now you've asked them a question that is most likely redundant, that, you know, you've now broken their flow to help solve your problem. And now you so. take that problem and just extrapolate that across the company. And all of a sudden you have this environment that is just a struggle to get through. And when I think of like, you know, what is developer experience? It's that it's the holistic view of what are all of the things that need to happen for a developer to actually get their job done. Yeah. And yeah, I think it's important to dive into this a little bit as we talk through, because your story, your narrative you just went into, I spent years working at one of the companies I was at where... I would deal with meetings and I would deal with interruptions all day long. So your senior developers, right? Five levels of developer experience, right? Your level one developers are finding your most senior level five developers to ask them questions. Unfortunately, they're getting the same question over and over and over again. And they're getting other things. And then people want help. And you think you're one person asking them one question. No problem. What you don't realize is your most senior people are being asked questions from people all day long, one right after the other, where they're not effective. So what would happen is I would drive home. And then as we'd watch TV at night after dinner, I'd pull my laptop out. And I would work until about 11, 12 o'clock at night before I went to bed because I had to find a spot where I could work. I remember that I would get off every year. I would take off on purpose every year between Christmas and New Year's. I wanted to kind of reset before the New Year. I... Would program and write applications I needed to use for the next year to run my team. Like I created a vacation database once when I was working at Bed Bath & Beyond and it was to help me, but then the company liked it. So then all of a sudden it became a thing and now I had to support it, which was a whole different problem because- This was just a quick, you know, I built it in my garage kind of thing to just try to manage my own. But that environment that you're talking about, that developer experience isn't like a plug and play tool per se. It's the whole ecosystem, right? It's, it's everything that's making it so that basically I have to come home, like you said, and I have to program on my couch. I have to program on a weekend. I have to not do other things. Yeah. If, if you're, if you, if the only way you can get people to leave you alone long enough to get your job done is to do it at night, there is a huge problem. Right. But that's what you mean by effective. So effective is because these are easy words to throw out. Right. But people are confused. So I want to be efficient and effective at what I'm doing. But OK, great. Those are just English words. Right. What does it mean? It means that I just posted the article on time blocking because it comes up all the time in conversation. I have developers that are so frustrated right now because they spend 90 percent of their time in meetings. And then in some of those meetings, they're getting yelled at, why isn't their stuff done yet? And then they're complaining outside, and rightly so, they're complaining outside that they have to work overtime. And then they're being questioned, why am I working overtime? So like a huge disconnect. So when you say the effectiveness of that, it's that, right? It's am I giving my developers the ability to have time to get into, like we did in the other podcast, right? Flow state. Am I getting into a flow state, right? Yeah. So to me, yeah, developer experience is about understanding and quantifying what it is that people need. Like, what are they going through on a given day to get their job done? And how do you... Put a focus around that. How do you anticipate what their problems are going to be, the things that they might need and be able to build efficient processes to help them get that job done? And, you know. I talk a lot about kind of a developer experience engine, and it's how do you build that? How do you understand a problem? How do you start to think about solutions? How do you start to measure what that might be? And how do you actually go and start to implement and measure the effectiveness of it? And, you know, building that cycle is. And, you know, is the critical thing. But the first step of that is truly understanding what it takes to go through, like to your to your point about meetings. It's like, you know, you might look back and go, oh, yeah, well, you're only in, you know. eight meetings a week. That's not, I mean, they're only there. You mean they're only 30 minutes long. I mean, that's not that, it's not that big of a deal. That's four hours of meetings in a 40 hour work week. What's your problem? Why are you complaining? Well, the problem is they're scattered throughout. And so, well, if I start my day at nine, we have our stand up at 10, which should be 15 minutes, but everybody knows it's going to be 30. And so you have that hour. What do you do with that hour? Well, you try to get stood up. Maybe you write a little bit of code, but you know that you have this meeting coming up. So you know that you're going to get blocked. So you try to find something to fit into that hour. Well, then you have from 10 to 1030. Then you have another hour after that. Then you have people start talking about lunch, those types of things. Then in the afternoon, you've got another couple of meetings. You don't really ever have time to truly dig into a problem. And when you're thinking about or, you know, and a lot of scenarios, you're having to really understand a complex database, a complex date like code. You're having to understand complex business logic, then figure out how do you actually make all of that better based on some other random requirements like that takes a lot of booting up. And if you don't have more than an hour ever to work on it, what do you do? You end up working at night or super early in the morning or whatever. Yeah. What's the average 20 or 30 minutes to get back into a flow state. If you can ever get back to it. I can remember many, many times where, like you said, meetings were every half an hour, every couple hours. And I have a little bit of time to work on something. And if I had an hour and I found a flow state and I found that I was almost there, I finally figured out the path I needed to take to solve something and Then I'd have to get up and go to a meeting. And when I came back, you never had that same flow state for that same problem. So you had to start over. I was like, Oh, what was I thinking again? Where was this going? Like, I like what I see, but okay, now I got to try to get back into it again. And, and it would, it would break it. The other thing we don't think about at all. I think this is my next article is, um, people's rhythms. Like, are you a morning person? Are you an evening person? Right. I'm a morning person and it kills me when I have to get up first thing in the morning and automatically get into meetings because that is my most creative time for me. Yeah. Like you and I talked about, I used to get up at four in the morning to write my articles that I would publish because it was quiet and it was peaceful and my brain was very focused. And like for code, it was the same. I could crank out designs for stuff and think about complex problems very well. Well, if I'm in standups all morning and I'm in reviews and I'm in all these other meetings, you've used my best time. Stand there and listen to people talk or for some other type of meeting that wasn't as important. So I'm now also making that developer ineffective. Right. And because it's going to take when I worked at night, it would take me three times as long to fix something than it would first thing in the morning just because where the brain sits. Yeah. And I think that's something that gets missed. Like we don't think about it. Well, and everybody's going to be different. I know I go back into my kind of early 30s, and I was most productive from probably 10 p.m. to 1 a.m. That was just when my brain started to work well. And now... I can't hardly stay up past 10 o'clock. And now my most productive time is from probably seven to nine in the morning. And, you know, everybody is going to be different. And so if you think about how do you design or think about, you know, a process for people that is going to allow them to be able to do those things and to get the maximum out of each of them and, you know, There's a million, you know, a lot of people come to me like, oh, where do you start with developer experience? And the answer is always, I don't know. I need to understand what your problems are first. Then we can figure out where it is because everybody's problems are going to be different. Right. And, you know, I know we've kind of talked about like a little bit about kind of poor documentation on meetings. But another another good example of this is. like local development environments or kind of build times in general. And, you know, if, if you don't have an adequate local environment to test your, to test your changes, and this is actually pretty common, what ends up happening? Well, You know, you might have three or four or five minute build times, maybe longer because your local machine sucks and it just can't chew through all that information very well. And your laptop is going to light on fire at some point. Or you just can't get an environment set up, period, because of all these other dependencies and everything else that you need to even get it running. So then if you can't do that, then what happens? Well, then you push things into your dev branch. So your dev branch becomes this like community testing environment. You end up now you have a whole bunch of merging conflicts. You have a whole bunch of other conflicts that come then. OK, well, that's a problem. Then what happens? Well, then you start creating more bigger branches. You have less commits because you're having to try to fight over the dev branch. And this thing just keeps kind of. it compounds on itself. And so now if you think of, okay, well, I'm just a, just a developer trying to fucking trying to test my stuff on my local, local environment. I can't. So now for me to do this simple change, it's going to take me an hour and a half because I'm having to fight over all of this testing and everything else. And it's like, that's a, that is a very real thing that I see quite often. Yeah. Yeah. And what I think is important here in our conversation is most people keep asking, like you said, the question, what is developer experience? What are we doing? Where do I start? So if like this is a great indicator of, OK, you might need developer experience. Right. If I'm having slow build time problems, then your output, your outcome, what you might be looking at might be. a physical review of the code and the program the guys are working on and the team is working on today, right? Do you have large artifacts? Do you have dependencies that are blocking you, right? How many code changes are going through at a time? Physically looking at how you're building your program today so you can figure out, are there techniques and ways that you can do this? Can you break down those? Like you said, can I break it down so I have smaller deliverables? I'm losing the word I want, right? Yeah. Can I break things out into modules? Can I break things out into libraries? Can I make sure my pipeline is cleaner and faster? Do I understand the stages of testing I'm doing and how am I testing through it? Am I understanding how I'm getting feedback quickly? Do I have the right dashboards and things like that? Like different techniques and ways of looking at the code itself so that... I can quickly and iteratively go through and test out whatever I'm building in a way that I don't sit there for an hour and a half waiting for a build. I remember a project we were doing where they would do the build around lunchtime so they could start it and go get food because it would take them a half an hour or better. And then they would come back with food or coffee or whatever else. So when they would do it during the day, they actually started scheduling that as part of the day's events to try to optimize it. Yeah. And, you know, you're one of the things, you know, you leaned in quite well. You know, one of the things you could look at from a developer experience perspective is kind of architecture and code like you're talking about there. But there's, you know, there's also kind of self-service. There's, you know, test data management. There's ephemeral environments. There's all of these other things that can come along with it that is going to heavily impact how well... And, you know, how well your developers are going to work inside of a given a given process. And, you know, all of that all combines together into what do you need to do? And, you know, the answer of what you need to do is pretty easy. You know, and that's why I always go back to kind of, you know, that DevEx engine, because all of these can be a problem. Like onboarding time and onboarding in general can be a problem. To go back to something we talked about last week, I met with the team. They have 117 repos. I'm not even kidding. They have a hundred. This is a relatively large team, but they have 117 repos and God knows how many Confluence pages. So how are you going to onboard somebody? Oh, yeah. And they have three separate architectures. So how are you going to onboard somebody to that? Right. The answer is like, God only knows. It's going to be a huge pain. Yeah. But if you ask them, most likely all that's come from other frustrations that they've been trying to work around that have never been fixed. And now it's just made the problem worse and worse as they move forward. Well, it's it's kind of kind of like Conway's law, a little bit of, you know, you're you're going to build around the constraints in which you have. And, you know, the you know, to me, the the. The fundamental premise of developer experience is to not build around the constraints in which you have. Make those constraints better or remove them altogether. And because you'll see all kinds of dumb stuff that was put together to help solve a problem. Well, make the problem go away. Rethink the problem and come up with a better way of solving things. And it might be painful. It might be hard, but... And it might take three years, but at least... you know, in incrementally make things better over time. And yeah, you might be in a complete disaster right now, but you know, elite organizations became elite, not because they built everything up front. It's because they are constantly trying to improve and make things better. And that, that to me is the fundamental root of a good developer experience. Yeah, it's had me thinking about first principles, right? And people know first principles somewhat because of what's the rocket, X rocket or whatever thing is called, right? It's early and I haven't had enough coffee still. Are you talking about Starship? No, is it Starship? No, it was the... SpaceX thing? SpaceX, that's the one. Okay. So it's early. So it's early for me. I slept late. So... But it actually comes from Aristotle. And the concept is breaking something down to its most basic components. So the story that's out there that a lot of people have read is for saying building electric cars with the battery that it was too expensive because of the battery. So basically, he broke the battery down into all the components, found out what was expensive, found an alternative way to source those components, and then made the battery affordable, thus being able to create the car. We break it down simple. Yeah. when we look at those principles for this, it's a different mindset. And I think that's where it's important. It's like you said, it's, I have to challenge assumptions. So start with the fundamental truths of what is it to build good software? Where am I? What does it look like? And then what am I not doing so then I'm getting rid of all of the noise now start to innovate on that a little bit and say okay once I've broken down to simplicity and I start trying to put it back together what is the noise what is actually needed there and this is not redesigning everything you have it's just getting clarity right and then from there it's function over form don't do it because you read it in a book do it because it's the best thing for your team and And and I mentioned this because and I think we should go through them. There's a couple of very key areas. Right. That can be identified. Right. Working with the team like you called out slow build times. Right. Inadequate documentation. We keep bringing up all the time because if there's not a good amount of documentation. Right. then you're going to keep asking everyone questions or you spend all your time searching on Google for the same answer. And like, I think these are important. So we probably should go through some of them because these are the things that a leader at any level on the teams or in the organization could use to start looking around and saying, okay, do I need developer experience? And why would I even want to go down this path right now? Yeah, I would say yes, everyone needs developer experience. Does it need to be some gigantic effort that goes across an organization? Maybe or maybe not. But there is a mindset to it and a culture to it that a lot of people try to say that they have, but very few people actually do. Because when you actually talk about... you know, taking time to do something, there's a big, big difference between talking about something and then doing it. And a, um, an interesting example is, you know, uh, there's a lot of correlation between tech debt and developer experience. And, you know, a poor developer experience is organizational tech debt, right? Or organizational debt or whatever you want to call it, because it's it's a bunch of bad processes that have never really been fixed or solved. And you just kind of go along with them. Very similar to tech debt. I had to make this decision because it was the right thing that I had to do right now. Three years later, it's going to suck. But guess what? We'll figure out how to fix it then. And, you know, by. you know, there's, there's a whole bunch of concepts for people to like drive constant improvement. Like, Hey, guess what? Every Friday afternoon after, you know, every Friday afternoon, you can do whatever the hell you want. Go, you know, go fix whatever it is, go solve a challenge, go do whatever, you know, it was the same route as like hackathons. Like, Hey, you know, let's take a week. Let's try to figure out how do we solve some shitty things around the organization. And, you know, actually get value from it. A lot of those things have been completely bastardized into, you know, shells of what they, you know, could have been. But or you get a lot of people like, oh, my gosh, we got too much to do. I can't let I mean, we can't take Fridays off to do this thing. We got too much to do. It's like, well, well, yeah, but you're going to pay for the you're going to pay for it. Like, you know, back to the, you know, the documentation example. If every time someone needs to answer a question, they're reaching out to your most senior people, guess what? That is going to drag, slow you down more than a boat anchor. Well, you're not going to build the innovation. You're not going to build in the uniqueness that you need into whatever you're building. Uniqueness is a word, but you're not going to have any kind of innovation. You're not going to have... Your design is going to suffer. Your architecture is going to suffer because the team itself didn't have time to actually go and design and plan. So now they're not proactive. They're reactive. That's right. So now I actually see this on something over the last couple of weeks where the team is being pulled into so many meetings. They can't whiteboard and they can't think about the design and the architecture to level the client needs. because they're trying to push to go so fast. So what suffers? The architecture suffers. And then, like you said, then they are going to have tech debt. And if they don't put a process, because you can manage tech debt effectively, but you have to put the process in place. If they don't put a process in place to effectively manage it, then it just becomes noise and a mess. And it's just a dirty application at the end of the day that you're going to end up throwing out and rebuilding at some point because there's going to be impossible to fix and impossible to manage. That's right. Right. So that time is more important than I think organizations realize. It's what was the statement? Go fast to go slow or go slow to go fast. Yeah. Right. That's right. And, you know, there's also like the, you know, taking incremental steps like you were talking about there. And then, you know, we're kind of talking about some of these kind of first principles. But this next one, I think, says a lot, which is, you know, decide for me, but let me have the final say. Which I think this, this actually comes up quite a bit in, in engineering. There's, you know, the, the concept of autonomous teams that there's a lot of benefit by letting people have levels of autonomy, um, You know, I've argued, I think, numerous times on this podcast that autonomy and anarchy are not the same thing. And by letting people do whatever the hell they want, whatever the hell they want for every possible thing that could happen is chaos. And you can't have that. So, you know, bounded autonomy, I think, makes a lot of sense. But, you know, hey, here is a set of, you know, and to overlay this into developer experience, here is a set of standards that. architecture standards. You know, here's what a good Java program looks like. Here's what a good infrastructure setup looks like. And I can give you all of this for free. You may need to tweak it to fit your needs so you can have the final say of what the final output is. But here's what we think a really good set of architecture looks like. And we're going to validate what this architecture is before you dump it into production. But you can have your ability to modify and customize the way that you need to a certain point. If you go past this point, you need to, you know, it's going to get flagged and you're going to come talk to me. So finding, you know, the blend of, you know, hey, here's standards. Yet we're going to allow you to customize is, I think, another. Really good principle to follow when you're thinking about developer experience. Yeah. And there's a larger piece to that that's important for organizations is that separation of where the guardrails as an organization of what we can use and then what developers have free reign to figure out as they're doing it. Like you said, their ways of working and the things they're doing, but the tool sets might be kind of set in place differently. But if for you to solve the problem, I'm not going to force you into a tool set. If you come to me and you show the value of adding a new tool set or a new process or something else for that to solve that problem, then great. Let's do that. Let's put it into the organization. Let's document what it is and let's use it. It shouldn't be anarchy. Everybody in every team shouldn't just be out there picking whatever they want. And even worse, consultants coming in shouldn't be able to pick what they want to use. I had that at one company I went to and they had a consulting company come in, build them a solution and then leave. And no one in the company actually knew the programming language. The entire tech stack was foreign to the company, which was their fault for not owning that. But like they had to now hire people in to solve the problem, like to own it, to continue to run it. So the... The other part I think is useful where you were going is leader's intent. A team themselves should be given the task in the end state and understand what it is, and they have the autonomy to figure out how to make that happen. Like I, as an organization, shouldn't tell you, you have to do agile this way and you have to do X this way. And like, it's not me to tell you to do it. You as a team, as a low performing team will need more handholding possibly than a very high performing team who just works so well together. They just get stuff done and they might not need story points anymore. They might not need cards anymore. They might not need to have a stand up every day. Because the way they work, it's just ingrained into their process. If an organization forces them to follow all these processes, and I know I'm on a tangent, but it's an important one. If they're following this process, you actually take away their ability to be reductive, right? Like you reduce the productivity and that's what we're talking about. Well, and you you reduce their productivity, but you feel better. And no, you can't. But a lot of people feel better. It's like, oh, well, I am managing this process. So it has to be better. Right. You know, me just leaving it the F alone can't be better than me trying to manage it through all kinds of other things. Yeah. Afraid to be hands off. Well, you know, to me, it goes back to those guardrails of, you know, you have autonomy inside of these guardrails. Go do what you want. You know, you are all really smart people. And as a good leader, I should be hiring smart people to tell me what to do. Not the other way around. And that gives a a team. True ownership. You hear people talk about ownership, ownership. Yeah, yeah, yeah, yeah, yeah. But if, if you have ownership and no control, you don't really have ownership. And so giving them the ability to do that and say like, look, like you do what you want. Here's the guardrails you live within. These guardrails are going to be just fine. Yeah. Because in the end, you know, you know, you talked a minute ago about bringing in new tools and those kinds of things. Like I would say from a an architecture perspective, from a leadership perspective is, OK, well, you know, we're a Java shop. We write Java microservices, whatever. Right. We we understand what this looks really good. We want to try to reduce fragmentation across the ecosystem, blah, blah, blah, blah. And you come to me and say, hey, I want to write these services and go because it has this new fancy feature that's going to make my life a whole hell of a lot easier. Blah, blah, blah. You know, probably shiny object syndrome, but whatever. And. My pushback to them would be, okay, if you want to write things in Go, that's fine. But you are now going to create a Go Center of Excellence. You are going to create and share starter kits. You're going to create and share pipelines. You're going to create and share all of these things. So it is now your responsibility to set up and manage all of this stuff. And so if you truly believe that this is the right thing to do, you are not going to create this thing and have nobody know how it works. You are now on the hook to do the rest of it. And there's there's two different ways of looking at that one. Oh, well, you're just trying to shit all over them so they don't do things. Well, no, that's not true. You're you're trying to weed out somebody just wanting to look at the shiny object and bringing in a new shiny object. And you're being able to say to them that, you know, if you do truly believe this. Great. Own it. Be the owner of it and drive it forward. And that's how you're going to be successful in an organization that doesn't just say, I have my stake in the ground. I will only ever use Java from now until eternity. But you can't also say that we have every language under the sun, including things no one's ever heard of and knows how to even write. Every application can't have one language or one variation of that language. So you end up with 30 different applications with 30 different, you know, tech stacks. It's chaos. And so if you think about, you know, there's the broader, there's the broader experience. So, okay, well, there's the actual act of writing, but then there is your quality. There's your security testing. There's your chaos testing and resiliency and pipelines and all of these other things. And I mean, if you're using containers, I mean, some of that stuff doesn't matter, but you have that whole process that you now have to consider when it comes to working with that new technology. It's not just a developer. Oh, well, I can write this code is totally fine. It's a lot more than that. And so thinking about that broader ecosystem, the broader architecture and the broader experience will help you make sure that you're operating as efficiently as you can. And you're not just jumping into every new JavaScript framework because it seems cool. Yeah. And that's where the standards coming down from, some people have EA group doing it or other groups, your architecture group, right? Review boards, things like that. But that's where that higher level guardrails come into play is this is our process. If we bring in a new language, we want you to follow this process and put these things in place. So it doesn't sound so dictator type of like you must do this to use that line, but- you know, these are going to help us as an organization to grow and scale this language. So if we're going to bring this in house while you're doing the project, start creating these artifacts that will help us to both support the tool and grow as an organization with it. That's right. It's a nice way. Well, and you have this really nice paved road. This can be a dirt road, but it needs to be a road. Like it can't just like, and it has to start carving that path. Can't be a cul-de-sac. Damn cul-de-sacs. Now, the other one that comes up a lot and we just mentioned a couple of times is tools. So it's fragmented or just a plethora of tools itself, right? That are kind of disjointed. Nobody ever really learned to use the tools or they're not connected together to create a seamless workflow. Like I've seen a lot of this and worked with it, supporting a lot of clients I have today with my teams in different areas. I see... where there's consistency of using certain tools and then inconsistency in how they actually use the tools. So very ineffective because I have a hard time switching context to different tools and then trying to find the documentation because the processes are not flowing in a logical way to find the document. Everybody's got their own flow and then each person wants to make it their own version. And it's very interesting. Or they default. Everybody goes, oh, I'm just going to use Confluence and just throw everything in Confluence with no rhyme or reason. And then, like you said, a year or two later, wonder why I can't find anything. I've got 100,000 pages in Confluence. Why the hell can't I find anything? Exactly right. And it's not that it's a bad tool. It's that it was set up poorly in the start, right, to grow. And it's not scalable. Well, and, you know, more tools does not equal more better. Yeah. you know having the right tools and the right way of thinking about it does and you know they're you know a good and this kind of leans in a little bit to you know kind of platform engineering and developer experience that are quite closely linked together but you know inside of a kind of modern devsec ops tool chain there might be 15 different tools that you might need you have your static code analysis your dynamic code analysis you have code provenance you have software bill of materials you have all of these different tools that are and you know doing some kind of thing that is incredibly challenging to figure out you know just take the sonar cube like api if you have to figure out how to work all of that on your own can you sure sure you can you But, you know, the myth of the full stack developer is becoming even more of a myth these days, because how do you understand how to work with all of this crap and be really good at writing Java programs or Go programs or whatever, and understanding infrastructure and understanding security and understanding, you know, all of these different things that go along with everything. And you end up with these tool sets that are... incredibly complicated because they need to be in a modern world. And there's a whole bunch of them that are even outside of what we are trying to do. Um, and, um, so, you know, if we think about a good developer experience, you know, how do you start to abstract away some of that complexity and, And allow them to be able to work in a much better, easier, faster way. So you mentioned you mentioned to actually in the conversation to this week, I think it was we were just brought up time and it's very different thing lately. And it was having like a developer dashboard. It was having the simplicity of or creating a simplistic way for me to find things for me to, you know, manage. And we were talking about one of your other pastimes you had before coming over to developer experience, you know, and your expertise. But like having developer portals and having other ways of hopefully simplifying my environment, I guess I would say. Yeah. So if you so if you think about think about something as simple as a like Sonar Cube. So Sonar Cube, not incredibly difficult to work with, but it's a thing that you want to work with. The output from Sonar Cube can be quite confusing and confusing as in like, what is the thing that I need to care about? Right. It's going to give you all kinds of stuff. And what are the things that I need to care about? Well, so if you think about this from a developer experience perspective, I am going to put this thing into I'm going to have it check my application inside of my pipeline. I need to understand how to work with its API, how to call it. I need to understand how to read the output of that. What are the things that are actually care about? Then I need to actually do something about them and then figure out how to, you know, how to proceed from there. Well, Every other application team inside of your whole organization is going to do that same thing. They're going to struggle with that same problem. So if you think if you put your kind of developer experience hat on and your platform engineering hat on, why would I not abstract away complexity? So instead of calling the the the Sonar Cube API directly, create an abstraction layer like a CLI. that says, Hey, I want to do static code analysis, completely forget that it's owner keep under the hood. And Hey, here are the things, here's the basic things you need to feed me about your application. Then I can cure like as an owner of, of that, that application, you know, doing that type of security analysis, I can then craft the output back to you that says, these are the things that you should care about. Oh, hey, here's only, you know, CDEs that are higher than seven. Like this is the thing that's going to stop you from getting deploying into production. You should think about these other ones. But these are the things that you have to consider right now. That you are lowering the cognitive load that it takes to understand that and be able to now move forward instead of spending all of your time trying to understand the mechanics. So from a developer experience perspective, if I can easily run this thing through an easy, abstracted and the the data that I get back is the things that I need to care about, then I can go back into my dev portal and see all the results. That's fantastic. That makes my life a hell of a lot easier. And you're removing toil across the rest of your organization. So instead of every freaking engineering team struggling with the exact same problem, you've now made it easier for everyone. And then you can also, on top of that, start to apply governance that says, OK, well, we are you know, we now understand what applications have, you know, CVEs greater than seven. Now we can actually start to think about actioning upon that and. And so as a as an architecture, as a leadership of my engineering group, I can now say, well, hey, guess what? We're not going to like if you try to deploy into your dev environment or your pre-prod environment, we're going to allow everything. But we're going to block things that are greater than seven. to get into production or whatever. I'm just making stuff up, but like you, now you can, you have a clear path for what needs to happen. It's easy. You've set a standard and you can control that standard from you centralized control over what's acceptable with federated ownership and ability to manage those things. Right. And simplicity and clarity of the message. So I know why I'm looking at something or what I'm looking at so that I can action it. Like you're saying, instead of going through 80% of statistics or details that I don't need, like today we're looking at this and I mean today relative today, we're looking at this for this problem, right? Metrics in general, and we've talked about it in the past, but they're, Once you try to manage by metrics, it fails, right? So you have to ask the questions of why are we measuring and what are we managing? We can manage to plan something, we can operate something, we can optimize something, or we can inspire or aspire, right? So if we understand why we're doing it, then we pick what we're using, like you just said, and then work forward with it, move forward with it. That's going to allow us to add a lot of productivity and simplicity into the process. Right. That's right. And, you know, if I were to, you know, to beat on like, you know, if I were to go talk to any leader that is interested in developer experience or developer productivity or any of those types of thinking, simplicity is the only way you're going to be able to actually solve these things. Right. And by abstract away complexity and then provide really good feedback loops and feedback mechanisms, because to go back to that example that I talked about, that problem is only just for one thing. So like just sonar cubes or just your static code analysis, you've got six or seven other different things that have to run through your pipeline if you're actually interested in having good security and quality and everything else like that. So. You know, as a you know, as an as an application engineer trying to just get freaking code into production, there is a lot of crap that you have to get through. But that crap is necessary. So how do you simplify that to make everyone's life easier yet keep high levels of quality? Because in, you know, in a lot of the last handful of years, there's been the, you know, kind of the concept of you build it, you run it. Well, I agree with you build it, you run it. But you have to have some level of centralized control because you build it, you run it is chaos. Because every team. Well, if everyone owns all the tools that you're using and they own all your environments and everything is out of your control, then it's impossible to do that. Yeah. The other thing, and you bring up what's really important, is the lack of understanding of what we do for a living as technologists. There still is today. Great. People play with their computers. They surf the web. They understand some chat GPT a little bit now. It doesn't make them technologists. They don't understand. Most people don't understand what really happens underneath. And what I mean by that is all these tools and all these complexities don't surface well inside of an organization to most leaders. And because of that, you see a lot of pushback to add more process into what we're doing, not to clean up the environment, which is why DevX is such a big thing right now. It's pushing down saying, oh, you must need better agile or I need to hire more scrum masters or like we look at it from a management control, which then slows down the development teams more. We're not empowering. We're, we're, we're, you know, diminishing their ability to be effective. So I think it's really important in this. You're, you're making the organization more complex. So if you're, you're like, you know, agile or well, at least the way that a lot of these agile transformations have happened, you've just made your organization more complex. Instead of making it simpler, you're making it more complex and no wonder things go slower and things you're not getting what you want out of it. And worst case is the developers now are done. They've had enough. You didn't clean up my environment. You're not helping me where I'm asking for help. You're now putting more people above me that are just asking me questions for more data that I don't have time to pull together. I'm leaving. This company over here just offered me good money. Their environment's simpler. I'm going to go over here. And what happens is all of your good people leave. Now you're left with the developers that don't know what they're doing or they're just bad or they don't want to work or whatever. It happens. And now you have all of this leadership roles per se that are pushing the last set of developers who don't really know what they're doing and do need more help. And what happens? You fall apart. Everything falls apart, right? Which is why, like we said in the other one, 70% of all IT projects are over budget and fail, you know? Yeah. And it's and it's because of the reason they fail is because of complexity inside of an organization. And you want to go do this thing and you have this kind of rosy view of how your organization actually works. And when you try to come up with a solution, you end up running into lots of problems. And so here's a great example of that. And, you know, ephemeral environments are really great. Hey, you know what? We could use ephemeral environments to help with our local testing problem that I talked about. So now let's actually try to play that back. Well, how do you actually go to have ephemeral environments? Well, you need declarative. You need to be able to declare what an application of dependencies look like. Well, you also then need containers and images to understand what those look like. Well, you need better base images. Okay, well, then you start to walk this thing back and it's like, whoa, man, I was just trying to do some ephemeral testing environments. Now you're talking about me having to change... All of this other stuff. Why? Well, because because if you really want to get, you know, having ephemeral environment sounds simple. It is a much harder problem to solve than just throwing it into there. And if you have really bad process everywhere else, this is just going to show how bad everything else is. Yeah. And I think that's where the first principles part comes back is when you take the time to break something down to its simplest components. Like you said, it does bring out where all the disconnects are, all the pain, all the pain is, you know, where are the bodies buried? Like, where are all the problems? Right. And and then from there, you have to decide, is this something that. Risk reward is the risk of pulling all this stuff apart and trying to create new things. Because like we talked about, yes, last night, right? It's a greenfield. Like you're never going to have a chance to do this greenfield. Everything is always brownfield. So do I want to start pulling parts of things apart in my environment? to provide this new functionality, this new service, something else I'm doing, but you have to come back to the basics. You can't just say you want it and then hope the teams are going to figure it out and then get mad at them when it takes them months to fix all this stuff either. That, that piece of clarity has to be identified. We have to write, what is that? What are the basic components? So we understand what we're touching. Yeah. Yep. And they're, you know, getting that, that, that clear understanding is key and um you know i know we're going to talk about this in in a future episode around like how do you actually go and start solving some of these so i'm going to i'm going to not dive down into that here but um you know i think a little bit is good but yeah we're going to do that in the next next couple right yeah i mean you know getting a really taking that step back and being able to say the emperor has no clothes is that first step of, you know, are your people able to say this sucks and we should do something about it. Right. And, you know, the, there was a, the, an interesting, um, Conversation I had on LinkedIn the other day from so Laura, who's the CTO of DX, put it out there, but she she she had a post that was something along the lines of like, if I if I was new to becoming a CTO, the first thing that I would do is basically get everybody to do productive complaining. And it's like, it's not actually a bad thing. I didn't have all this information. Right. Like, tell me, like, what sucks? Tell me all about it. Right. Then there's there's a there's a like the other side of that is just like everybody complains about everything and you can't get any any decisions actually made. But that's a whole nother problem. But. understanding where the issues are and the problems that are there is just so critical to this. It's like, Hey, I got to put in, if I want to do onboarding, I got to put in like 15 different tickets to get people access to actually get their job done. Like that's a problem. But in a lot of organizations, that problem exists. But in very few organizations, will they actually do anything about it? Because they don't consider that as, you know, they look at that as a governance mechanism. Right. And just, you know, oh, well, this is the way we've always done it. Well, yeah, but that has a rippling impact across your organization around productivity and everything else. And so rethinking these kind of big systems and processes is the way you're going to be able to be productive. And it's a snowball effect. Which one of these things? Yeah. Yeah. Sorry. Snowball effect is what I keep thinking of when you say that, because it, it does every one of these little pieces, these little components adds up. And instead of it being a 32nd difference in a pipeline, or, you know, it just takes you a minute to put that ticket in it, it equates to weeks or months worth of lost time. Well, you know, if, if you think about, think about a ticket that, And a ticketing system that, you know, I think a fast turnaround for a lot of tickets would be what? 48 hours is probably like a kind of SLA that people would have around getting a ticket approved and put together. Might be longer, but yeah. In a lot of ways, it could be way longer. But even if you take just like, OK, it takes two days. Well, what actually happens? Well, I need a thing. to do the thing that I need to get done. Okay. Well, I need to put in a ticket. So what am I going to do? I'm not going to sit here and twiddle my thumbs. So I am going to start something else. So I'm going to, you know, be like a good little, little engineer. And I'm going to go to my, my Giro board and I'm going to grab another, another card off the, off the shelf. And so I'm going to move that into in progress and, My other ticket is in progress, probably move that to blocks, but still. So now I have two cards that are open inside of my sprint. So now I am going to start this other new thing. Then I'm going to be in the middle of that. Then the person I put in the ticket is going to give me, hey, what about this particular question? Then you're going to have to jump off of that thing back to this thing. Then I'm going to have to answer them and I'm going to go back to my thing. I'm going to try to get that done. Oh, my ticket's done. Nope. Got to jump back to that. And all of a sudden you're playing back. bounce around of things and you're having to constantly change what's loaded into your brain around what you're trying to accomplish. And that's going to make you way less productive. You're, you're going to never get into a flow state and you're constantly bouncing around. And then you get to the end of the sprint or even through the first week. And it's like, well, shit, now I got three cards open. And I'm not making any progress on any of them. Then your scrum master or product owner or whoever is like, hey, man, why didn't you get your stuff done? Well, now all of a sudden you need more management. And this stuff just it's a snowball effect of just constant battles of, you know, I'm not actually trying to do my job. I'm trying to fight through the organizational challenges that are there. And that stuff exists everywhere. Well, the part that I think is, and I agree with you, my brain went the same place you went. It goes all the way down to at the end of that sprint, I'm getting yelled at because I didn't finish my stuff. And you don't want to hear it when I tell you that all this stuff is wrong. You just think I'm screwing off. Yeah. So now that person ends up getting escalated to their manager. And now they're in conversations of why are you a low performer? And maybe I'm going to have to get rid of you or maybe I have to put things in process because you're really not doing your job well. And you've completely destroyed that person's enthusiasm to do their job. Or think about it from the team level of, well, the team can't get their sprint goals and their sprint plan done. And, you know, they're constantly having to move, move things from this sprint to the next sprint to the next sprint. You know, and why can't you guys get your stuff done? Well, what you're going to need more management. You're going to need more metrics. You're going to need, you know, in, you know, my favorite is, oh, instead of just having a refinement session, I need a pre refinement session. Yeah. You need a pretty pre-refinement session to make sure that we get our plan right, because we can't keep, you know, moving cards out because the product owner, the scrum master is being judged by how well their sprint plans go. And, and so you now like you have all of this management stuff going on that has nothing to do with really anybody not doing their job. Well, it's having to wade through the bullshit of the organization, right? And that's where you're spending your time on, not around actually make, not actually solve it. Like what's the damn thing that you want to get done? It's whatever the hell this is on this, this feature, not all of this other nonsense, but you're forced to. Yeah. You, you bring up two, two good things in there too. One was that the metrics that we're using end up screwing up the process more because if we're going to judge you based on your story points or your success of how many cards you got done or whatever that is, we're not actually measuring to the right thing because the output we want is the code, not the process. Stop measuring the process. And that happens way too often. I talked... with another really great leader i know and and he had mentioned the thing he's like just give me a couple people in a garage and i'll just get this done in a third of the time you guys are going to get it done and i thought of that again i'm like he's right because if you gave me that development team a couple pizzas and a couple beers i guarantee you we'll go build your product in a third of the time because all the things that slow it down we're going to get out of the way now that It's harder in a large organization if I have to follow along with all of, like we said, the brownfield that's already there. If I have to deal with all the environment, it's going to be slower. But if you allow me to get rid of all the noise and create a high performing, excited development team, I guarantee you they can move mountains. The problem is almost every organization that is out there today has gotten stuck more in what we're comfortable with, which is building processes. And then we start to... The thing, the way I see it, it's a whole nother podcast. Maybe we should do two on this is people fall into process over read out over outcomes. And it's because we feel comfortable when we understand the process. So we default to the process instead of the outcome. The outcome is unsure the process I know how to do. So I get excitement and satisfaction in saying that I know my process is good and I'm making it better, you know, and that's where I think a lot of this gets lost. Yep. I blame Peter Drucker. you know i mean what can't be measured can't be managed i mean and how do you funny thing is everybody takes the last part of that quote off but like he wasn't even really saying it the way we use it but yeah i know i know but like i know you know it's been bastardized and twisted over time and all those kinds of things but i mean it's that kind of thinking of 100 you know you need that's why people manage by metrics now is because of that quote yeah Yeah. I mean, and, you know, there's an awful lot to it. And it's it's sad to see that that's where we've come. But, you know, it that's that's what it's turned into. And if if, you know, I know one of the we haven't talked about this in a while, but we had the concept a while back around has software engineering become a blue collar job? Yeah. Oh, yeah. We talked about that a long time ago. And, you know, that kind of overlays into this. Like, are you trying to be innovative or are you trying to, you know, just crank out widgets? And, you know, what do you want? And it is up to leadership to be able to make that happen. And, you know, again, we'll get into this in the next one. But, you know, what? and how you actually go and create these things, but you need to really ask yourself, if you're thinking about developer experience, what are you, what is it that you want to be when you grow up? What are you trying to accomplish? Are you one faster typhus? Or are you actually trying to change your business and trying to drive better outcomes? And a lot of people, I don't think they're doing it on purpose, but a lot of people that talk about developer experience and developer productivity want a faster typer. They don't want to do the hard work. They just want they just want, you know, the poor little developers fingers to light on fire because they're typing so fast. That's not the problem. Ninety nine percent of the time. But, you know, that's what they're actually looking for. And, you know... Well, change the focus, right? And again, this is maybe a good article. I was just thinking about this. When you think about outcome-driven and driving towards outcomes, if I have a sales team... And I say, I want the whole team to work as a team. So we sell more and grow the organization. But then I only give the vacation to the person who sold the most, right? The one person wins. It's not a team effort. Everyone works against each other to try to be that top performer. We do this all the time by measuring the wrong things for all the team members that are on the project of work, the program we're doing. If I change it, And I make sure that the vision and the outcome we're working towards, right? Simon Sinek's The Why. It was a good book for it, right? If we're all focused on the same why, which is driving increased value to a customer, and then I reward all the people who worked on that project for driving that value out to the customer in some way. Now, how we do that could be different, but we all drive someone in the same direction. All of a sudden, that team will work together to drive the same common outcome. Right. Because you're getting your salary based off that outcome. We all are not. Scrum masters telling developers how to do their job, to try to write code, to try to solve something that nobody's paying attention to. Like I said, mindset, it's a culture shift, right? And that is possible to change. But like you said earlier, and I wrote this on my last article I just put out, stop thinking of technologists as line workers and realize that they're artists who need to have time to be creative and solve problems. So it's helping them and helping the whole team to solve problems. That's right. We don't do that. We don't realize we don't do that, I would say, because everybody wants to do it. Like we're not bashing anybody for this. But the problem is cultures in itself creates a certain mindset. And in that mindset, we stop looking at the moving target. We start looking internal and trying to fix all the internal stuff instead of external. Wow. So in the end, and I think we've probably beat this thing to death as usual in the end, you know, figure out what it is that you want to do. Listen to your developers, create a better overall experience. you know, environment for them to work in and remove waste and friction. And if you can do that, and if you can do that, you can, you know, this was one of the slides at the end of one of my conference talks recently, like, is that light at the end of the tunnel, a train to come run you over? Or is it the sunlight shining through and a brighter day is on the way? And if, if you can focus on, you know, getting people to believe that it's a brighter day coming down the line, that's, that's really all you need to do and build in processes to constantly make things better. Cause if people, people want to be a part of, of something. And if people believe that, um, You truly want to make things better and you're going to give them ways to provide feedback and allow them to be able to go. You're going to look back in a couple of years and go like, wow, we have gotten way better. And if you don't want to do that, then it's going to be it's going to be the same shit over and over again. Yeah, it's not going to change, right? Reducing the frictions will allow team members to be happier, which thus creates high-performing teams, which will then have 10x output without even asking to have 10x output. So you're no longer pushing and measuring for it. It will just happen. Just happen. Happy people enjoy their job. They don't spend all their time complaining about their job. So they're generally working more anyway. So fix one thing to get the other. It's an indirect thing. correlation. It's not a one-to-one. I don't just put a tool in and it works. I don't just add a new role to manage and it works. It doesn't do that. Now, the last piece I think that's important moving away from what we have here is this is identifying that you need it. And we believe every organization needs some form of this. The other thing is getting the current state, which you and I have talked about a lot. Some organizations don't want to take the time to go ask the question. And it's important to get this data because without this data, as we've just identified, there are so many places to make changes that you could get lost in trying to make effective changes. So I think it's important, right? To whether you use a tool, you do it yourself, whatever else, but It's jumping in there and getting this current state assessment kind of done. So at least know where to focus first. So I can do like an 80-20, right? I know these 20% of my changes are going to have an 80% impact on my teams and thus more productivity going forward. Yeah. I think we'll, I think we'll save most of that for next time because I totally agree. And that's a whole topic we could spend an awful lot of time. And to that point, I think that's why I bring it up is because I think that's going to be one of our next discussions we talked about as well. And so people who've heard this and they're interested, we're going to dive a little deeper and we're going to have another couple of sessions coming up, but we're going to continue to dive into this topic and really explain it. and really help people to try to understand how do I get into this? How do I go forward? What should I be looking at? What does good look like? As we go through, we're going to continue to share that content. Love it. Thank you, sir. Excellent as always. Later.