Crafting Code Podcast

[00:00:16] Allan Stewart: Welcome to the Crafting Code Podcast, where we discuss the importance of doing the right thing at the right time with the right tools. I'm Allan Stewart, a software architect, and lately I've been thinking about the timeline for completing my journey through the history of Middle Earth.

[00:00:31] Dave Adsit: I'm Dave Adsit, an engineering leader, and recently I've been thinking about software in the real world. In other words, the intersection of software and hardware.

[00:00:40] Allan Stewart: Our topic for this episode is AI and modern software engineering practices. So the question that we want to pose here is what is the intersection? Now that the industry is adopting AI, what does this mean for the practices that we use? I know for myself, at the beginning of 2026 is really when AI turned the corner for me and I started using it regularly for work. Before that, I would play with it, try it. Sometimes it was kind of okay, but usually it didn't seem like it was worth it. But now it's at a place where I'm using it daily. And so what does that do to our practices? What things do we need to consider and possibly change?

[00:01:27] Dave Adsit: Yeah, I will say that my teams, I had them start using AI tooling LLMs back in August of 2025. And we made several evolutions of the tooling that we were using and how we were trying to use it and what we were trying to accomplish with it. And I think that this is an area where there has been a lot of change in a very short period of time, including the change in our practices. And I think that's a huge part of the perception between you and I of the utility of the tools and how much value there is in investing time and energy and money into them. When we first started with them back in August, it was kind of a novelty and it was more because people were interested in it than because it was actually producing a lot of real value for my team. And I would say that around the beginning of the year is when we started seeing the most benefit as well. Or at least... There was a notable change in the amount of benefit that we were perceiving while using the tools. And so what does that mean? Basically, when we are working in an evolving system, our practices have to co-evolve with our activities, right? So if what you're doing on a regular basis changes, then the effective practices, the practices that allow you to do that are going to change as well. So which of our existing engineering practices, the ones that you and I leverage and teach and have found so much value in, which of them continue to be valuable? Which ones become more valuable? Which ones become possibly less valuable if you are leveraging an LLM or an AI heavily in your workflow? Right.

[00:03:25] Allan Stewart: And just... One simple example that I like to pull up when we talk about this co-evolution of practices and activities is horizontal versus vertical scaling, right? So looking to our past to kind of understand this concept, right? It used to be that you would do vertical scaling on your servers because that was the thing, right? You've only got so much space in your server rack. And so you would buy a new machine that was more powerful, add more RAM, More hard drive space to your SATA drive, whatever it was. Like you would keep vertical scaling, just make it bigger and throw away the older stuff that was smaller. But then there was a shift at some point where we said, oh, well, let's scale out. Let's have more hardware. There are... There is virtualization and there was cloud providers and things that made it feasible to say, hey, let's scale out. And when we started scaling out, then the practices changed, right? We don't name our servers anymore. Yeah. And give them clever names because that server is going to be gone in a day or two anyway, as soon as we do the next deployment with our immutable deployments, which is a practice that you would never have done in the old realm. And so the question is kind of where does that take us with AI, right? What kinds of shifts do we think we might see? And what practices, as you were saying, are still important?

[00:04:55] Dave Adsit: Yeah. So I recently... I recently started a new project and I decided this was going to be a project that I was going to very heavily leverage AI on partially because it's a mobile application. And I haven't written a mobile application since the Compact iPac was a thing. Most people probably don't even know what that was. It's so long ago now. But the era of the PDA, right? The personal digital assistant. I know it was after the Newton. Come and gone. So I'm good. It was after the Newton and long before the iPhone. And probably before any of the Blackberries. So I'm not an expert at React Native, obviously. So I needed some assistance on that. So I didn't want to think too much about React Native, but I did want to make sure that this was a project, a product that I could actually continue to evolve over time and continue to work on and would not feel embarrassed to have posted publicly on my GitHub repo. So one of the very first things I did is I told Claude, the LLM I've been using, to go read the four rules of simple design and integrate those into the Claude file. Because I know that one of the things that we've seen many times is that LLMs don't necessarily take the shortest path or create the fewest number of things. And so I wanted to make sure that, A, it passed all the tests, which means it had to be writing tests for everything that was created. And I wanted the code to reveal intention because if I've got to review all this code, I need it to reveal its intention to me and not be, you know, the obscure, obtuse code that's challenging to read. I wanted to make sure that there's no duplication. And, you know, that is something I've seen several times already when you're letting the AI write code. If there's two bits of code that are pretty similar but not exactly the same, it'll just keep them both. Sometimes that's the right answer and sometimes it's not. And then I also wanted to have the fewest number of elements. I want to not have any, you know, unnecessary stuff, clean up all of the dead code, clean up the, like, keep the number of classes and functions low so that it's more easy to reason about the whole system. And so for me, I was thinking one of the practices that I've used extensively on many projects in the past that I brought forward so that I could reason about this project where I wasn't going to be writing the code is four rules of simple design. And to me, that made it even more useful. Because as a, human programmer, I, you start to inhabit the code base that you're in and you know where the good code is and where the warts are and where you've buried some skeletons and you start to get a feel for the whole shape of the thing. But given that I wasn't writing the code, I wasn't actively in the code as much. I felt like I wasn't going to get some of those, some of that intuition and familiarity. And so I wanted to make sure that I put in place some guidelines to help me quickly acclimate. And understand what was being created. Yeah. The four rules of simple design are very powerful.

[00:07:59] Allan Stewart: And I think that they hold up really well as we look into like how has software been in the past? What do we expect in the future? That adhering to those rules brings a lot of benefit, right? So like the duplication one, potentially there's no problem to let AI just keep on duplicating, duplicating, duplicating the same stuff. Over. And over. Because it's a machine and it will deal with that. But eventually you do run into various little quirks, right? Like, oh, well, it takes a little bit longer to compile. Or now we've got a bunch of tests that are testing the exact same thing. It becomes harder to understand, which is probably the most important thing when we're reviewing this code and trying to understand what's going on. But there is definitely an interesting question about like the judgment call. Of what is actual duplication and what is not. I remember a friend of ours used to always talk about the rule of three. Basically let the code get duplicated a couple times and only on the third pass or later, do you go back and look at it and say, Hey, is this actually duplication or are they just two similar things that aren't quite the same?

[00:09:16] Dave Adsit: Right. And so in that case, you're looking at like the concept in language learning is a false cognate. It looks like a word. Or a language or a word from another language, but it actually doesn't mean the same thing in that language. Right. And so an example that I see often is your DTOs for talking to the database data transfer objects. Those have a very similar shape in many systems to the, um, the view models that you send out through the API to the front end. If you're doing CRUD operations, those may look very similar. Um, but I wouldn't. Categorize those as duplication, um, because they are in fact a false cognate in that, in that way, one is intended for one purpose and one is intended for another purpose and they change at different cadences for different reasons. So overall, this is still a practice that is as important and possibly more important. And I would say one of the things about duplication. And LLMs is that the more copies of something you have, the more context below you get, and the less qual. The lower the quality of the output from the LLM, if it's dealing with a whole bunch of unnecessary context, in addition to you having a harder time reasoning about it as a person, in addition to having an over and an over bloated test suite, because you're testing the same concept. Over and over and over because you assume, you know, assuming that you are still passing all the tests, you're going to write tests for every one of those duplicates.

[00:10:47] Allan Stewart: Right. And then in turn, that hurts the revelation of intent. Right. You want it to be really clear what's going on, you know, related to practices with the AI. A practice that I have long held is that I avoid abbreviations. If there are some, there are some abbreviations that are just so commonplace that, that I use. Occasionally it will be something in the domain that we just, we, we repeat it so many times that it just needs a shorter name or it'll be something like SQL, HTML. Right. XML, JSON, right? Like all of these things that are more commonly referred to by their abbreviations. But I was working just the other day on something and it created this variable CM underscore something, something, something. And I was like, what is CM? And even though I was working actively in that code and telling the AI what I wanted, it took me a minute. There's this mental speed bump to figure out that CM was short for crew member. Right. And I was working in the context of crew members. And so then I had to go in and instruct the AI is like, Hey, update your settings. We're not going to use abbreviations anymore because I want to reveal intention about what's going on. And like, honestly, it was never that big of a deal to have longer name. I mean, sometimes it's a little annoying, but it's been a long time since I've had to actively type out a whole identifier more than once because. Yeah. You know, you get things like code completion that does it for you. And now the AI is writing it. So it's typing. Yeah. You do the typing. You can add, you can add a few more characters here to make it more readable for me.

[00:12:36] Dave Adsit: Its strongest attribute is its ability to type really fast. So don't cut corners on the thing you're good at. LL. Right. So. One of the other practices that it was kind of mentioned in passing. Yeah. It's not just testing with four rules of simple design, but test driven development. Does I've, I've told, I've told the AI tools that I've worked with, Hey, we're doing test driven development in this project. Go familiarize yourself with the rules for test driven development. In fact, go familiarize yourself with the rules for acceptance test driven development. So we can double loop this thing. Does it really do it though? What do you think? It's such a great question.

[00:13:16] Allan Stewart: I think that there is an important question here around. Can AI actually. LLMs specifically, can they actually do test driven development or is the best that you can achieve to force them to, to output the tests first? And I say output the tests first, rather than write the tests first, because I don't know exactly how some of these things get generated. And when the sense that I have from what I've played with is that it's literally already designed all the logic. And so it outputs the tests first and then it outputs the implementation. And I don't know that for sure. I I'm not an expert about how they work, but it's definitely not iterating from something simple, right? The simplest thing that could possibly work and building up from there, which kind of makes sense because what it's trained on is how tests should look and how code should, should be, but not necessarily. Yeah. Thanks. Thanks. Thanks. Thanks. Thanks. Thanks. Thanks. Thanks. Thanks. Thanks. Thanks. Thanks. Thanks. like to give it general, Hey, let's build a feature that has X, Y, and Z. I'm much more liable to tell it build this piece. And then I'll tell it the other two or three pieces and connect it all up at the end rather than just giving it free reign. And so, so yeah, I don't know if it's doing any kind of design, the design aspect of test driven development. Yeah. I would say that most of the

[00:15:06] Dave Adsit: time when you and I talk about it, we, we refer to TDD as test driven design because we are writing a test that expresses intention and then going to see how to, how to make that happen in the code. Now I will admit that for me personally, I go back and forth between test first and test driven design, depending on how clear I am about what I expect to happen. I'm maybe I'm not holding it all character for character in my head, the way the LLM would, but I think there's a lot of times when I'm writing the test and I know exactly what the code it is, the code is going to look like that I'm testing. So in that case, I'm not actually doing test driven design either because the design already pre pre-exists before the test. Um, the, the thing about it is that I, I think, I think it feels to me like the code is always written first and then tests are added on top of it. And that maybe that's okay. Maybe this is one of the practices that changes for a human having the test as a, a, a scaffold or as a, a vice or a clamp that's holding your, your mind in place, holding the concept in place for you while you're writing the code is super valuable. And maybe for an LLM, it's not super valuable. And what becomes more valuable is. You know, we talk about the, the suite of tests as a side effect of having done good TDD. And maybe they're the primary effect when you're working with an LLM. When you tell the LLM, you want tests. Maybe that's the primary effect of having told that you want tests is that now you have a suite of regression tests that you can run every time it makes changes across your whole code base and catch some of those unexpected changes that would have otherwise introduced a bug or a failure.

[00:17:05] Allan Stewart: Yeah. And I'm, I'm curious too, if this practice is still valuable, but it just is context dependent, right? So like if I was working with another developer or maybe a team of developers and I was going to review some code that they wrote, but I'm not going to actually actively work on the code with them. It's kind of the same result. Yeah. I look away and when I come back, there's tests and code. Yeah. Both were done where it wasn't test driven. No. Sometimes you can't really tell. Did they, did they, were they already good? They knew what they wanted and they wrote it. And then they wrote the tests after at some point, it doesn't matter. Right. If I'm not involved in the design and I'm not learning at that level, then maybe it doesn't matter. But maybe, but if I don't know how something should work, even with an LLM, maybe I need to take some of these concepts from test driven. Development and say, Hey, I don't know what this design should be like yet. And iterate on it, not telling the LLM, the final desired output, but one piece at a time. What do I need to help me understand? Not, not because it changes the behavior of the LLM, but because it helps me. Yeah. And there's a bunch of related things with that too, right? Like there's the transformation priority premise, which is basically a, a set of rules for how you go from simple to more complex in your implementation. And then your, your tests also change. And I, I'm trying to remember, I think the tests get more specific as the code gets more generalized, the implementation gets more generalized, something like that. These, I think those are still useful, but it might just only be useful in those cases where you're trying to learn or practice. Or figure out a problem for the first time, rather than, Hey, this is the 17th ETL job that we're creating. And they all follow the same extract transform load. And you just say, Hey, LLM, I want another one just like those, except for ABC XYD.

[00:19:28] Dave Adsit: Yeah. Different source, different transformation, same target. Exactly. Yeah. I will say that as I've had, the LLM write acceptance tests, first of all, it's not as good at those. It doesn't have nearly as large of a corpus of training data for what is a good acceptance test. As it does for unit tests. It also has been interesting that if you watch the output, which I don't always do, because I feel like it's only there to keep me entertained. Like writing things to the console as it's doing development in the background, that's mostly for my enjoyment. It will run the failing acceptance test multiple times until the final unit test is in place and the final code is in place. And then the whole suite will pass again. And so I have found that it will do that if I'm very specific about what I want it to do with acceptance tests and that double loop. I think that the assumption for many developers is the test suite should never fail. And so having that failing acceptance test is. Maybe a less common practice.

[00:20:37] Allan Stewart: Another practice that we should consider in the age of AI is coding standards. Things like rules, Lentine comments, naming. These have traditionally been pretty important to us. And one big reason why was because we wanted to be able to read it and understand it. And there's different schools of thoughts about how this will work going forward. Right? There's, there's definitely. Yeah. A school of thought around high level coding languages being akin to assembly level code. And we don't look at that anymore. And so maybe we don't need to look at the, at the high level language either. And we just get the LLMs to work. I'm not there yet personally, for a variety of reasons. And I want to see what the code is doing. So I, so personally, I think that there is still value in making it human understandable, but I've also noticed that when LLMs operate on code that I consider good, it seems to do a better job and be less error prone than when it works off of code that I think is bad.

[00:21:49] Dave Adsit: Yeah. I've been told the LLM wrote the code. Therefore it understands it. And first of all, let's set aside understand, because I don't think any LLM understands anything ever. It can process it. And that's a totally different operation. But. Setting that aside and saying, Hey, if the LLM wrote the code, can the LLM therefore process the code, enhance the code, et cetera. I don't think that that necessarily follows. One of the things that makes it easy for an LLM to extend code well is if the code is clear and obvious and follows the four rules of simple design and has a good test suite. If you have those things in place, the LLM. The likelihood that the LLM extends your code correctly is exponentially higher than if you don't have those in place. And your code is just a big mess with no consistent linting or formatting. And the comments are all lies. None of them follow. And the names are all bad or misleading. Like those are all clues and hints used by the LLM to determine how to improve or extend the code base that you're working on. And so for me, these become more critical because a human can be taught. Hey, in this case, when we're, when it says that we're working on doctors, that doesn't have anything to do with doctors. That's actually clinics. And sometimes that's clinic networks. And so doctors work for clinics, but this table is called the doctor's table. And we're never going to refactor it because it has so many lines in every system. Every application in the system touches it. Right. So you can teach a person that you can possibly create a mapping for your LLM so that it also will run it through that extra processing so that it doesn't do the wrong thing when talking to the doctor's table or the doctor's model. Right. That is, that is something you can do, but you get better code. You get faster, cleaner, more, more effective output when you follow all those rules. And so for me, one of the first things I'm going to do is I'm going to say, Hey, LLM. I want. Good linting for this type of project and just make it and install it itself. Yeah.

[00:24:08] Allan Stewart: I think it's real interesting that the things that we have found useful as far as coding standards, what makes good or clean code seems to also be super beneficial for the LLM. So for me, I feel like that that's a practice to continue. If anything, it's like, this is a place to double down because. It's working great. Yeah. In all cases. And if it turns out that later on, we don't need to go back and read that code. Fine. If it made it better for one computer to make another computer program or evolve your computer program. Well, great. It was worth it.

[00:24:48] Dave Adsit: The code needs to be extremely communicative. If you want to be effective with LLMs. The funny thing about it, I've been told, Hey, the LLM can just write the comments for you. And then, you know, they won't be lies. And I'm like, first. First of all, that's not true because every once in a while it will just hallucinate a comment, like even for code that it wrote. Also, I have found that as I am enhancing a project that I've been working on iteration after iteration after iteration, if I don't remind it to go update the read me and all of that, it just doesn't. And then the read me falls out of sync and starts telling lies, just like a human development team would do. Like the last thing you're going to do is make sure all the comments. Are accurate. Like your primary responsibility is to make sure the code works properly. And if you have time, which you don't, then you can go update the comments, which you won't. And the LLM behaves very much like a human in that way. So the next thing that we talked about is boundaries for AI usage. We've, we've used a lot of patterns for boundaries, like creating modules or microservices or. Um, in the context. Of domain driven design will create a bounded context, which is. Effectively a domain within the larger system that has boundaries around it. All of these are tools that we use for reducing the complexity that we have to consider at any given time. As a human. And I see a lot of benefit for those same practices as applied to systems where you are leaning heavily on the AIs. So. If we are. Uh, for one reason, if, if only for one reason, which is that every AI tool that we have available right now has a limited context window. Right. And you want to stay below the context window limits to get good output. And so if we are working in a large monolithic code base. Where every part of the code could potentially touch any other part. LLMs are going to have as much or more of a problem with that than humans. Are. Maybe as a human, I'll draw a pathway through the things and connect the ones that I'm curious about or working on. And I'll know that there's a possibility that I might break something outside of that. What I'm currently considering. The LLMs just start producing worse output. The more you fill the context window. And so having a way to limit the scope is going to produce better outputs.

[00:27:24] Allan Stewart: I think there's some interesting questions here too, around the context of applicability with some of these patterns. So for example, something like microservices. Historically, there was a strong impetus to use microservices to mirror your organization or your desired organization. If you're going to do a reverse Conway maneuver. And how you're going to break up people into teams and what those communication patterns looked like. There was a lot of match there to, to how you do your microservices. And you didn't want to break up into microservices too early for various reasons. But one of the big ones is that there was a lot of overhead. You have to manage a lot more things. But LLMs are pretty good. At setting stuff up and doing some of that boilerplate work. And so it makes me wonder if one of those preconditions around when is it a good idea to use microservices or, or to use, you know, like a modular monolith or, or some of these things that were a little bit more complex. Thanks. Thanks. Thanks. Thanks. Thanks. Thanks. Thanks. Thanks. Thanks. Thanks. Thanks. Thanks. Thanks. Thanks.

[00:28:55] Dave Adsit: Thanks. that happening. I haven't yet experienced it personally. I haven't done it. I haven't said, Hey, we're all going to follow this architectural pattern where, you know, basically every project is its own microservice and we're going to go all in on this concept. But I could definitely see that because of the reasons you state, like the, one of the barriers to entry. So what do we always use to say microservices is a technical solution to a people problem, right? People can only scale so big. And when you go into a microservices environment, you're trading off the, you're increasing your local simplicity at the cost of greater overall complexity across that whole system, right? You're going to have a lot of duplication of things like, okay, we need data. We're pulling, we're setting up APIs. Well, you can do a lot of that stuff really easily, especially, especially once you have established a clean pattern, right? One of the things that the LLM tools are really good at is saying, here's a pattern from your software. Let me copy it or rather, Hey, LLM, here's a pattern. Here's a repository. Here's my implementation of a repository that I really like copy that do the same thing again for this other entity. And so you can do a lot of that really quickly and more consistently than if you gave the same instructions to a developer team and they didn't have good understanding or good training around, what do you even mean by a microservice in this case? Right. And I wonder if that could

[00:30:23] Allan Stewart: also help, especially when we think about things like domain driven design and the bounded context, this idea that you want to keep related things closer together. And so they have ubiquity of language and things. One of the things that I've often done is have different repositories for like the front end and the backend of some web project that I'm doing or the API and the mobile app I think there starts to become some interesting questions around, do you put those things, start putting those things together more so that the AI can operate in a through line from the database, through your API stack, through your front end stack and help you build out a whole feature end to end, right? Like the vertical slices that we have often talked about and advocated that can be kind of difficult depending on the way that you're working with the API. How your system is constructed, right? Micro front ends is still something that we're as an industry is kind of grappling with and trying to figure out how to do it well. And I wonder if some of those practices could get better and give you those opportunities to narrow your context to, hey, this is just this one bounded context that we're going to talk about, but we can talk about all the pieces of it, the mobile front end, the web front end, The API, but it's a trade-off with some other tricky technical practices that we're not

[00:32:00] Dave Adsit: completely great at as an industry. Well, and one of the other things to really consider there is that one of the reasons why we would leverage a service-oriented architecture or a microservices oriented architecture is because of the scope of the team required to build a feature in a reasonable timeframe, right? If we're talking about, instead of talking about things like lead time or throughput, one of the big drivers of increasing the size of a dev team is that we need features in this area of the system to be built faster. And so maybe you can have a dev team of four to seven developers be responsible for a much larger piece of the system because tools that help generate the code, you can generate code in a shorter time window. Like the lead time from we've committed to this idea to this idea is in the hands of the alpha customers for testing can go way down. Things that would have taken us days or weeks now take hours or maybe days. And so maybe the drive to increase the size of the teams in order to build a feature in a system is to reduce that lead time goes down. And so the scope of work for each individual team gets bigger than it would have been before. And maybe you don't have the pressure to divide into microservices or split up your monolith quite as soon as you would have otherwise. Especially if you're following all of the other practices that allow you to have clean, understandable code that you can reason about as it grows. Solid is one of those things for me.

[00:33:53] Allan Stewart: And I wonder and have been wondering if something like the solid principles can help us find smaller contexts for AI to work in so that you get a really specific result back. Because they're non-deterministic. The LLM won't respond every time to the same prompt in the same way. There's a lot that happens. And so yeah, these are things that I've been wondering about and trying to explore the boundaries of... This is changing. And how do I get the results that I want? Because at the end of the day, I think we've mentioned multiple times in the past about code being a liability. I'm not interested in the code because I want to have a bunch of code or that it has to be particularly pretty or have a certain aesthetic to it. Because I'm not using it for art, although you can. I'm not. I'm trying to meet business objectives. And so where do I shift my mindset in order to achieve these benefits more quickly, reliably, sustainably as the LLM is introduced into the picture?

[00:35:05] Dave Adsit: Yeah. One of the things that that makes me think about is a practice that I've pushed really hard for a long time, which is iteration. I want my teams to work in very small slices and deliver continuously so that we can validate that we're on the right path. So we're looking at what we've produced and we are validating that it meets a customer need before we build the next piece of it. And I have seen a lot more people talking about just writing a plan and then letting the LLM one-shot that code. Now, I know that's not the style that you're using. You've been very clear that you are iterating with the LLM, constraining it very tightly and only letting it operate on... Little pieces at a time. But one of the developers on your team is kind of going the opposite extreme. Sure. And I'm curious.

[00:36:01] Allan Stewart: And even for me, I'm operating in bigger pieces than I would have before. What would often happen for me personally is that I would kind of deconstruct a problem in my mind and say, oh, like even especially ones that I've done a lot. So, oh, for this new feature, I know I'm going to need some changes to a database. And so therefore, I'm going to have a repository that goes with it. There's going to be some kind of a controller. There's going to be... This is because I do a lot of API work. There's probably going to be some kind of like a workflow, maybe some commands and queries. And so in my head, I would break these things down, but then it would take a while to go through each of those pieces. The LLM can build those out more quickly. And so I'm still operating there, but I'm telling the LLM bigger writing it myself.

[00:36:56] Dave Adsit: It's interesting. I have, I've tried a few different scopes. And one thing, even when I've been using the plan modes and, you know, coming up with what seems like a pretty reasonable plan that I want, whatever, I have found that when I do bigger things, even in a very clean expressive code base, the bigger, the feature that I try to get it to do all at once, the higher the likelihood that something went wrong or got missed. The other day I was working on a new feature where I was like, okay, I need, I was describing the feature and we were going back and forth. I was doing the thing where it's asking me questions for clarification. And we, we came up with a plan and I'm like, okay, I really like this plan. This seems like it covers all of the bases. And I was like, okay, go build it. And it worked on it for maybe 30, 40 minutes and then came back and was like, okay, it's done. And I was like, it's not there. It was like, no, it is. It's there. Everything is in place. And I'm like, well, okay. I see all of the code, but you never added any UI affordances or signifiers for the feature. Like I can't use the feature because it doesn't show up on the screen. And they tell me, no, the, the components exist. The components are in the component library. And I'm like, yeah, but not on the screen. So every piece of it was there. And it was like, I was arguing with it. I was like, load up the app and take a screenshot and then draw a circle around where I click so that I can access this new feature that you just wrote. And it was like, oh yeah, the feature isn't added to the screen yet. It's only in the design library. And I'm like, aha, aha. And so what I found is that if I keep things smaller to, you know, what it can get done in five to 10 minutes, maybe that's my current window. My current scope is like, I can describe it well enough that it can execute for that amount of time without me without supervision. But when I try to go bigger than that, the plans are not specific enough. The instructions aren't quite right. I don't know what, I don't know what's missing exactly, but the longer it runs without intervention, the further it gets off course from what I actually wanted. It was like 1% error compounded becomes a big error. Once you've had a lot of iterations and time may be the ultimate arbiter of you will do

[00:39:30] Allan Stewart: iteration, right? Things change in the industry. You need to make changes, new feature ideas come up, whatever it is, you're going to iterate. And I still believe that those small batches, right? Like the same mathematical principles that brought us to small batching in like lean queuing theory. They still hold. And so I, I can't help, but think that that's still going to play in the space of I'm using the LLM and it can, it can generate things.

[00:40:04] Dave Adsit: Well, I mean, I think that it's the same as with any human dev team. If the longer you let them run without feedback, the less likely you are to get the output you want, right? The LLM is the same, but it's writing code at a much faster rate. So you have to check in more often, at least in my experience. I've been told by some people that you don't need to at all. I'm like, well, I don't have an unlimited budget for tokens. So I might need to check in sometimes to make sure we're on the right track.

[00:40:34] Allan Stewart: Another practice that I think still is pretty useful is trunk-based development, contrasting that with long-lived feature branches. But what does trunk-based development look like when there's AI going on? One of the things that I've noticed is, as we've experimented with, in my job, as we've experimented with code reviews, code reviews fit really well into a pull request model. And so you can do, you know, make sure that you have certain standards around how you do a code review and when it gets triggered and all of these things. But it's definitely a potential change for me and how I'm, I've been used to just commit things right onto main. I've been used to having a lot of code reviews, but I've been used to having just in time code reviews with colleagues who are either mobbing or pairing with me. And so we're doing it together or they're working on something else, but it's in the same general area of the code. But the further that we pull apart, because AI lets us work faster on different parts of the system, it may change my activity to need to use more pull requests. And one of my coworkers, a friend of ours is using get work trees and some tooling on top of that to let there be multiple agents. And each one is working on a different work tree, which is slightly different than a branch, but essentially this similar concept of, okay, over here, I'm making one change and over and simultaneously, there's another change going on. So I still think that trunk based is the way to go as far as all of that kind of gets into the code review. And I think that's a really good structure.

[00:42:39] Dave Adsit: Kurt Fanning Sr.: Thanks Kurt. Kurt Fanning Sr.: Thanks Kurt. Kurt Fanning Sr.: Thanks Kurt. Kurt Fanning Sr.: Thanks Kurt. Kurt Fanning Sr.: Thanks Kurt. Kurt Fanning Sr.: Thanks Kurt. Kurt Fanning Sr.: Thanks Kurt. Kurt Fanning Sr.: Thanks Kurt. everybody has the same code. Trunk is the source for your pipeline. It should always be deployable. It should be ready to go. It should be able to move forward. You should always be able to move forward with what is in trunk. And you want to eliminate those long-lived feature branches. Well, your feature branches don't live as long if the code that used to take you a week to write now takes you an hour and a half to write. There's no need to keep those long-lived branches, those feature branches. I think it becomes more critical to integrate more often if you have multiple developers who are all writing a lot of code really quickly by leveraging tools like LLMs. If you're using the AI coding tools, you're going to produce more faster, and then you have to integrate more often. And so I would say trunk-based development is probably even more important for an AI-heavy team to be able to do that. It's a lot more important than it is for teams that are not using those tools or teams that were not using those tools. That feedback loop and ensuring that everything you have can always integrate with everything the other developers on the team are writing becomes that much more critical. And I've also thought about Git Worktree and leveraging that in some of the projects I'm on so that you can have multiple agents working on related things or even different things at the step up into that higher level of abstraction where you're being the PM and the architect instead of being the coder. And it hasn't become necessary for me because of the scope of work that I have found success with. And maybe I get better at bigger chunks of work the more I leverage these tools. I'm not sure yet. I'm not sure why some people are having success with letting their agent run for three, four hours before they check in on what it's produced. And other people find that their agents get lost in the weeds if they're unsupervised for 45 minutes. I don't know what the difference is yet, but I'm open to finding out. But I do think that tools like trunk or practices like trunk-based development become even more important when we are producing code and order of magnitude faster than we used to. I just worry that the temptations

[00:45:21] Allan Stewart: to create more branches and get more distracted. And so I think the human discipline becomes very important because otherwise you may create, it used to be that all your merge conflicts came because you had five people on a team and they were all working the same day. Now you can easily make the merge conflicts for yourself by instructing five agents if you're not careful.

[00:45:51] Dave Adsit: Well, and I've been told merge conflicts are solved because the LLM will just fix the code. And I've seen that go very well. And I've also seen that be a point where significant bugs or defects were introduced because it doesn't understand context. It doesn't understand. And so it didn't merge the two features properly. It solved all the code merge problems, but not the conceptual merge. Right. Of the two features that intersect where they intersected. So I've seen both of those happen. I assume that that will continue to get better, but that doesn't mean that there's, there's value in having a bunch of incomplete work. I think that's one of the things that we need to focus on is staying on track with getting work done. The value of software is in delivery, not in a branch on someone's machine. Right. And so discipline becomes critical. Like even if you are working with get work tree and you have four branches out at a time and you're letting the agent do all handle all the merge conflicts and get all the code in for you and all those things, you're letting it commit and deploy under your name. Those, even when you're doing those things, you need to be checking in often in order to ensure that you don't have a bunch of, you know, like deviations in your code. Like we've got different threads going in different directions that are, you know, becoming compatible.

[00:47:22] Allan Stewart: I mentioned code review before in the context of pull requests and how that relates to trunk-based development. But the code review itself is also a practice that is co-evolving with activities. I've seen some mixed success. It seems to be getting better. And the more you can curate it seems to make a difference. But having LLMs review code can bring up like one of the things that I've often found useful in a code review is when somebody thinks outside the box, right? Like then it's kind of like the thing where you're an author and you write, you've written some sentences and in the middle, there's like an extra word and you just cannot see it. It's not until somebody else reads it out loud to you or that you're like, oh yeah, that's like a glaring issue. But I couldn't see it because my brain was remembering what the, you know, the code review was. Intent was rather than what the actual word is. And I think LLMs are really great at bringing that fresh perspective. The other thing that the LLM is better at than I am as a human is to look at some code and evaluate things about it and about its behavior and how it will work, where there might be some potential bugs or race conditions. A perennial problem with pull request-based code reviews when you're getting another person to do it is actually getting them to do it, right? Are they going to actually check out that branch? Are they going to run that code? Are they going to inspect? Are they going to play around with it? Are they going to try it? Are they going to run the test? That's kind of difficult. And so there's a temptation for humans to just say, eh, looks good enough. The LLM can catch some things that humans are not as easily incentivized or aren't going to naturally catch. In the same way. And so to me, it's not a replacement because the humans are going to catch other kinds of things that are more about like the intent and the cohesiveness of the system. But as a kind of a backstop, LLM code reviews seem pretty useful.

[00:49:38] Dave Adsit: That was one of the first things that my team did was to integrate Copilot into our CI pipeline so that we could get feedback on all the types of things that you just said. The structural things about the code that are useful to review, but are kind of low stakes as part of the review. They're not the conceptual layer where you need to make sure that this is cohesive with the overall system. And so we did find some benefit to that. It's interesting to see when people would argue with the LLM in the pull review or the pull request comments. When the code would get changed and then the LLM would say, Hey, there's a problem with this code. And the person would be like, no, it's the way I want it to be. And the LLM was like, it doesn't follow this standard. And so you'd see people and like, you know, you're arguing with a machine, right? Just do what you think is best as a human. And we'll figure it out later. I will say that it has eliminated the looks good pull request review Such a problem forever. You like, you at least get one solid review for it. Does it follow your linting rules at least? Does it follow, you know, like you've got your linters built into your CI pipeline, but you, you know, the, does it do the things that you said it was going to do? I've had, I've, I've seen some member, some members of my team have their, their PRS get flagged because they described a change. And then the LLM looked at it and said, You also made this other change that you didn't document at all. And they're like, oh yeah, well, I mean, that was supposed to be part of the feature. And then they'll have to update the documentation or whatever so that it, it matches or realize that they accidentally committed a file that was intended to be uncommitted or part of a different thing that they were working on, which comes back to that whole concept of having too many plates spinning at once and too much context in the air. But I don't think that we're going to stop having code reviews. I don't think that the, I don't think that we should dump all of that work onto an LLM and stop trying to do it ourselves. I think one of the things that is critical is that we as humans continue to understand the systems that we're building because ultimately we're responsible for them. The next practice that we've talked about extensively over time is CICD and specifically building CICD pipelines. This to me, feels like something that is even more critical when you are letting an AI write code for you. It's, I wouldn't want code from another developer that couldn't be compiled, pass linting, pass the test suite, et cetera, be deployable to our test environment, whatever that is. You know, I don't, I don't want code from another person that can't pass through my CI pipeline. I also don't want... I don't want code to pass through my CI pipeline that was written by an AI, or I don't want code to bypass my pipeline just because it was written by an AI. I want the same set of rules, the same set of checks to be run on all the changes to my system. Also, I feel like this is a good place for us to add additional, additional checks and additional stop points where we can say, hey, really improving our structure. Thanks. Thanks. Thanks. Thanks. Thanks. Thanks. Thanks.

[00:53:39] Allan Stewart: Thanks. Thanks. Thanks. Thanks. Thanks. Thanks. Thanks. Thanks. Thanks. Thanks. use them to great effect in the past for various automations, right? It runs all the tests. It runs the linter. It does these things so that you don't forget. It's also this great place to put some additional things. It's an opportunity to invoke AI in an additional way to get some other kind of a check that was perhaps difficult to do previously, but now can be done through natural language, have a better way to describe something that was difficult or squirrely to understand before. So for me as a self-described software architect, I think about things like, I'm trying to remember what the name of the book was. There's a book from Neil Ford and some other people at ThoughtWorks, Evolutionary Architectures, I think is the one. And they talk about automating certain kinds of checks to see if as like a fitness function for, is this code following the architectural guidelines that we want to create? And I've always felt like that was a really interesting concept, but I haven't always had the tools to be able to do some of the architectural boundaries or they're just more complicated. This potentially gives you a way with natural language to just tell it, this is what I want. And it can help you check for those kinds of things. So I think that's a really good point.

[00:55:15] Dave Adsit: Well, one of the things that I think about when it comes to CI and CD pipelines is that, you know, back in the day, we used to just log onto the box, the production box and make changes to it. And that was a very high risk activity, low safety activity. And the CI and CD pipelines completely eliminate that. Every change goes through all the validations and you know that it's done the same way at each time, you know, that level of automation. Right. And to me, that de-risks changes. Like maybe I have, maybe, maybe I have the database interaction layer for schema management, manipulation. Maybe it doesn't let me drop the database in the production environment. But if I just let my developers or my LLMs, my AI agents just go make changes to the production database, maybe they would drop the database occasionally. And thing. So for me, it's just one more, one more safety check that prevents us from having problems or creating problems for ourselves by, you know, bypassing those checks. So yeah, I think that CI CD is, is definitely here to stay. I think that one is going to become more critical and to your point, going to expand further into additional types of checks that we may have wanted and never built in the past. The question then becomes what, what are the expectations for developers who are using AI tools? It kind of feels like we're asking for the moon. You know, I remember once upon a time, I worked at an organization where our leader had decided to step away from the company. He was a founder. So he was still going to be very closely involved, et cetera, et cetera. And he asked us to help write the job description for his replacement. And we were like children Mary Poppins, right? We asked for every single thing. We wanted somebody who is expert in databases, expert in system architecture, expert in human interactions, expert in XP and agile, expert, basically an expert in all of these things. And also didn't want us to work weekends and understood the importance of building a reliable, anti-fragile system so that it, you know, basically we were asking for everything. And Mary Poppins did not fly down our chimney and then not become our new manager. I worry that we're kind of asking developers to do the same thing with the AI tools. I've seen teams talk about how you're not a developer anymore. Now you're an architect, but also you're a manager because you've got two or three LLMs or two or three agents that are working at the same time. And you're checking, you're checking in on them and getting their status and setting them on tasks. And so now you're managing and you're architecting. Also, you should be able to handle some of the design and some of the product role responsibilities as well. You know, like at a certain point, you're like, did we just ask this one person to replace an entire team's worth of functionality that we had specifically as an industry split across multiple people, because it's important

[00:58:38] Allan Stewart: that each part be done well? I mean, I think those are really great questions. It's a, it's a It's a really important thing to, to be looking at. And as far as, you know, that question and, you know, the expectation for developers using AI, I just see AI like an accelerant, right? It makes things, it makes more, more of whatever you were going on the floor. If you want to burn the building down. No, no. Well, yeah, but potentially, right? Like you can pour that gasoline accelerant into the, uh, into the gas tank. And so that's properly flowing through your automobile. And it gives you power and this is really useful. You can also pour it into a different part of the car and then stop having a car shortly thereafter. So AI is kind of that same way. It's an accelerant like gasoline. It's also an accelerant. Like it makes everything happen faster. And, you know, like everything is just in motion. And so to me, I don't know what the answer is to, to, to some of those questions, like, is everybody need to move up the stack a little bit, but at a minimum, I think the expectation is you've been handed this powerful tool, this potentially deadly weapon. And so you need to use it responsibly. And we're still figuring out what it all means to be responsible, um, or, or disciplined. But I think there's a lot of that, that discipline required. If you're going to have

[01:00:12] Dave Adsit: long-term success using these tools. I definitely agree with that. I've thought about it as a chainsaw. Chainsaws are super powerful tool in the right context. And when being used by someone who understands them and treats them with the appropriate respect, I've seen people make chain, uh, you know, log art using chainsaws as their, as their carving tools. Amazing. Usually they're carving bears for some reason, but I've also seen people, people who are just very quick at like rendering a tree into usable wood. And also I have personally taken people to the hospital who let a chainsaw jump back on them and hit them in the face. It was very minor, 10 stitches. And we were back at work the next day, but I've also had, you know, seen people let their chainsaw slip and they, it hits them in the leg, very dangerous. So these are very powerful, very useful tools that should be in the hands of, everyone who's doing that kind of woodwork, but also not tools to be treated lightly.

[01:01:19] Allan Stewart: So related to that with the expectation of developers, you know, learning this discipline, there was a interesting podcast that I enjoyed from Hank Green, where he was talking about all of his worries about how the world is going to change and like all the secondary and like tertiary and quadrennary effects of LLMs. But he, he interviewed somebody and there was this question, why lift weights when a forklift is better at lifting heavy things? Well, the answer is that you go to the gym to make yourself stronger, to improve yourself, to be more healthy, those kinds of things. It's not because you needed things moved around. Right. And so I think that the same thing applies with AI. AI is a coding forklift. That doesn't mean that you should always be using you need to be using it in every situation. Right. There, there might still be some things that you do by hand because you need to learn. There might be some things that you, you do differently because you don't have the discipline yet. Maybe you're not ready to use the forklift because we don't want Michael Scott to jump in the forklift and destroy the warehouse. Michael Scott is not forklift certified. Right. So that's one of the things I think about. And then just as these new practices, emerge, like it may also give us new learning opportunities and new ways to think about code. So for example, I think about higher order patterns, right? That's something I'm interested in because I've do a lot of software architecture work, but in the past, it could sometimes be difficult to go through some of those things. So for example, if you really want to understand the strategy pattern really well, you need a bunch of different compatible implementations of strategy to use. That takes a while to develop, but the LLM can write that code very quickly. And so I wonder if there will be new options for learning higher order things, because you can play with it. You can try it and say, Hey, what would this look like? I don't want to spend two days rewriting everything from a strategy pattern to a template method pattern just to try it. But if I can spend 20 minutes waiting for the AI. To do that, and I can compare and contrast and like decide which one seems better. That seems like it's pretty powerful. So again, the discipline then is understanding like, when should you be using what aspect of this, a swift Swiss army knife forklift thing that you've been handed. So at the end of the day, recapping this idea, our practices are going to co-evolve with the activities that we do. And so in as much as we as an industry are adopting LLMs and AI agents and whatever phrases you want to use, it's going to change what we do. And I think it's really important for us to be thoughtful, mindful, intentional about how we let those things change and not slip into an accidental result.