Crafting Code Podcast

[00:00:23] Allan Stewart: Allan Stewart, Software Architect, and Lately I've been thinking about the complexities

[00:00:27] Dave Adsit: of managing sales tax. And I'm Dave Adsit, a VP of Engineering, and recently I've been thinking a lot about team culture, culture events, and department morale. Our topic for this episode is premature

[00:00:41] Allan Stewart: optimization. I think sometimes it is just so easy, so tempting to want to go in and optimize some code, often because we're worried we'll never get another chance or have some other reason for trying to optimize our code. Maybe we've been burned before or something else. But this is generally considered a bad idea. Why? Why is this bad? What makes it premature?

[00:01:12] Dave Adsit: Isn't all optimization good? Well, one of the things I think about is that we often start optimizing before we know what the actual bottleneck in our system is. And that's when we know from the study of queuing theory that if you try to make an optimization that's not at the bottleneck, it will affect the overall system. So if you think, for example, I have 10 API endpoints in my system, and one of them is very slow, and I think, oh, I'm going to go optimize that. I'm going to go optimize that. And then you realize it only gets called once a day or once a week. You may have wasted a lot of engineering effort on something that wasn't the bottleneck in your system and didn't actually help you improve the overall performance of the system. Donald Knuth said, we should forget about small efficiency, say, about 97% of the time. Premature optimization is the root of all evil. And yet we should not pass up our opportunities in that context. Right. There are opportunities in our system where we can actually make huge benefits because they're critical path. They are bottlenecks that cause the system as a whole to be underperforming expectations or our desires.

[00:02:44] Allan Stewart: I like that quote. And I think it's one of those quotes that you have to remember to broaden the context of the quote. It's easy to just take the little zinger, premature optimization is the root of all evil, and then just throw that around. So you have to kind of look and make sure it's like, oh, well, is this actually a premature optimization? Is this the one that's actually going to be the one that matters? Right? Because if you're just going to play the odds and it's 3%, it mattered, and 97% it didn't, then yeah, you're not going to likely make a difference. Not unless you measure it and understand, yes, that there is a reason for it.

[00:03:26] Dave Adsit: One of the things that I think about when it comes to premature optimization or timely optimization is the scale of the system that you're building. There are just so many different scales that a system can operate at, and they each have different needs, and we have to operate in different ways with them. When I was early in my career, I worked with some guys who built line of business software for small companies, the scale of the system that you're building. And we talked a lot about the different ways that you touch data, depending on how much data there is. One of the guys that I worked for, he would say, if there's 10,000 lines, or if there's 10,000 rows in the table, you query it one way. If there's 100,000, you query it a different way. And if there's a million, you query it a different way. And those are all pretty small systems now, and I probably wouldn't query the data different ways, given the hardware I have available and the database engines I have available and the a million lines is just getting started for a lot of line of business tables. But he was right. At the time, in the context we were operating in, it really mattered how much data you were looking at, how you would choose to query it, and what you would choose to do. And I think that that's where we get lost. If we treated every database query the way we do a table with extremely high row counts, well, it would slow us down substantially. And it substantially, making it hard to implement any new feature or build anything useful for our customers.

[00:05:02] Allan Stewart: It makes me think that optimization isn't a pure good, right? There are different axes along which we can optimize. And if we optimize too much on one thing, we can say performance, but even then it might be like memory performance, or it could be time performance, or it could be optimizing for database usage versus your, like, server code optimization, right? There are different things that can be optimized. And just optimizing anything isn't necessarily good, right? We've, I think most people have encountered bits of code that were heavily optimized for some task that made it much more difficult to understand or to process or think about. So I think you have to look very carefully at what you're going to optimize and why and what are the consequences of doing that. And the, you know, the premature status often comes from the lack of that looking from the lack of the measurement and saying, Hey, does this actually make sense? Do we do we really need it right now? I found that in a lot of cases, it's premature, because somebody will get overly concerned about a specific issue. Or perhaps they're overconfident about a particular solution, they're hyper focused on, well, it's got to be this thing, this, this design, or this principle, or this pattern. And then you get into some of these other problems, right. So like, one of the ones that we've seen quite a lot are early extraction into microservices, where somebody says, Oh, we need microservices, because microservices are, are the thing. And we're going to optimize, we've got a monolith, and those are bad. And suddenly, they're moving out into a much more complex and complicated space that is harder to maintain. And it may not actually be the same.

[00:06:56] Dave Adsit: fit the context of the business. Yeah. One of the things I'm thinking about is, you know, as we talk about optimization, I think the natural thing to think about is performance optimization, but you could also be optimizing for other, any of the other, uh, illities, the other architectural characteristics of your system. One that we, we have a heuristic for is the, the concept of the, um, the reliability metrics, right? We talk about, we talk about reliability in terms of nines of uptime. And we often say that each additional nine costs an order of magnitude more to deliver. And so you have to be balancing things. What is the cost of building and running your system versus, um, the reliability that you need or the performance that you need or whatever, right? We could all go by, well, I say we could all go by the biggest beefiest servers and run huge clusters. And the biggest databases and whatever, back in the day, we used to say, we're going to buy, uh, there was, I don't know, a specific data center server that was 32 cores. And that was a lot back then. And you could get Oracle, a full rack of Oracle exadata to run your data on. But the thing is, is that most of us can't do that. We don't have the budget for a $3 million per year database server, right? And so we're going to be, we're going to be, we're going to be making trade-offs regardless. And so we should be making trade-offs in a smart way, in a way that aligns with the use of the system. And I, I really think that measuring first is the most critical thing that we can do to understand where, where are we meeting the needs and where are we missing the needs of our user group? One example of premature optimization

[00:08:45] Allan Stewart: that I have seen, uh, multiple times is database tuning. And sometimes it happens because there's, uh, like a dedicated, database person or somebody has a specialty there. Sometimes it happens just because, uh, there's a developer that gets a particular drive to go in and let's make, make these changes. And it can often be useful. Uh, a lot of the legacy systems that I've worked in will have issues, right? There are things like missing indexes and that there are things that, that slow it down. Um, a lot of these systems that I've worked in, And so when I'm working on them, there's a ton of database joins, right? Like the, the database is highly normalized. And in order to get anything out of the database, you got to join one table to a second table and then that one to a third table and then the first table to another different table again. And so optimizing that can be good, but you have to be careful because what I've seen is occasionally people will go in and they'll spend a lot of time trying to make all of those more indexes for various query patterns. And eventually you still have a lot of the data coupling problems that you originally had, but now you've also got a new kind of performance problem because the indexes work better. But if you've got a high write load, it's slower and there's more contention in the tables. And so it can be, it can be a little bit tricky to judge that. And you have to take that step back as part of that optimization work and say, hey, we have a long-term goal that we want to break up some of this data. We shouldn't be joining all of these things together because they aren't, they aren't together. For example, in domain driven design, you might look at things like aggregate routes to figure out what things you should be able to pull from the database in a single, single join. You might want to have separate tables for like analytic style reporting or something else. But if you just work on optimizing, the database to, to the exclusion of other solutions, it can, it can be damaging. I've, I've seen us, I think there was one time when we were working together where we accidentally had a outage for almost an hour because we wanted to run what we thought would be just, just add this one index. It's not going to be that big of a deal. Turns out that it was kind of a big deal because the database locked up while we were waiting for it to create, that index.

[00:11:25] Dave Adsit: I mean, it didn't lock up the whole database all at once. It just locked every single page sequentially. So it was fine. But I think, I think honestly, when we talk about database normalization, we are talking about a form of optimization and we need to be, we need to understand the characteristics of that normalization and whether they apply to our system, right? Database normalization is an optimization around storage space. Storage space for a long time in computing came at an extremely high premium. Storage spaces doesn't really come that high of a premium anymore. You can get a very large drive for a very low price if you really needed storage. But if you look at normalization, it is an attempt to, it's an attempt to do a couple of things, but one of the primary things is reduce the total storage volume for the database by not repeating data. And so you've got, you've, you've extracted everything that's potentially repeated so that you can use inexpensive foreign keys to optimize for storage space. It definitely counter optimizes for performance. If we were to optimize our database for performant reads, we would have flat tables, right? We would have heavily indexed tables that are pretty flat. And we would optimize the indexes around the specific queries that we're performing. If we wanted to optimize our database for very heavy writes, we would, you know, follow a different pattern. We would not have things like indexes that slow down the write throughput.

[00:13:13] Allan Stewart: Another one of the things that I think normalization attempts to do is to have data integrity, right? So that you could say you, you, Like if a, if a customer has an address in your database and you need to reference it in various ways that you want that address to be normalized. And so that it doesn't get, if you're storing it in multiple places and multiple tables, because you've denormalized it, then sometimes it could get out of sync and it might be wrong. And, but if you're not careful and you over optimize and normalize your database, you get into some weird situations where it's like, oh, well there can only ever be one of this address. And so these two different customers who happen to live at the same like mailing address, or, you know, they used to live in the same house and now one of the roommates moved out or something like that. And all of a sudden they changed their address and it changes it for another customer. Or they changed their address and it changes the history of all orders that you've ever shipped. Even though you used to ship it to their old house and now it ship it to their new house. And so, um, yeah, th th there are some interesting problems that can occur in that space.

[00:14:30] Dave Adsit: That's an interesting concept, right? If you you're optimizing for data integrity by ensuring you only have a single representation of the data, but the data has historical relevance, such as shipping an order and generating an invoice. Well, I need to know what the current order was shipped. But if I want to look at an invoice, I want to know what it was, was at the time that the invoice was generated. Right. And I we've both worked in systems that have not understood the full life cycle of that type of data, where if somebody goes in and updates their address later, it changes every invoice in history and the invoice then loses its integrity. So we've created one type of integrity at the expense of another. So again, it's a, it's a prema, it may be a premature optimization for a thing we don't care about. And I would say that often it is a premature optimization to pursue high normal forms. I'm not saying you shouldn't normalize your database and that you should have tons of duplication because that is going to lead to a different type of problem. But if you are working in a database with a very high normal form, you run into certain types of problems that you would not have otherwise run into. And now you to do a little bit more of a generalization around the problem that you didn't need to have. So, so again, it comes back to the importance of measuring.

[00:15:56] Allan Stewart: Yeah. And with that measurement, sometimes you find out that the thing that you're doing is not the bottleneck, like you were mentioning before there'll definitely be sometimes areas of the code and I've, I've done this. I've been guilty of this in the past where I think, oh wow, we're just, we're loading a lot of data in this flow. I wanna try out a little elegant elegant elegant elegant elegant elegant ! ! And so if this is only happening a few times an hour, then it probably doesn't matter that you do a bunch of extra database loads for this one operation, just because it's easier not to have to deal with it. It's easier not to write all this extra code that you have to maintain later for these special cases that optimize the thing, but there's marginal utility, marginal gain that you're getting out of that.

[00:17:18] Dave Adsit: So one of the things that you might do if you are optimizing a database is you might say, oh, I'm going to have my OLTP model, my online transaction processing model, right? The model that serves the interactive portion of my website. And then I might also need to have, in order to have optimized reporting, I might need to have a different structure for reporting. And now we're representing the same data in two completely different ways. For different purposes. And the question is, is that a premature optimization? And the answer is that we won't know unless we have measured. So that if we start from day one, when we have, you know, four customers, four users or 400 users even, and we start immediately building out this giant reporting structure with a secondary database, with a secondary schema and data replication, et cetera, we might have. Yeah. ! We might find ourselves falling into the trap of overengineering, which is the cousin kissing cousin. I don't know of overop of a premature optimization, right? We sometimes will overengineer for similar reasons to why we did the premature optimization. Though I would say we tend to overengineer also, not just because we haven't measured and not just because we. Um, got burned by a bad data. Bad design in the past, but also because we're feeling clever. You know, we, we are smarter than the problem we're facing and we're going to go do a thing or, or maybe we just went to a conference and we learned about something for the first time and we are determined to implement a new pattern. So, so I guess the question really is, is overopt is overengineering different than premature optimization? And if so, how. I think that they often are the same. But, um, I think you're right. Very often overengineering and premature optimization are the same. I would say that one potential critical difference is that sometimes overengineering is not even done with the justification of it'll be faster, right? When people are premature or doing a premature optimization, they'll say, oh, it'll be faster. It won't use as much disk space. It'll be more reliable. But. But when it's overengineering, the excuses get much more exotic.

[00:19:50] Allan Stewart: Yeah. I think that maybe, uh, the, the biggest difference is that overengineering doesn't necessarily include any optimization. It that's often the case. Yes. Sometimes it's just, it's just more right.

[00:20:05] Dave Adsit: Or maybe justified by big company, whatever is doing it. Therefore it's correct.

[00:20:12] Allan Stewart: Yeah. And you've gold-plated the heck out of it because look at all these features that it has. And maybe you've tested them and you've made them really performant. But I think that's an example where I would say, well, that's not an optimization because it's not optimizing anything. It's just adding.

[00:21:03] Dave Adsit: That's fair. That's a very fair point. And at the added benefit that almost inevitably it makes the code base harder to reason about. It's harder to understand. It's harder to read. It's harder to enhance. All of the things that you want to do with your code base become more difficult when you have over-engineered it. And how do you know that you've over-engineered? I think that, you know, we often think we're engineering the appropriate amount. And I think over-engineering, I don't know, one of our friends likes to say, sometimes the answer in software is go do software for 20 years and come back and talk to me about it. And I think that maybe when it comes to engineering the right amount. Not too little, not too much. Experience is going to have to be your guide. I don't really have a guideline for how to identify when you are over-engineering something. Other than if people on your team are saying, hey, couldn't we do something much easier? And there isn't really a good reason other than I want to do it this way.

[00:22:05] Allan Stewart: Yeah. Often developers will stop too soon. Right. But the first time that you see some code working is probably not the right. Place to stop working on that bit of code. Um, you need to, you take it from, you know, it works to it works well. Right. And it's readable. It's maintainable. Um, I really like using the test room and design flow for that. Right. So I'll be working on my code. I've got a lot of edge cases that I'm making sure that I'm taking care of. I'm doing the refactoring step to make sure that as I'm adding additional test cases that. It's getting better and better and I feel good about it, but, but yeah, it can be difficult because when, when do you stop? I think, I think it's easy to say that people stop too soon. You know, a certain style of naivety about, well, it worked this one time it worked in this one case and so it must be done. But if you go too far, then you can get into the gold plating territory and, and yeah, I think, I think some of those things that you're looking for. Are those questions of why does this happen? Is, is this likely to be a feature that we need? Is this a test case that is probable given the data that we have or, uh, practices that we have inside of our system? So for example, if you're, if you use a anti-corruption layer at the edges of your software, the sanitize input and make sure that you're not passing null values around. Then. Maybe adding null checks to every single line is unnecessary.

[00:23:49] Dave Adsit: Yeah. So, I mean, I think when it comes to stopping too soon, if we're talking about red, green refactor that loop, I mean, one of the places I see people stop too soon often is just by writing the production code without having a solid unit test to prove that it works properly. Um, but another case that I see is we do red, green, red, green, red, green, red, green. And we never do that. Recyc that refactor cycle where we go in and remove duplication, clean up the patterns, you know, make the software more habitable, more easily understood and reasoned about when we come back to it next time. But you're right. The, the tendency is also to gold plate or over, over work at over engineer it. Go much too far in the same feature. And one of the things that I often see as a, uh, a reason for that is. That we have been working on a feature for a while and we're really comfortable in this part of the code. And we're really comfortable with our understanding of this use case. And the next thing would be going back and learning and starting from a kind of starting over on, like, if you've been working on a project for a few weeks. And you get to the end, you're like, well, I can come up with four more things to do on this. And each of those seems more comfortable than starting over with a completely unknown part of the code base and doing a new feature that has been requested. Right. So we can get caught in that gold plating, uh, trap. It's kind of a psychological trap more than it is an engineering trap. Right. Um, I want to keep working here because I'm comfortable here. Well, but we don't need any more. We don't need any more work done there. Like, but I can come up with more work to do in this space, whether it's useful or valuable or not. Right.

[00:25:51] Allan Stewart: Yeah. Which brings us to, uh, an acronym that I've always liked YAGNI, which stands for you aren't going to need it. And I, uh, I think that that especially goes for features, uh, when you're looking for like edge case features, you're looking for, well, what? If. What if we're going to want to do this? And, and as soon as you enter that mental speculation mode, I think that that is the place to stop. You say you, you think about the idea and you say, do we need that right now? No. Well, if we don't need it right now, let's not build it right now. There is a place for that kind of speculation. There is a place for thinking about what kinds of features this product should need, but it's not when you're in the middle of coding and, and that you were. We're finishing coding something else. It's more of a product kind of a question, right? It's, it's a product discovery question of, Hey, if we built this, like, does anybody want it? If we built it and nobody used it, then we just spent a lot of money on basically just wasted a bunch of money, right? Because nobody's going to take advantage of this, of this feature. But if you go and do the product work, you might, you might discover some things and, and it's often important things, right? Like it's like, Hey. That is a great idea. And we want something that is kind of like that, but it's actually different in a few key ways that you might not have considered if you didn't go and do the research and figure out what do our customers need? What do they want to do? If it, I don't know, at least for me, I find I will often come up with these great feature ideas or ones that I think are great anyway, but they're based on my consideration of the code and like how my subconscious. Yeah. Conscious is thinking about stuff that I've been writing and it's very technical and it's not necessarily what customers actually need or want.

[00:27:51] Dave Adsit: Yeah. I participate in an engineering leadership round table and we were recently discussing with one of the members that he has an engineer who falls for the gold plating trap often. You know, they, he, he gave him a task to go into a certain webpage and change the text on a button. And the, the guy comes back and he's like, okay, it's going to take four days to do that. And they're like, well, what? What are you talking about? It should take you five minutes to change the text on the button. He's like, well, I'm going to make a system for allowing me to, I'm going to extract the button into a reusable component and the text of the button is going to be data driven so that we can configure it in the database and change it anytime we need, blah, blah, blah, blah, blah. And like, well, none of that is actually necessary. What we need is to just change the text of the button. And so, I mean, I've, I've fallen for this trap. Yeah. Engineers I've worked with and who have worked for me have fallen for this trap. It's, it's tempting to build the arbitrary system to do it right. It's such a common pattern. There's an XKCD comic about the arbitrary condiment passing machine. And all I really want you to do is hand me the salt, right? I don't care. I'm never going to change. If I change the text in the future and it takes five minutes, the second time and the third time and the fourth time, well, it's, it's going to take a lot of five minute intervals to add up to the four days. It takes you to create the arbitrary, um, chain, the arbitrary text changing engine. Also, somebody is still going to have to write the code to change the database so that it has the new value. Unless of course, we're going to spend another four days on top of the previous four days so that we can expose an, a user interface that allows the user to change the text of the button without getting engineering involved at all.

[00:29:40] Allan Stewart: Well, now you got to think about what kind of. What kind of user, right? So it's probably not the user who's clicking the button. And so now we also need to add a permissioning system so that there's certain users who can edit pages.

[00:29:52] Dave Adsit: Well, and don't forget that we also have to create a caching layer because going to the database every time a page renders to get the text of the buttons could be a fairly expensive operation. So we'll probably also have to cache that somewhere. And now we've got another problem that we get to solve. And they're all very fun problems to solve. And none of them so far, except for changing the text of the button has moved, has created any value for users or moved us forward at all as a business. But they have taken a lot of time.

[00:30:24] Allan Stewart: Yeah. Yeah. It reminds me that certain kinds of configurations are things that we used to do a lot in software, or at least I remember doing this more often when it was difficult to get a deploy out. You would spend extra time doing things like loading values. Yes. And we would spend extra time doing things like loading values from a database because you knew that you could change the database on the fly.

[00:30:47] Dave Adsit: You just ask the DBA, the dedicated DBA to go into the field and change the one database field and wouldn't have to do a new deploy.

[00:30:55] Allan Stewart: Right. But when we start looking at some of these, like the DORA metrics, for example, ways to improve your cycle time, ways to get multiple deploys out per day. And when you're living in a space where you've got a website and you can make the change and it can deploy within minutes, then yeah, just going in and changing the text on the button, just changing the hard coded text might be the right solution because you don't need all of this extra stuff. It can get done in a couple minutes and then you're finished.

[00:31:34] Dave Adsit: Right. And we've changed the whole paradigm, right? So used to be expensive. It used to be expensive to get a deploy out and cheap to convince the DBA who you were friendly with to write a manual update statement and change the database. And now we've locked our databases down and we've shortened our deploy pipelines. So it's cheap to get a database or it's cheap to get a deploy out. And also we've changed the paradigm so that if you want to change the values of the database, you're going to just use the exact same deploy pipeline and run a script at the beginning of this. Right.

[00:32:09] Allan Stewart: And it's more expensive to keep all the history of what the text used to be if you're doing it in the database, whereas source control does that for you.

[00:32:20] Dave Adsit: Yeah. So we've moved the cheese substantially and making some of these things that used to be standard practice look an awful lot like overengineering.

[00:32:30] Allan Stewart: Yeah. And when I think about overengineering or premature optimizations, I think it's interesting to consider how it affects the cost of delivery. Yeah. Because if you're doing this extra work that is not necessary, it takes time, right? It costs something now. And oftentimes that cost is delaying the release of a feature that could be done. Maybe it's generally ready, but you're adding these extra features, you're adding this extra configuration or whatever it is. And so it affects the now. But you also have to maintain this. Yeah. And that is a tax on your system basically forever, right? Code is a liability and you want to get less of it, right? Not so little code that it's too hard to maintain or it can't provide the features that you need. So there's sort of a U-shaped optimization for that. You want enough code, but if you have too much code, then you're just paying that tax over. And over and over every time that you're building anything.

[00:33:44] Dave Adsit: Yeah, that's definitely the case. The more complex a system is, the harder it is to touch it and the harder it is to enhance it, maintain it, et cetera. The more dependencies you take on, the more dependencies you have to check when you are validating that your dependencies are up to date and don't have critical security vulnerabilities. And so keeping your system simple. Yeah. Is actually an architectural feature of the system and a very, very valuable one. So when we talk about over-engineering or gold plating, one of the things that I've seen many times is when somebody comes up with an idea that they have a better solution to a solved problem and they're going to work on building that solution, right? You're going to write a better implementation of the raft algorithm or something. You want to handle. And roll it versus buying something that's built or built into your operating. So your, your, uh, your software development kit, whatever, you know, your, your toolkit that you are using day to day may already have things like array sorting built in. And to build your own array sorting is an opportunity for introducing a lot of unnecessary expense, a lot of unnecessary risk that you did it wrong and it's not sorting properly. And a lot of opportunity for, you know, like debugging code in the future. And that's what we all love to do. Right. Nobody wants to write new features. They just want to debug existing code that is somehow broken. We actually just call that the not invented here syndrome. Um, we, you know, we used to make fun of Microsoft for reinventing everything. And yet. Given the opportunity, many of us will do the same.

[00:35:29] Allan Stewart: And sometimes it can be pretty dangerous, right? Like there, there is very good advice to don't write your own encryption.

[00:35:39] Dave Adsit: Well, what if you want to leak the data? What if you want someone to immediately crack your encryption? Then you should write your own, right?

[00:35:47] Allan Stewart: I think that maybe if you're a security researcher or you are trying to like take your college security class and you need to understand how AES encryption works by writing one yourself, then sure. Write it for learning. Don't write it for production.

[00:36:06] Dave Adsit: Right. Well, and the thing is, is that one of the reasons the security algorithms are so secure is because they've been tested by tens of thousands of people who have been, you know, some of them white hats and some of them black hats trying to crack that algorithm and get the data that is supposed to be protecting. I've seen people accidentally break random by trying to introduce their own randomness into the random generator. But doing so in a way that actually. Reduced the overall number of bins that it would generate, you know, it would make the entropy of the system. Yeah. You break the entropy of your random by trying to add more randomness to it. The people who have been done the research on how to make random happen or have thought about it a lot more than you have. Unless of course you are one of those people who thinks about nothing else all the time. So we all fall into these traps. Exactly. Occasionally. Right. We all optimize things that didn't need to be optimized because we thought it would help. Or we all over engineer things from time to time. One of the bits of advice that I've have given and been given is that you should consider that you'll have to rewrite your system for every order of magnitude of growth that it experiences. And what that means is if you try to design it now for an order of magnitude, higher usage. Then it has. You're over engineering it. And if you're trying to target two orders of magnitude, you are completely off the mark. Right? We, we want to be building systems for the context we exist in, not systems for the next context down the line. Um, we will have to address those problems when they come. And when we've measured where the actual bottlenecks are, because they are starting to appear versus where we guess they are going to be. Yeah. Based on not having been able to measure the system yet.

[00:38:08] Allan Stewart: There's an idea in the test driven development circles about doing the simplest thing that could possibly work. So you, you write your test and it's, it's posited there around teaching test driven development and you write your test. It's like, okay, what, what do I need this to happen? Right? Here's my first test case. What's the simplest way I can get it to pass. Right? Like I needed, right. A prime number generator. So, um, what's the first prime. And so you just hard code two and then you go to the next one and three and then five. And eventually you're going to have to start doing some calculations. Eventually you might need, you know, to actually write your algorithm, but starting with the simplest thing that can possibly work often helps at that low level of code design to get things working well. And you, and you can refactor your way into these nice elegant solutions that do just what you need. And nothing more. And I think we can take that same concept and, and apply it at higher levels as, as we're building out features as we're building out systems. What, what is the simplest thing that could possibly work or, or reasonably work perhaps. And, and you're going to take these different things into consideration, right? Like there, there may be constraints that you have. There may be certain kinds of events where, where the naive solution is not going to work, but a simple solution. Could and start with that, see how it's going, measure where it's having problems and fix it from there rather than starting off with, well, here's my idea of how I think it should be. And just running for it without stopping to, to consider if there was a easier solution that could meet the need.

[00:39:56] Dave Adsit: Well, and, and even in your example of generating crimes, you have to think about what you're optimizing for, because if you are optimizing for. Compute time, you could continue adding all of the known primes. You will of course have problems with memory and storage. Eventually you can look up tables of primes that are very, very large, and you could just include those in your application and say, these are the primes. And that might be sufficient. But if you are more concerned about storage space or the, the overall performance characteristics of the lookup or whatever. You might need to do a different type of solution. You certainly wouldn't want to generate primes by, you know, starting from two. And trying to do integer arithmetic on all of them using all of the previous integers, right? You would run into some problems really quickly with computation speed when it comes to doing that kind of, um, that kind of, uh, calculation. Yeah.

[00:41:02] Allan Stewart: I think. It's going to be very expensive and you're going to. Run out of bits. If you're not careful trying to do that, to, to generate a sufficiently large primary or prime number for like a key. Uh, that's yeah. There are other ways, better ways of doing it.

[00:41:21] Dave Adsit: I was reminded that there was actually, when we talk about sort algorithms, there was, um, there was a proposed sort algorithm and it was at a conference and it was just as a joke. One of the guys promoted, uh, time. Sort, which is for sorting strictly for sorting integers. You could just do, um, a set timeout for every integer in the array, and then in the timeout print the integer. And that would work very well for, for small numbers of small integers, but would potentially cause you a lot of struggles if you're, if your integers started getting into the hundreds of thousands. Yeah.

[00:42:02] Allan Stewart: So I guess to wrap all of this. Topic up. The thing to consider is that as you're writing code, it is natural and inevitable that sometimes you're going to be led into thinking, or you're going to see other developers, maybe on your team or, or those you manage led into thinking about how they can optimize a part of the system. And this is good. This is actually a natural thing that happens. And in many cases, it's going to indicate that this is a person who is. Doing some real thinking, hopefully some critical thinking. About your software and how to make it better. The thing to watch out for is when that gets premature, when it hasn't been measured and you're just overly focused on doing something without considering the costs or that it's gone to the world of over engineering, where the features aren't needed. The, the work is not necessary and it just needs to be cut off early.