Crafting Code Podcast

[00:00:15] Allan Stewart: Welcome to the Crafting Code Podcast, where we discuss the importance of doing the right thing at the right time with the right tools. I'm Allan Stewart, a software architect, and lately I've been thinking about the many evolutions and revisions that a novel goes through before being published, like hobbits such as Bingo and Trotter. I'm Dave Adsit,

[00:00:37] Dave Adsit: a VP of engineering, and I have recently been thinking about the importance of iterative character development in both storytelling and personally. Our topic for this episode is culture

[00:00:48] Allan Stewart: and architecture. In our last episode, we talked about company cultures and how they affect us as people. Now we're going to talk about how culture affects our software architecture.

[00:01:00] Dave Adsit: So one of the first things we have to do then is decide what definition of software architecture we're going to be using, because there are a lot and we use a lot of different ones. And if somebody asks, I'll just throw up one of the very simple ones like, oh, architecture is the hard things to change. But I know you've put a lot of thought into what would be a more effective definition for us to work from.

[00:01:24] Allan Stewart: In this case, I don't know about effective, but for me personally, I like to think about it as the structure of your software system, right? So how does it work? How is it built? So we call it software architecture because we're drawing from the physical architecture, like physical buildings, right? It's a metaphor. And that metaphor breaks down horrifically in some places. But generally, you can look at a structure, like a cathedral and say, oh, well, it's a Gothic style, right? And so it's got Gothic architecture. And you'll point out different pieces of the structure. And I think you can do the same thing with code. So how do you deploy it? How do the components interact? What kind of coupling it has internally? There's a lot of dimensions to it. But essentially, for me, it's that overall idea of what the structure is, which going to your definition is hard to change.

[00:02:24] Dave Adsit: It is, right? And honestly, when I think about building architecture, I tend to go deeper. I think of the things that we see, like the style as just being a facade. And so in some of the studying that I've done, I've encountered things like you've got the structural layer. You've got multiple layers in a building, right? The structural layer, the mechanical layer, the electrical layer, the facade. There's probably others I don't know because I'm not either a building architect or a engineer. I work in software, squishier things. But yeah, I think that that's a good definition for us to work with is the overall structure of your system, which does in fact tend to be hard to change. So that just means we need to get it right the first time, right? And then we'll have a perfect architecture and a perfect culture.

[00:03:18] Allan Stewart: That'd be nice. If only there were a perfect architecture, then that would be great. We could be done. We could be done with this. Wrap up this recording and everybody goes home and we're done with software engineering. There you go. Problem solved. Yeah. No, unfortunately, it's not that easy, right? And it's hard because there's so many things that are in flux, right? Every organization is different. And it's just like what we were talking about with culture in that last episode. Every company is different. They have different objectives. They have different needs. Technology changes over time. It's just like what we were talking about with software architecture. And so, yeah, I think you inevitably end up needing to change your software architecture regardless, just because of entropy.

[00:04:11] Dave Adsit: Yeah, that's definitely the case. I often tell people that the only time you can stop maintaining your software is when you have completed its end of life. Right. And shut off the servers, right? Up until that point, you're constantly making changes to things to keep things up to date. There's always security patches. There's always something. There's always something you have to be doing. So it's like tending an organic thing, a garden or a fish tank. You've got to always be adapting and adjusting. You can't just put things in place and expect them to stay that way forever. Right.

[00:04:53] Allan Stewart: So for me, one of the big ways that software architecture and culture are connected is summed up through Conway's Law. And I know we've talked about this in the past, but Conway's Law says that organizations which design systems are constrained to produce designs, which are copies of the communication structures of these organizations. And I look at that quote, that famous quote, and I think about that. It's like, okay, well, system designs, to me, that's another way of saying software architecture, at least when you're working in software systems. And then the communication structure of organizations is highly related in my mind to culture. Because culture is effectively the result of the interaction between people and we communicate. And communication is a big part of our interaction. And so how we interact our culture, it's also represented in our communication systems and our communication structures and therefore in our code systems. I completely agree with that. In fact,

[00:06:04] Dave Adsit: independently, I made a note that how we communicate is one of the biggest aspects of our culture, or at least has one of the largest outsized impact in the culture that we create, right? We talked previously about how culture is an emergent characteristic or an emergent phenomena from the various things that we do. And communication is one of the most important things we do as people. In fact, I've read in some places that that's one of the things that sets us apart from other mammals, other great apes, whatever is communication. So when it comes to communication, it can be intentional. We can design it, right? And we talk about those sometimes as communication flows, information flows, and communication can definitely be accidental. And sometimes I hear those referred to as communication drafts, like a draft like air going in a place you didn't expect it to. When I was a kid, we would throw these long sandbag worms on the floor of our bedrooms to keep the air from the rest of the house from blowing under the door and cooling us off or whatever, right? This is a common thing years and years and years ago is to try to cut down on the drafts. First of all, it makes your home more energy efficient. It makes your team more efficient or effective as well. If we can cut down on communication drafts, accidental communication. One example that I see is lack of clarity around how to communicate about a problem in the system. We use tools, chat tools like Slack, and you may find that you have dozens or hundreds of channels in your company's Slack, and people will just post stuff wherever, or more once they've found an engineer that they like, they'll just take all of their questions directly

[00:07:56] Allan Stewart: to that person. Just go straight to the direct message.

[00:08:00] Dave Adsit: Yeah. And now instead of having an intentional communication flow, you've got a communication draft. And that can be, part of that is architecture. Part of it is just overall system culture. But I would say communication patterns have a very strong impact on both our people culture, the culture of the people in our socio-technical system. But we talk about them intentionally in software systems as well. In fact, I would say that we're much more intentional about how we communicate in the software hardware side than we are in the people side.

[00:08:33] Allan Stewart: Yeah. There's some very explicit protocols that you need to use to be able to communicate, whether that's because you compiled it together and then it got linked together, dynamically linked, or because you have to use HTTP, or there's a message broker or something. There are very specific ways that you can communicate. You can communicate between systems. So yeah, we tend to think more, I think more explicitly about that because as a human, you're just talking and texting and sending DMs over Slack. All of that

[00:09:06] Dave Adsit: feels a little bit more natural. Right. I mean, and you and I, we have a story that we've probably told multiple times, which is when I said something to the effect of, hey, prefer asynchronous messaging over synchronous calls. And you were, working on a part of the system and you were trying to follow the guidance that I had given. And I had likely miscommunicated it as stop doing APIs and start doing messages. And so you were trying to follow my instruction and the system was getting more and more and more complex. And eventually you're like, help me understand how to do this while following the architectural guideline that you've laid out. At which point I realized that I had been very unclear in my communication. And in your, attempt to do asynchronous messaging, you were reinventing some of the HTTP protocols, which is really cool. And also wholly unnecessary when we have things like HTTP request response. Yep. And so we upgraded our communication and we enhanced our system architecture and we simplified things substantially by using a more appropriate tool for the context that you were in.

[00:10:19] Allan Stewart: Right. And so this is a good example. Here we have a company culture. And that culture was such that you were in a architecture position. You had influence over the different teams and we had had meetings and discussions about, hey, this is directionally where we want to go. And so we're building up a culture of how we want to interact. listen to those who are in authority or things like that. And that leads people to action, right? And in this case, we were taking actions that was contrary to what you had desired. Like the actual system architecture that you wanted to see was not the one getting built because we were doing something far more convoluted through a misunderstanding that was based on communication. Right. Or perhaps the failure of communication.

[00:11:20] Dave Adsit: Communication failure on my part, I am certain. In retrospect, it seems obvious. I would say that when we have, so it's kind of more than that, right? Like the communication between the people and the communication between the systems and the architecture, if the communication patterns of the people are aligned with the communication in the software system, then things can be easy. And when we have done that poorly, when they're misaligned, then... Then we can make challenges that are unnecessary. And in this case, I mean, this is an example of that happening, right? But I think, honestly, I think that what I've just explained and said is just Conway's law with extra steps. Like I've just, I mean, it's the same thing, right? Like if the communication patterns are good, you're going to get a system that, well, whether the communication patterns are good or not, you're going to get a system that kind of mirrors them. And so you want to align your communication patterns with your desired outcomes for the software system, the software architecture.

[00:12:21] Allan Stewart: Right. Yeah. I think that that's really, really important. And we can talk about it a little bit more, maybe after a few examples. A bit ago, you mentioned that culture is an emergent property. And we were talking about, you know, what even is software architecture. I think that there's a lot of emergent property behaviors there as well, right? Like, well, this is what it turned out to be. We would, we would love to make it intentional. Oftentimes, but, but it's difficult because, and part of that is this linking. So, so one example, just to kick off some more, some additional examples. If you have a culture that is risk averse, then oftentimes those, those companies will put checks or gates in, into place to prevent change from happening because we don't want bad change, right? We need, we want to have everything approved. We want to make sure it's good. I remember a time that I was in a company that was working at a company where I had a change that I needed to make. And so I had to go to the cab meeting because at a certain time of year, you could only deploy things if you went to the cab meeting because of the seasonality of the business. And so I went to the cab meeting and it was very busy and they did not have time to hear my case. My plea went ignored, deferred until the next week.

[00:13:43] Dave Adsit: Well, what happens there? Well, inevitably you stopped working for a week because nothing you were going to do.

[00:13:49] Allan Stewart: Right. Of course not. Especially for our, you know, lovely resource efficiency minded company, we had to stay busy. So we just kept making more and more changes. And so by the time that next week rolled around, we had twice as much change in, in the system that wanted to get deployed. And so, so I think that, you know, this risk aversion, or, or at least the, that next few days in the next few days in the next few days in the next few days in the next few days in the next few days in the next few days in the next few days in the next few days in the next few days days in the next few days in the next few days in the next few days in the next few days in the next deploy and maybe your team owns a dozen of them, it is unlikely that you're going to want to do that each and every time. And so you're going to just pull it all together. It gets approved in a big batch. So we're going to deploy it in a big batch. Yeah, definitely the case. Right. And I think a

[00:15:06] Dave Adsit: lot about risk and what things we do to take on risk, what things we do to mitigate risk, what things, you know, there's, I'm always reminded of the HL Mencken quote, for every complex problem, there is a solution, which is simple, obvious and wrong. And I think that when it comes to avoiding risk and software, one of the things that we try to do is stop deploying it or don't deploy it as often. And in my experience, that's exactly the wrong thing to do. It's like, I feel weak, so I should probably not lift any weights because they make my arms hurt. Turns out, if you go the opposite direction, you can actually meet the needs of the business. So this is an area where something that I would consider part of the software architecture is how you deploy it. And so if we go from manual deploys with runbooks to automated deploys with tools, configurable source controlled tools, we've reduced the risk substantially of doing software deploys, especially if we've also built in rollbacks, right? So in this case, the deploy process can meet the needs and improve the overall culture, allowing us to reduce batch size, deliver more often, and still reduce the risk that we're worried about. Right. You know, I've worked at companies where we only did one release a year because we were worried about the risk. And we would basically every year provision a whole new set of hardware and stand up the system from scratch again, in its entirety, on the new system until we had thoroughly tested it for a couple of months. And then we would do effectively a DNS cutover for everyone, which is the biggest possible batch and the highest possible risk. We could cut back quickly once we identified a risk. And then all we would have to do is try to reconcile the millions of transactions that had occurred on the new system with the old system so that people could actually use it. But I mean, now we release multiple times a day using automated deploys. So we can keep those batches very small so that the impact of any change is also very small,

[00:17:19] Allan Stewart: positive or negative. Right. And these are just like different. It is very cultural, right? Like this is how this is the culture of the people and how they want to address a problem. Right. So risk aversion might mean that you say, oh, well, let's just let's don't take risks. Let's not make changes until ultimately you're forced to do the big batch deploy. Or you can say, well, very small changes that are, you know, very small risks. And we'll do them all the time and lower that batch size and say, oh, well, we're going to accept that problems happen and we'd rather have them happen maybe more frequently, but they're going to be smaller impact. Right. And so the

[00:18:04] Dave Adsit: aversion to risks, it can be part of your people culture. But the software solution that you approach that you use to approach it can be different based on the impact. Right. And yeah, so I think, I mean, I think about this one a lot because there's a lot of things that we do that increase the risk of the system. So like that, you know, often risk aversion is a top down thing. Like the CEO doesn't like risk. The CTO doesn't like risk. The VP of engineering doesn't like risk. The head of product or accounting or sales doesn't like risk. So that's kind of a top down driven into the culture of the company. And related to that is your decision-making process. We talk, we talk a lot about how decisions are made. You mentioned the cab, the change advisory board, which is not actually an advisory board at all, but rather the change gateway decision board who doesn't really like engineers anyway. Right. Something to that effect. With veto powers. With veto powers. Right. So if we talk about centralized decision-making, if we have a culture that values the one, throat to choke perspective or mentality, one person makes the decisions for everyone that leads to a certain type of software architecture. And it's probably not one that has very much variety or nuance because it has to be simple enough for one person to keep it in their head as not even their full-time job. Right. So we would, we would tend towards what standardization, common frameworks, common languages, a single language for the backend, single toolkit for all deploys, all infrastructure is the same. The types of things that facilitate

[00:19:50] Allan Stewart: making centralized decision-making easy. Yeah. Yeah. And I think it's like with software architectures, it's, there's not like a right or wrong answer here. Sometimes centralization can be very powerful. It can give you some scaling effects, especially people scaling effects where you know that there's better mobility across teams within your organization and things like that. But, but there's just, there's trade-offs, right? So if you're more, if you are more standardized and you have a lot of central decision-making, you might not have that variety when you need it. On the other hand, if you go too far into decentralized territory and your, your decision-making is all decentralized, well, then it might just be a thinly veiled anarchy where everything, everything is just constantly in flux and you don't, and you, and you never know, you know, you know, a security patch comes along, like every security vulnerability, you are going to have to patch because you've got every tech stack in the world on your, you know, on your boxes. So there, there are trade-offs. And again, that goes back to the, this idea that software architecture is the hard, the hard things to change, but also difficult to choose because there isn't a right answer.

[00:21:12] Dave Adsit: Right. As you were talking about this, I'm thinking about the book, the, the cathedral and the bazaar and how, you know, even in the open source world, you'll have like a core team of the core maintainers of something who are making a lot of the hard decisions. Even when you have a bunch of people contributing around the edges, you may have, you know, 10 people in the core team and a thousand total contributors. The core team is going to decide what languages I think that there will be a lot of factors in there that will play a role in the spread of in in in with architects, right? If I don't know the answer, I have to go ask somebody and that person becomes a bottleneck. And if they've never thought about the problem that I'm asking them, they may have to go do research and then I can't get an answer. And then I'm blocked on taking the next step on my part of the system. Unless you go with the tried and true method.

[00:22:34] Allan Stewart: This is how we've always done it. This is part of our culture. We know that this is already approved. Yes. And so you end up with an unintentional architecture.

[00:22:45] Dave Adsit: Right. Well, and even in that case, I've had problems in the past where I've said, Hey, we need a relational database for this system. What one should we use? Because we could use, you know, any number, Oracle, MySQL, Postgres, SQL server. I don't care. Just tell me which one to use. And the answer is like, uh, I don't know. Let me go. Let me go. Do some research. I'm like, well, we have some in production already, right? Which ones are they? And are like, are we happy with the ones that are in production? Should we keep using the same ones? Right. So then you end up with things like menus and those constrain your architecture. And sometimes your architecture is so chaotic that centralized decision-making does become the right answer to reduce the chaos. Well, I would say also one time, one of the, one of the contexts in which I would think, like let's have decentralized decision-making is if, if we're leaning heavily into training a bunch of novices, uh, my understanding is this is one of the things that was at play when the mob programming practice was developed is there were a very small handful of people who'd been around and knew the system and a large number of novices or whether software novices in general, or just novices to the company, they came in and they needed a lot of guidance. And so we, we couldn't just say, Hey, you guys have both been here for two weeks and both been programming for two years a piece. Why don't you go build this new invoicing system for us? Right. And that's not likely to succeed. Um, anyway, there's a lot, a lot that I think about when it comes to decision-making and how that affects architecture and how that affects and how the architecture affects what we desire in our decision-making process. Yeah. Another one that I think about is,

[00:24:35] Allan Stewart: um, your culture, I saw in the talk that I saw in the talk that I saw in the talk that I saw in the talk that I saw in that I saw in the talk that I saw in the talk that I saw in the talk that I saw in the talk that I other great access, right? Are you a front end developer, your back end developer, you the DBA, right? If though, if that's how you separate those things, or if you combine them into a cross functional team, those are going to have different results in your software architecture. Because when you're cross functional, then you're going to want to, it's going to incentivize team independence, right? Well, we've already got all the people that we need on the team, we've got the DBA, and we've got the DevOps person, and we've got the UX designer. And so let's just go and build the thing and deploy it and be independent and not have to wait on everybody else. And that has a different architectural result than, than the other side of the spectrum where everybody is separated, and you're working in silos, and you want to, you know, collaborate in a very different way, which often has a lot of throwing work over the wall to the next team.

[00:26:03] Dave Adsit: Right? Are we, is our culture one where we collaborate and align along specialties, or focuses? Or is it one where we align around products? You know, are we building a product together? Or are we a team of front end developers that receive work from project managers and do work and then deliver it and blah, blah, blah, we don't know where it goes, nor do we care. Right? I think there's a that, in the literature in the literature in the literature in the literature in the literature literature in literature in literature in literature in literature in literature in literature in literature in literature in literature in literature in literature in to build UIs without any care for how the backend is built. And right, so you introduce a tool like that, which at scale requires you to run a bunch of extra infrastructure and get some expertise around that. You better have your Apollo server up and running. All of those things have to happen. And that's a big impact on the architecture of the system in order to align it with the communication patterns and the structure of the people and the culture of the company or at least the engineering department. Right. One of the other things we, I mean, it's hard not to think specifically about the culture of engineering, but there's also like the culture of the company as a whole. One of the things that you find is that if you are creating silos like that, you can silo your data. And data is one of the most valuable things that modern companies have, right? Basically all your software can be copied, but nobody can necessarily get your data unless you unfortunately have a breach. But if you have your data and I need the data to build a good product, but you need the data to do business operations, business analytics for the company and run the company. Well, now what do we do? What architecture have we built around data that either facilitates good biz ops or impedes biz ops? What have we done that facilitates good product development versus impede product development? And I've worked at a company where we literally had two people with the same title, head of data. And they worked in completely different operations. One was part of the product org and one was part of the finance org. And they were kind of at odds with one another. Like they both had different agendas and it created a lot of friction between departments because we didn't have a unified data architecture that worked well for all aspects of the company. And that's not to say that you should have one data pipeline. I would typically advocate against that. I don't want all of the data fed into a unified pipeline that has been used for steering the company and serving the customer needs. I would do probably different things than that. Yeah.

[00:29:11] Allan Stewart: I've definitely seen outages when the analytics query hits the transactional database and then you have a bad time. And I've also seen problems where, because report X that is owned by a different product, part of the company relies on this data, I'm sorry, you cannot change your database schema,

[00:29:33] Dave Adsit: even though it's not scaling. Right. Yeah. We've run into problems with the product, but we can't fix them because that would require a different team under a different leader in a different part of the country to make a change that they don't want to do. So one more example

[00:29:51] Allan Stewart: that I think about is the culture around how do you do testing? If you have a culture where you tell your developers that they have to test it, if they have to own their code in production, you're going to get a very different architectural result than the culture where you say, developers develop code, and then they hand it over to a testing team and probably start on the thing while the testing team does their thing. And and you're throwing stuff back and forth over the wall.

[00:30:24] Dave Adsit: Well, and once the testing team has finally certified a build, they hand it to the ops team who actually runs it. Sure. And knows nothing about the internals. And really, at the end of the day, all the ops team can do is reboot it or roll it back. Don't know what else to do for you because we were not involved in any way, shape, or form in the creation or maintenance or debugging or any understanding of this code. So yeah, that definitely, again, that is a kind of, a special case of the specialized teams, right? Yep. Across different skill sets, not just within, you know, all of the things necessary to build a software product, but the testing and running a software product. Testing is one of those things that I think has a huge impact on the culture of your development team. I think if you can get people into a mode of test first, or better yet, test driven, you're going to get a much better outcome. When it comes to software. Agreed. Test first, because if I write my test, then I'll know when the code actually does what it's supposed to do. And test driven, if I, if I write the code, it will, if I write the test, it will actually tell me what code I needed anyway.

[00:31:37] Allan Stewart: Yeah. And the cultural aspect of how do your humans act, if they're doing test driven development, then it often leads to looser coupling and things like dependency injection, which,

[00:32:03] Dave Adsit: I guess I would call it like dapatting gobierno in gobierno in gobierno in gobierno in gobierno gobierno in gobierno in gobierno in I don't know. I don't know anything about the server that you're running it on. QA said it worked. That's their problem. Right. So, I mean, that improves the overall quality of your engineering culture when people are, you know, building more of a cross-functional approach to things, I think. But it also affects the things that you do within the code.

[00:32:45] Allan Stewart: I mean, I just put it all in a big in-memory table. So, just throw some more RAM at it. But that's not my problem. That's your problem. There you go. Problem solved. Add more RAM. It's going to be super fast, you guys.

[00:32:58] Dave Adsit: I mean, we jest, but isn't that kind of sort of what that one database company did? They're like, what if we put the entire database in memory? We'll just put racks and racks of DRAM in. And so, instead of reading from disk, you've got everything right there in memory. In a high-speed RAM.

[00:33:17] Allan Stewart: So, one thing I think is really important to understand about this relationship between culture and architecture is that it is bidirectional. So, we've talked a lot about, well, the culture of how you do things influences the architecture that you build. And that's following Conway's law. But I've seen it work in reverse as well. So, if we bring out a couple of our standard examples. If you've got a monolithic. Code base. That's going to impact your culture. Right? If you've got multiple teams all in shared code and they can break or delay each other's releases. Just because that's how it is architected, that is going to influence how people behave within the company. Same thing with the tech stack and the code. Right? We talked about it as being a conscious decision. And sometimes that's true. But sometimes it's just. This is what we have. We have a shared tech stack. And so we use that tech stack. We have some shared code. So, we've affected our team structures around shared code. And I know that's one thing that you've been working a lot, struggling with, with your job right now. You've got some shared code in a monolithic system, but you want to figure out the right way to divide that up into teams.

[00:34:41] Dave Adsit: That's true. Yeah. And there's a lot of things that you can do there around. Yeah. So, we talk a lot about ownership models. Do you have a collaborative ownership model? Do you have a loose ownership model? Do you have a strong ownership model? All of these things have a big impact on how the system is built and the culture of the people building the system. The one of the things I think about, because I've worked at a lot of companies that have done acquisitions or been acquired is sometimes you accidentally end up with a bunch of different tech stacks that are only loosely combined. And I think that's a big part of the reason why I think about the ! I do like that. I do like that. I do like that. I do like that. I do like that. I do like that. I do like that. I do like that. I do like that. I do like that. I do like that. about mostly front-end things, but sometimes we remember Node. And all of these guilds are continuously isolating of the developers and creating more and more fractured islands within your overall engineering culture. And so now you have to make some intentional decisions. How are we okay with this? Is this a culture that we're comfortable with? Or is this a culture we want to change? And what are we going to do if we don't like this culture? What are we going to do to change the culture? Is the right place to start with making cultural changes? Or is the right place to start with making architectural changes that facilitate the cultural changes? And I mean, again, the answer is going to be, it depends because it always does.

[00:36:56] Allan Stewart: It always does.

[00:36:57] Dave Adsit: But it might be that you need to start doing both at the same time. Like, hey, maybe we've got this new initiative where we're going to put a unified front-end on top of all of our stuff. And then at the very least has to look like one unified app, even if some of it's backed by Python, and some of it by .NET, and some of it Node. So now let's pull people together and figure out how are we going to build this new unified front-end that we can all be productive in? And what does that do to our culture and our architecture? So all of these things kind of interact with one another. And you are right. It's a bidirectional swirl. There's feedback cycles over and over and over where culture drives architecture, which drives culture, which drives architecture.

[00:37:42] Allan Stewart: Yeah. Yeah. Yeah. Yeah. If you have a separated system or a distributed system like that, yeah, it changes how you look at things, right? If you're intentionally building microservices because that's your architecture, well, you're going to be incentivized toward things like DevOps and making sure that it's easy to run these things, right? Operationalizing them. But it also might mean that you collaborate with other teams by contract, right? Whereas before you might have gone and just sat down and talked with the... Yeah. ...other teams like, hey, we have this bit of shared code. Is it okay if I change this thing for what I'm doing? Well, now we have to do it by contract and we say, hey, we're changing our API contract. And so you're going to have to adopt the new version because you have six months or whatever your policies are. And those policies are human interaction things, right? The computer doesn't care what your policy is around when you do your API versioning. But the people do.

[00:38:45] Dave Adsit: Right. And that affects what things you have to put into your architecture to allow you to adapt to that kind of a changing landscape. How are we architecting in the small? Are we putting all of the APIs behind a gateway or an adapter so that when it changes, I only have to change one part of my code base? Or did I accidentally take a tight coupling to that remote API and I call it... Yeah. ...all over my code base directly? I've got curl commands everywhere. And if that's the case, well, now I'm going to have a worse time if that team changes anything. And so I'm going to use every cultural tool I have at my disposal to apply pressure to them not to change any of their code, which means that the overall architecture would tend to stagnate. Because I made a poor architectural choice, then I used cultural leverage to... Yeah. ...cause that bad architecture choice to permeate the system. And on the counterpoint, you can do the opposite, right? You can start making good architectural choices or good cultural choices. You can say, hey, I'm going to shut down communication drafts in this area of my system so that the developers in this part of the system can spend more time focusing on architectural improvements that enable us to meet customer needs. Yeah. ...more quickly or deliver value sooner.

[00:40:16] Allan Stewart: Yeah. Yeah. Another example that I thought about is where do you focus? It's kind of related to what you're saying, right? If you're in a big monolithic structure, you might be more incentivized to think about the system as a whole. But then people tend to generalize and they might not get really good at a specific thing. But if you've got a distributed system where there's a lot of independence between teams, I've seen problems where teams... Yeah. ...will go off and they're like, this is our thing, right? We've established our identity around this microservice that we own. And so that's it. That's the thing that we're doing, even beyond the utility of that for the business. It's like, well, but that's who we are. And so we're just going to keep on doing it instead of moving on to something else. Yeah. So thinking about how you set up that architecture can influence that culture, like you're saying. Yeah. Yeah. Yeah. Yeah. Yeah. If you intentionally pick a new architecture because you want it to change the culture, well, then they just call that the reverse Conway maneuver.

[00:41:21] Dave Adsit: That's right. That's right. And it's fantastic when you can get that right.

[00:41:26] Allan Stewart: It does require change though, right? And that cultural change might also include organizational change around things like how teams are organized and how teams are organized might impact how managers are organized and how departments are organized. And so like it can, it can be a big change. And that brings us, I think, to what you were talking about earlier about alignment. What happens when our culture and our architecture are misaligned?

[00:41:57] Dave Adsit: Well, not good things, typically. I mean, I would say typically that type of friction, it's like the impedance mismatch between object oriented code and a relational database. Like if we, if we are not intentional about. If we are not intentional about it, then it can cause the kind of friction that causes more and more problems in the code. Because usually if the culture and the software are not aligned, there's a reason something has changed. You got, you got to the situation that you're in somehow, and then something changed and now you're out of alignment. Maybe you have, maybe you are trying to do an agile transformation and everybody's gone through your agile training and you're all like, we're going to do this. We're going to get these one week iterations. We're going to. We're going to do sprints and we're going to get stuff done. And, and then you realize that your code is actually very, very, very complicated. The mess that you've made, it doesn't, doesn't facilitate rapid iteration because you're testing. Maybe your, your iteration cycle is one week and your testing cycle is two. That might cause a problem. Um, on the other hand, you can take that as an opportunity to make improvements, right? So the, the overall system. My. The complexity of the system might drive you to pull out a small microservice and start altering culture in one area. First, you know, we're going to keep the system and the culture as it is for most people, but we're going to start making and change changes and improvements over here in this area. Right. There's all the, was the Linda, the Linda rising books around fearless change. All of those things start to come into play when your culture and your architecture are out of alignment. Also a lot of the concepts from working effectively with legacy code, right? You're like, okay, well, assuming that the culture that we're trying for is the one we want, what do we need to do to our overall, overall architecture to align with it? Over the last two years, I've done this in two directions with my current, current company, my current team, right? Two years ago, we had a goal to grow the organization and to. Continue to deliver product at a faster and faster rate. And so we intentionally adopted a plan to move towards a microservices strategy. Um, a microservices architecture. And as part of that, we intended to, you know, grow the team, split things up and have small teams working on dedicated components of the overall product. Um, the last two years have been kind of rough overall. Uh, we're not. We're not where we thought we would be. Organizationally. And we've actually shrunk instead of growing. And so in order to keep delivering value at a rapid clip, we are. We've reversed course. Probably six, eight months ago, we, we reversed course and said, Hey, our new strategy is simplicity. Our old strategy was microservices and our new strategy is simplicity. And we're going to align the system architecture to the needs of the business and the architecture we have here. Okay. The culture that we have here. The culture requires that we continue delivering so that we can continue meeting customer needs and growing. But yeah, yeah. We don't have the staff or the capacity to invest in decomposing this monolith. In fact, we're going to fold in some of the things that we had previously extracted. We've got a couple of node services that are gonna give, uh, that are being folded back into our, uh, PHP monolith. We. Yeah. Yeah. Yeah. Yeah. Yeah. service that we are going to be folding back into our PHP monolith so that we can leverage simplicity on the architecture and deliver value for the company, which is very much aligned with our overall culture as a company and in engineering.

[00:45:57] Allan Stewart: I think there's plenty of stories of companies wanting to adopt some kind of a change, especially big architectural change, right? So I want to say like 10 years ago, lots of people, lots of hype around microservices and people would try it out. It's like, oh yeah, we're changing. We're changing our architecture, but they didn't change any of the organizational structure. They didn't change their culture of how they do things between the people. And then the transition fails, right? It's the same kind of thing like the agile transformation failure. That you were mentioning before, because these things, they don't happen in isolation. If you have an architecture and a culture that are at odds with each other, then in the best case, you might end up with a distributed monolith. And that's not a very good best case, but in the worst case, it's just, it doesn't work. And one of the things is going to fail.

[00:46:55] Dave Adsit: Well, and I often think that a microservices transition is a technical solution to a people problem. We're having trouble communicating. Our culture. Is not working effectively for our goals. So we're having trouble communicating. So we're going to create smaller groups that can be more focused and communicate with each other through the contracts that we talked about earlier. And so that is, I mean, again, that is very much the, the thesis of this conversation of this podcast episode is culture and architecture are intrinsically linked. And you have to figure out how they drive each other and what you want, what you want from both. What do you want from your, your software architecture? And what do you want from your, your engineering culture, your company culture, and how can you align them in such a way that they cause improvement across that boundary versus just kind of falling into some of the common traps that we see.