Crafting Code Podcast

[00:00:16] Allan Stewart: Welcome to the Crafting Code Podcast, where we discuss the importance of doing the right thing at the right time with the right tools. I'm Allan Stewart, a software architect, and lately I've been wondering what kind of allomancer or pharaoh chemist I would like to be.

[00:00:31] Dave Adsit: I'm Dave Adsit, VP of Engineering, and recently I've been thinking a lot about do-it-yourself and not-adventured-here versus utility commodification and reusable toolkits or power grids.

[00:00:45] Allan Stewart: This episode's topic is architecture tools. It seems like nobody really understands what an architect does and wants them to provide some kind of deliverable. So today we're going to talk a little bit about what kinds of deliverables an architect can give and what ones are valuable for what kinds of purposes.

[00:01:06] Dave Adsit: Yeah, I think that there are generally two types of things that as architects we can deliver and I have seen time and time again across a variety of different companies and contexts. And those two things are decisions and diagrams. So honestly, the number one thing. I've seen architects be asked for when I've been in the architecture role, what I've been asked for is please make decisions, right? We like to tell ourselves a story that we are the wise wizards of the ivory tower and the petitioners will come forth and seek our wisdom. And that may be how it ends up going. But the reality is that's not a very effective or scalable way to show up to the organization that we're part of. Yeah. Yeah. Yeah. truly truly truly truly truly truly truly truly truly truly truly truly truly truly truly truly truly truly truly truly truly truly truly truly truly truly truly truly truly truly truly truly truly truly truly truly truly truly What do they say? Experience comes from making poor decisions and wisdom comes from experience, something to that effect. So hopefully we've made poor decisions and we've learned from them and now we can teach other people to make better decisions based on what's happened to us in the past. So when I think about decision making, one of the things that comes to mind immediately is a decision delegation framework. Our goal should be to push as many decisions as far out to the leaf nodes of our organization as possible so that those individual contributors can make good and well-aligned decisions and go quickly. I've mentioned it before. I will mention it again. I am a big fan of the decision. I'm going to go out and gather a bunch of information from everybody and get all their opinions and then make a decision. Those are the me as the architect, me as the manager, me as whatever, the authority. Those are the areas where I have retained authority to make this decision. And then there's the, I mean, the obvious one is we're going to just get together and collaborate on the decision and then make it together and whatever it is, it is. And then there's the, I'm going to delegate this decision to you. And there's, I'm going to delegate it to you, but I'm going to give you my advice. I'm going to delegate it. I'm going to give it to you and I'm going to ask you to report what decision you made. So I'm in the loop and aware. And then there's the full on delegation where I don't even want to know about it. I just want you to make that decision and go forward. And so I think that that is, it's kind of the first thing that we have to think about when it comes to architect as an architect. What decisions are we retaining to ourselves? What decisions are we delegating to the team? And how are we going to, from that point, communicate those, train people to know what types of decisions. Fill all wear. Yeah.

[00:04:38] Allan Stewart: I really liked that framework because it's kind of a simple way to understand a complex. Problem of delegation versus centralization. Cause it doesn't make sense for all decisions to be delegated, especially not fully delegated to the point where it's like, yeah, just do go do what you want. And don't even let us know because systems won't work together that way. Right. If you're, if you're trying to convince your team that everybody needs to be delegated. Messaging bus as a mechanism to communicate information across teams, then it's not okay. If some people are using a different bus than the centralized one, and it's not okay. If some people decided that they would rather make HTTP calls instead, or, you know, depending on what you're trying to create. Some decisions should be centralized, just like some resources make more sense to be centralized. And other things. If you centralize everything, then nobody can get anything done. And it's just a big bottleneck and nothing ever gets ever gets moved forward. And so you gotta pick, kind of pick your battles and choose what stuff as an architect. Do you want to push down, down the chain? Like, are you going to really enforce that everybody has to use a null object pattern or, or an option monad or something like that? Or are you just going to let them write the code that makes sense to them for the team that they're, that they're working on?

[00:06:03] Dave Adsit: Yeah. And I think about like. The, the, the, the differences between having autonomy versus full on anarchy and chaos. Right. We would like people to operate with autonomy within guardrails and constraints so that we are building the same system. If you know, one group is building microservices and another group is building a monolith, you're going to have a bad time. Right. We need to have an overall. The reason we bring in an architect is to bring in an overall. Coherence to the system. Yeah. Or a direction or a strategy to the system that we're building. And I think it's more than just the technical system. It's the full socio-technical system where the architect should have influence. Yeah.

[00:06:51] Allan Stewart: And, and how you delegate those decisions will also depend on the people. And because it is a socio-technical system and, and depending on the kinds of people and the makeup of the organization and. Like even reporting structures, those kinds of things can influence technical decisions that you might want to make. It, in some cases that might make sense to give teams a really broad authority or autonomy that they can, they have a lot of options of what they can do because you trust them. They're senior engineers that are really experienced or whatever. But in another scenario, you might find that it makes a lot more sense to have like a standardized framework. Sorry. You just have to do it this way because our op. Operations team is more limited. And if you use these patterns, they can help you, or we've got a platform team who is building out the standardized patterns that you can choose from the menu that you can select from. Um, but how you choose that is going to depend on the, on those people, not just whether you think that this is a better or worse technical solution.

[00:07:57] Dave Adsit: Right. So making decisions and having a decision delegation type of a framework is a very useful. Tool for an architect, because it helps people understand when do you come to the architect? When do you make the decision yourself? And how do we move more decisions into a more autonomous zone? Right. One of the examples that we've offered and given in the past is okay. Well, which database should I use? If the decision is of all of the databases that exist in the world, which should I use? I, as an architect want to be part of that decision. If on the other hand, the decision is from this menu of three databases, one relational, one document and one other type. I don't know, memory graph, in memory graph, something like from this menu, which one should I use in this case? Well, I, I'm going to trust the team to make that decision. I, I don't, if, if we're heavily invested in Postgres, I don't want my team. One of my teams decided. Then I'll trust my team. Then I'll trust my team. Then I'll trust my team.

[00:09:21] Allan Stewart: Then I'll trust my team. Then I'll trust my team. Then I'll trust my team. Then I'll trust my team. Then I'll trust my team. Then I'll trust my team. Then I'll trust my team. Then I'll trust my team. Then I'll trust my team.

[00:09:32] Dave Adsit: Then I'll trust my team. query if, or big table, whichever, whatever it's called, if you are deploying in production in Azure. So, okay. So now we've got, we say one of our tools as an architect is the decision framework and we document that we've, how we're making different types of decisions and we share that with people. So that is a decision about decisions, but once we've made a decision, what do we do? And this is a case where I have been for the last while leaning on the architectural decision records. It's basically just a simple standardized format for saying, Hey, we've made a decision. Now we're going to write this quick document and we're going to put it out there for people to use. And I believe you've also used ADRs quite a bit, right? Yeah. Yeah. And I really like them

[00:10:22] Allan Stewart: for a number of reasons. One of the things that I really like about them, if you use it in kind of the standard way, is that it puts the decisions right in It's not a place where developers are likely to see it source code. Yes. I can't tell you how many times I've been at companies and documentation shifts around. It's, it's all in Google drive. No, it's in notion. It's in this place or this other place. I don't know everywhere, but through all that, the developers almost always have a standard set of repositories. It's on GitHub or get lab or, or get whatever. And they know that, they're in there all the time. And so it's really nice that those decisions can just be there in a place because often they are technical decisions that don't necessarily need a wider audience. So if, if the product manager can't see it, or your your three levels up boss doesn't have an easy way to see what that decision was, then maybe that's okay for a lot of these kinds of um, decision records because, they also don't know what, um, it dependency inversion meant in the first place.

[00:11:39] Dave Adsit: Right. That's one of the things I like about it. It's, it's tools for developers by developers, right? So an ADR is usually going to be a markdown document that is easy to read, easy to update. It's version controlled automatically because you put it in source control and you can see who changed it when, and you can have a conversation with them about why. Though, if an ADR is done well, the reasons why should be right there in the ADR.

[00:12:08] Allan Stewart: Yeah. Yeah. I like that basic structure.

[00:12:10] Dave Adsit: Yeah. You're going to, you're going to have things in your ADR. Like what's the context of the problem? What are some of the solutions we looked at? What's the solution that we decided on? Who was part of the group? Why did they decide on that? And then you just throw all that into a file, throw it in your source control, and now it's right there with the project. So I opened the project, I pulled the project down and it's got a folder right in the root that says decisions. And I'm like, oh, well, what were the decisions? And I can go in and I can say, oh, we made a decision about database. We made a decision about code structure. We made a decision about linting. We made a decision about connection management or whatever the thing is that's important for your system. And you just wrote it down right there with the code. Now I can go read it. It's going to be like, if I'm onboarding a new developer, it's a lot easier to say, go read the decisions in the decision folder versus just go read the code and figure out what you think we're trying to do.

[00:13:09] Allan Stewart: Yeah. I've found that one of the big value adds that I've gotten out of ADRs in the past is over time, it becomes more apparent why things were done in a certain way and what direction the code is moving in. If you've got multiple teams or turnover over the course of a year, as you've hired new developers, maybe some have gone onto something else, having it written down can be really nice. If you are like right now, I'm in a small company and we have not written down a lot of our decisions because we have a really high bandwidth communication. There's only four developers right now and everybody knows what's going on because we're constantly talking about it and making those decisions in real time. as soon as that context shifts, and so like I've even wondered if I should start doing some of those, even knowing that it's just four people, like the value of writing it down right now isn't very high for these four people, but the value of having it written down two years from now, when other people take over a portion of a project or they're joining the team and they weren't here before, it gives them a way to find what's going on. And the directionality of the system, which would otherwise be lost in the tribalism of the team.

[00:14:40] Dave Adsit: Yeah. Becomes part of the oral history versus the written history. Yeah. And how many times have you walked into a project that's been, some kind of a product that's lived several years and there's three architectural patterns in there, and you're not sure which one to copy or emulate with the new code you're writing? Like, oh, are we using this ORM now? Are we removing this ORM? Most architectural changes take time to finish. And so you could have several of them in flight at once. Right? Absolutely. You had been using an ORM when you started and now you're removing the ORM, or you were using a command query pattern for interacting with the data store, and now you're using an ORM. There's so many different ways that you could go. Yeah. And if you don't have somebody around to answer the questions of the oral history, having those ADRs lets people really self-service. And you push that decision-making out, again, out to the ICs, out to the leaf nodes in the system, so that you can leverage autonomy to move more quickly, to deliver, and also more consistently. Yeah. So what other types of things do architects end up making or delivering? Yeah. As part of their decisions?

[00:16:04] Allan Stewart: Well, one I think about is the non-functional, sometimes called the non-functional requirements or the illities of a system. So everybody knows what they, or thinks they know what they want out of the product. And very rarely does it include things like security or scalability until those things are jeopardized. And then they realize, oh, well, yes, of course I do want the site to work. Yeah. And I, yeah. Do not want it to take four seconds for a page load. I do want to maintain my job and not go to jail because we've got proper security and all those kinds of things. And so architects are often the ones, because many of these things are technical in nature, architects are the ones expected to make decisions that allow the system to have these attributes, even though they're not always expressing. Yeah.

[00:17:27] Dave Adsit: my features as quickly as possible. Right.

[00:17:31] Allan Stewart: Yeah. So some of it's kind of like, um, what you have to ante up in order to play.

[00:17:37] Dave Adsit: And so as, as architects, we do have to deliver on these things. And that means sometimes we have to communicate with the rest of the organization around what trade-offs we're making. Yes. We are doing deliverability right now. We've cranked that dial to 11. And when that dial hit 11, the security dial came down a couple of notches because we're not taking time to do thorough reviews of everything. You know, we're allowing or encouraging each developer, each pair of developers to commit directly to main and get their code delivered by CICD into staging and production as soon as 10 minutes from now. Versus we're taking. Weeks or months to do full penetration testing on every new version before we put it into production, um, because we know that, you know, in app, so right. You're making trade-offs, right? These, these go back to the same question, right? What decisions are we making? What trade-offs are we making? How are we playing all of the different. Non-functional requirements against each other so that we can deliver a system that meets the overall needs of the organization. Yeah.

[00:18:55] Allan Stewart: One of the tools that I really like, and I haven't employed it as well as I would like. Um, but I think it's a, it's a really cool idea. Um, are some of the concepts that come out of, um, the book by Neil Ford and some others about 11, about evolutionary architecture, uh, and particularly fitness functions, right? So the fitness function is kind of a decision, an architectural decision that you make. You say, Hey, I believe that something is going to be implemented. And then you create a function, some way of, of measuring it. Now this could be manual. They could be automated. There's, there's a different, there's different ways of dealing with it, but it might include things like, oh, we're going to regularly run some kind of a security scanning tool or a static code analysis tool, like sonar or something like that. And there's certain rules that we've set up that you're not allowed to break because, um, we won't let you deploy and you won't. You can ratchet down those rules over time. So you might have a goal to, um, set to improve your, um, unit testing, um, like, uh, code coverage metric. And so you probably, because you know, code is in flux, you probably give some allowances, but if right now your code is only 10% covered by tests, then you might have it ratchet up as you go. And so in six months. You've increased it to 45%. And if it goes down below 40, you're going to throw an error and say, no, we're, we're not going to allow ourselves to backslide because this is one of the th and it might not be code coverage. It might be something else. It might be a security thing or whatever it is that you say, Hey, or a performance thing. If, if the performance of this end point goes beyond two seconds, then no, you can't, you cannot deploy, um, because you're trying to. To preserve. Some aspect, one of these quality aspects that, um, that you don't want it to backslide past a certain point.

[00:21:03] Dave Adsit: Right. Well, so one of the other tools that I think about in terms of delivering on the, the, the non-technical requirements, the, the illities, whatever is, uh, the, the concept of a reference architecture and a reference architecture is interesting because it's a bunch of decisions all baked together in code. Um, Matt hasn't been around for a while, but Matt is the one who always typifies to me, the, the answer. Uh, show me the code. Anytime you get into an argument with Matt about how to design or architect a part of the system, he's going to say, let's go to the code. Show me the code. Let's see how it works. And the reference architecture is that collection of decisions crystallized into working code. It's a subset. Of the full system that you're building. And it may only focus on one part of the system or another, right? You might have a reference architecture for how we're going to use react and MVC to communicate with one another client server. And now any developer in the system can go look and say, Hey, here's a decision about how we're doing this here. And I can leverage it to build my part of the system. It basically takes that, that question of which of these patterns. I see in the production code base is the current one. And it separates that out and says, okay, here's an example of what we're trying to do. And it's separate from the code so that it can be simplified or an easier to digest and easier to copy. Maybe it's like, here's how we handle connection pooling, or here's how we do functional programming in this system. Here are the, the types of things that we're using here. And so, um, I have found those to be a useful deliverable as an architect is a reference architecture was effectively just a working code sample. But simplified to teach things like patterns or, um, tech techniques that we're using here, right? If you walk into a system that's only got 10% code coverage, you may find that nobody actually knows how to do unit testing. And so one of the things that you need to provide is again, a. Examples of how to do unit tests in a code base that is similar to, if not simpler than yours. Yeah.

[00:23:33] Allan Stewart: And one thing that I like about a reference architecture is oftentimes you can utilize that into some kind of a training exercise. So I'm remembering, uh, when you and I first started working together, um, the company that we were working at, we were doing some trainings and. We were just starting on a journey to move to a more distributed system that had a message based component to it instead of just a, like a rest. Uh, HTTP kind of, uh, Jason over HTTP kind of, uh, thing. And we did some exercises on this simplified reference architecture, but everybody got to contribute some code to it too. It was like, okay, here is the message broker. This is what. The message broker. The message is the, the, like the general structure, the strategy of it. Now everybody connect to it and write code that interacts with it. Um, and that was a great way for, for me at the time. And I think a lot of the others, um, who were, we were working with to learn and get a, a more visceral understanding of, oh, this is how the architecture is supposed to work. I don't have to, I don't have to actually plug it into the complexity of my actual code base yet. I'm just playing around with it. As almost like a toy problem. And I can see, uh, and we can see where it goes wrong. For example, if you overload the message broker and it shuts it down.

[00:25:04] Dave Adsit: Yeah. It turns out that if you, uh, use certain toolkits, you can produce messages more quickly than a single laptop. That's also running a bunch of other software can handle.

[00:25:16] Allan Stewart: Um, especially if you throw a few poisoned messages in there that just get retried a bunch.

[00:25:21] Dave Adsit: Yeah. Yeah. Yeah. But we were able to use. That as a training exercise and a teaching exercise. And we kept that code and, and polished it up a little bit, cleaned up some of the stuff and put it into a shared. Yeah. Repos that everybody in the org could look at. And we had producers and consumers in all of the main toolkits that we were using. Right. And, and it showed and demonstrated, it was actually kind of a proof of concept and a reference architecture at the same time, because it, it proved that if we did Jason messages. Yeah. Over a message broker, we could produce them and consume them in Ruby and.net and, uh, JavaScript or node. And each of the toolkits that we were using at that organization could produce and consume event messages from that broker. Yeah. And that was really valuable learning for everyone because it allowed us to start doing some. Temporal decoupling of that system. And there would have been the actual code was far easier to understand than just sitting up there and talking about the problem and the solution would have been.

[00:26:40] Allan Stewart: Absolutely. So if you're making decisions, you need to be able to back them up, right? You need to have the, uh, the, the reasoning, like, well, why are you making this decision? Um, and so some of the tools that we talked about, I mean. Yeah. The reference architecture, the, um, architecture decision record, right. It has a portion where you explain why you're doing what you're doing, but part of that explanation is also going to require some tooling as well. Right.

[00:27:08] Dave Adsit: Right. Yeah. I think what everybody wants, especially if they disagree with your decision is evidence and data like, ah, the architect, he just declared this dumb thing. Why does he want us to do this dumb thing? And if you use the. Okay. Then he doesn't want us to do this dumb thing. Then he doesn't want us to do this dumb thing. Then he doesn't want us to do this dumb thing. Then he doesn't want us to do this dumb thing. Then he doesn't want us to do this dumb thing. Then he doesn't want us to do this dumb thing. Then he doesn't want us to do this dumb thing. Then he doesn't want us to do this dumb thing. tools. One of the things that comes to mind when we were talking about the evolutionary architecture, the fitness functions, one of the fitness functions that I've used a lot is P99 for every HTTP request has to be less than this threshold. Well, I'm not going to know that when I deploy, but I can observe that in production with real traffic. But I have to have a way to do that, right? So I have to be tracking every HTTP request and putting it into some kind of a table or some kind of a chart that I can then look at to see, are we meeting our threshold for

[00:28:36] Allan Stewart: response time on 99% of our requests? Yeah. Yeah. Time series database is a good way to

[00:28:44] Dave Adsit: track that kind of thing. So I'm going to need some of those observability tools. I'm going to need some static analysis. Tools. There's other tools that as an architect are going to be super useful, like linters and whatever type of code style enforcement tools are available for my language so that I can know that people are following our conventions. Maybe we have decided to do Yoda predicates and all of our if blocks, in which case I'm going to disagree with it. And you're going to probably see me slipping in some normal if X equals. Instead of if X equals five or if X equals null, you're going to see me slipping those in when we have decided as a group that we're going to do if null equals X so that we can never accidentally assign when we meant to compare. So that linter is going to catch some of my PRs or some of my commits and it's going to reject them because I didn't follow the pattern.

[00:29:43] Allan Stewart: Follow the pattern you did not.

[00:29:46] Dave Adsit: Follow the pattern I did not. Correct. Right. What are some of the tools that we could leverage as an architect to help us understand the system more deeply and provide evidence about the decisions that we're

[00:30:02] Allan Stewart: making? Yeah. One other thing that I think goes a long way in explaining your reasoning, including evidence, is explaining how it helps the business. It's very easy for architects to get wrapped up in just the technical side of things. And the developers as well. And developers are one of the primary audiences of the architect. And so if you're only worried about a decision because of how technically elegant it is, you might be missing out on something really important about the business. There may be some reason. It might be socio-technical again, where it's something about the people that is informing our decision. how our org chart is set up that influences the architecture. And so therefore we, we don't want to be fighting against Conway's law and there, and we've made this decision in, in respecting that. So helping people understand like, well, how does this help the business? And this is also very important because developers are not the only audience for the architect. And when an architect needs to talk with other disciplines or other members of the company that they can explain why they've made a technical decision, what some of those trade-offs were, and how is this going to help the business as opposed to another decision we might've made.

[00:31:35] Dave Adsit: Right. What are the decisions that, or one of the areas of decision-making that I think is often very important for an architect is around costs. In the book, Principles, product development flow. Don Reinertsen says, talk to the business in the language of money. You know, that's one of the, one of the principles that he offers for effective decision-making, right? If you can talk in terms of money, you're going to get a quicker decision and you're going to get, you're going to get more consistent buy-in. So if you, as an architect are making choices that would require us to spend $50,000, well, maybe you want to show us show me as your manager or whatever, how is this also going to increase our revenue or save us money down the road? Or like, don't just tell me you want an unlimited allowance to go buy all the cool toys. Show me how those toys, how those tools are going to be useful and valuable to what we're trying to deliver for the product or the company or whatever. Right. And so having some of that financial data is going to be super valuable. Like, oh, if we make this decision, we can reduce the number of server instances we need for the system, or we can increase the amount of load that we can put on the existing server instances. Or if we make this decision, we can leverage this tool to deliver 15 features that we've been looking, that we've been putting more quickly. Like, you know, maybe, maybe you want to make a case for buying versus building an authentication system or a database system, or so many things that are provided by our cloud vendors, that if you let somebody else handle the management, it's going to cost more in terms of upfront money, but it might save more money than that in terms of training people, right? So how do we heighten these costs? Tommy Tommy Tommy Tommy Tommy Tommy Tommy Tommy Tommy Tommy Tommy Tommy Tommy Tommy Tommy Instead, I can put that money towards some other valuable thing in the system. So that is one of the primary deliverables of an architect is decisions and all of the things that go into making good decisions, documenting decisions, justifying decisions, communicating decisions with the whole organization. But that's only one of the two primary things that we've identified as deliverables for an architect.

[00:34:43] Allan Stewart: That's right. The other one is diagrams. It seems inevitable that the architect is asked to or just can't refrain from drawing up all kinds of diagrams. And it's an important communication tool. Humans are very visual, typically. And that's why we get these phrases like a picture is worth a thousand words. Right. And a thousand times. Technical words are there really a lot of words. And so sometimes it's just better to have a picture that somebody can look at and see. And so there's so many tools out there. I did some searching online when we were discussing this topic. And almost every tool that I came up with in an online search was some sort of diagramming tool. So you might want to use. Right. Archimate or you want to use star UML or I like mermaid because I can just put some markdown in my markdown file and get hub renders it for me. That's pretty nice. But, you know, maybe you're old school. You want to use Visio. You want to use something like an online whiteboard, like Lucidchart or Miro. Like these are all options. There's so many tools out there. Like I've even just used like slides like PowerPoint or. Like Google's Google Sheets or Google Slides.

[00:36:18] Dave Adsit: Well, and, you know, I'm never far from a whiteboard. I have so many whiteboard markers and I have whiteboards everywhere I go. Everywhere I work from, I'm going to have a physical whiteboard that I use, which is fantastic if you're collaborating in person and terrible if you're collaborating remotely.

[00:36:39] Allan Stewart: Yeah. Yeah. And then so as we talk about diagrams. That's that kind of leads into one of the first things that you need to ask yourself. What is the purpose and audience of a diagram that you're creating? Does this need to be formal? Does it need to be informal? Do we need to have a tool to draw it out or possibly create the diagram for us? Or is it OK for it to just be hand drawn? Because there's these different scenarios. And one of one of the things that I really. Kind of my personal philosophy that I've been following for many years was inspired by Martin Fowler's UML distilled book. One of the early chapters he wrote that basically sometimes he would do this thing he called UML sketching. And the gist that I got from that was basically that, yes, there is a lot of detail in UML, but it's not always necessary for you to do all of it. And to make everything exactly correct to the UML spec. Sometimes it's enough to just say, well, I've got this box that and an arrow that goes to this other box. Because what you're doing is you're communicating between people. And if if that sketching is enough that the people all understand, then going to the nth degree of making sure that this UML diagram. Has all of the all of the information and all of the private variables and the public variables and all the method. A lot of times that is just not needed.

[00:38:24] Dave Adsit: Right. And I will admit that I have never understood which direction the arrows in UML are intended to go. Is this pointing to something I'm consuming or something I'm depending on or something that's depending on me? I don't know. It almost never matters. When I'm standing at the whiteboard or in front of some kind of a drawing tool with another person and we're talking about the thing.

[00:38:49] Allan Stewart: Yeah.

[00:38:50] Dave Adsit: Right.

[00:38:51] Allan Stewart: And that's that's how I have leaned personally. I almost always go with more of a sketching approach because I like to get involved with the teams that I'm working with and have an an active conversation. But that's not always the context that an architect is in. If you are in more of like an enterprise context. And you need to deliver a diagram to help developers understand how something's going to work. And this diagram is going to be shared across the world and in different languages and things like that. Then it might be important that you use a standard like UML or maybe the C4 model or something like that so that you can so that everybody who looks at it has a shared understanding that, oh, if the arrow looks like this, then we're talking about. Inheritance. And if you if the arrow looks like this, then we're talking about composition or something like that, because those details sometimes really matter.

[00:39:51] Dave Adsit: And they can communicate. It goes back to a picture can be worth a thousand words, right? If you have a consistent way of drawing the document, then or the diagram, then it can communicate a lot with very little syntax, if you will. Yeah. And I, you know, I like to use things. Both formally and informally, like UML at a C4 diagram or C4 model, right? The C4 model is just basically four layers that each have a C in them. One of them is the top layer is the system context, which describes the collection of all the things. And those things that are collected are then called containers. And this is not to be confused with Docker containers or any kind of containerization framework at the OS level. This is just like an application or a data store, the kind of thing that is containing either business logic or data or something to that effect. Right. So we've got the system context and then we've got the containers and the containers are built from components which may or may not be reused. And the components, of course, are made of code. And so you just say, hey, at the lowest level, I'm dealing with code and I'm going to bundle that code up into components and those components do things. They have. Interaction. Interfaces, whatever. And I'm going to bundle those components up into containers, which are applications or data or something to that effect. And then I'm going to put all of those containers into the system context. So top to bottom is four C's and they're simple and easy to understand. And they help us to navigate the different levels of scope and complexity that we are responsible for as software developers, as software architects, as system architects. So it. It can be very formal or very informal. Again, just like UML can just be a box and some arrows. The C4 model can be done very informally or it can be done very formally with all kinds of code annotations that help us understand exactly what's going on where and all kinds of things.

[00:41:59] Allan Stewart: Yeah, absolutely. And there's other kinds of models, too, that we can or kinds of diagrams that we can use because sometimes the boxes and arrows and containers that are. Describing bits of code. That's sometimes that's not sufficient. Sometimes we need something else. So some other kinds of things that we've talked about in the past are things like scorecard. Right. Sometimes we're doing a very different kind of view on the system. So instead of saying, oh, this is the system view that tells you how many web servers there are or how many classes there are in a component. Well, a scorecard. Might be how well are you doing in the latency of your API endpoints or something like that? Right. Another diagram that we talked about recently was a tech radar. Right. That communicates some important things about your system in in a visual way, but without but without having to necessarily be about the code. Right. Like this doesn't have to be about the infrastructure. Or the code or how many modules go inside your web server or how many microservices are in your system overall. Like for some purposes, those kinds of diagrams are important for other purposes. A scorecard or a tech radar might be a better fit.

[00:43:32] Dave Adsit: Right. If I want to know what technologies we are adopting now versus evaluating versus avoiding tech radar is a great way to communicate that effectively and quickly. With a team and easier to consume than a decision record, an architectural decision record, most likely. Yeah. One of the other tools that I've used, which is going to be, again, similar to UML diagrams are CRC cards. So basically, the idea here is that we're going to get three by five cards and Sharpies, and we're going to write down the classes and the responsibilities of those classes and the collaborators that those classes interact with. And we're going to move those cards around a lot as we're doing this. We're doing architectural diagramming and discussion. And we're going to say, hey, maybe we don't care about these cards right now. We're going to focus on this area and these cards. And then we're going to blow some of these cards out. And we're going to do this so that we can communicate effectively. And then maybe we'll stick them all up on the whiteboard with some blue painter's tape and then take a picture of it and upload it to the wiki or whatever. But again, it's a tool that allows us to create a diagram. In this case, a class diagram that has a little bit more detail about responsibilities and collaborators for those classes.

[00:44:49] Allan Stewart: Another diagram that I've been more recently introduced to that felt very, very different from the boxes and arrows that I was used to is a Wordly map. On a Wordly map, it's a diagram that helps you understand what your product is offering in terms of a value chain and how evolved each component on the value chain is. For example, can you get something from a commodity, right? So like, do you have to build your own database or can you just ask Amazon or Google or Microsoft to just give you one? And so like discussing what... Even what a Wordly map is maybe beyond the scope of this episode, but I found them very useful because they are strategic in a way that I had not considered previously. Really helping you understand how the progress of technology can open up new opportunities and help you identify places where you might be able to make strategic business decisions. That will have a long-term impact on your architecture or on your product.

[00:46:16] Dave Adsit: Yeah. And I liked... I'm obviously not great at Wordly maps. Maybe it's not obvious to anybody but you, Allan, but it's obvious to me as well. But I like to think of a Wordly map as a diagram of our technical strategy, our product strategy, of our strategic decisions that we're making in the system. And you think about... I think about the ideas of the evolution of a technology from its initial genesis when it's first innovated through its product phase where you're doing custom development and then you move into, well, it's been productized because everybody uses it kind of the same way, into it's become a commodity. Like, you know, those are the types of things that knowing where you're playing in that space and knowing what you're building on. And maybe what I want to be doing in my... in my product is the value I'm delivering is an innovative new use of a bunch of commodity products. Like, I'm going to combine these commodity things together so that I can deliver this innovative thing. And so you think about that and you're like, oh, great. Well, if that's my strategy, then that leads me to make certain decisions. I'm going to leverage cloud vendors for all these commodity things. So that I don't have to waste any time maintaining or building or patching or whatever those things. So that I can focus all my effort onto this spot where I'm either innovating something new or I'm doing custom development for a customer or maybe I'm building a first of its kind product. Right. And so the Wardley map is kind of your diagram of our product strategy.

[00:48:04] Allan Stewart: Yeah. Yeah. I remember we were working together. On a product one time that needed speech recognition. And there was a little bit... At one point, there was some wondering. It's like, well, should we standardize on a particular one? Should we even be considering building our own? Because some of the things that we're trying to do are very specific. And we might get better performance out of, you know, if we build our own. And creating a Wardley map was one way. That helped illustrate, at least to me, it became very obvious looking at the Wardley map. It's like, this is that rapidly approaching commodity. We could build our own, but it's very complicated and expensive. And you can just, you can just get one off the shelf now. For so, so, so it just made a lot of sense for us to make that our strategy. And instead of trying to own it and be able to fine tune all the details instead, And instead, we need to have a flexible arrangement of how we were doing our data science so that we could operate potentially with multiple different providers.

[00:49:17] Dave Adsit: Well, and that's an interesting thing where the diagram is the deliverable, but it's also an input into the decision-making, which is probably more common than we think about when we're right. We deliver these diagrams and then developers pick them up and look at them and they're like, oh, based on this diagram, I now know how to make this decision. Over here. Right. So that is how this tool can be reused. Right. Yeah. So there's a lot of other ways that we like to visualize things that are code, not just about our code, not just, you know, the box in line diagrams or the strategy diagrams, but there's tools like. Code C or moose tools or code city that actually allow you to take a code base and turn it into a visualization. We're more comfortable with. Right. So both moose tools and code city will do something like create city blocks based on module boundaries and then show you the volume of code based on the buildings that are being created in those blocks. And that gives you an idea of like, how habitable is this code base anyway? Right. So those are, those are kind of a static analysis that allows you to visualize the code or the infrastructure. There's also more of the dynamic analysis of like, what is the traffic between the different components or the different. If we're looking, if we were to go back to this C4 model and say, Hey, we've got these different containers or applications that are running on these different servers. How are they communicating between each other? Where are the bottlenecks? Right. So we could do static analysis and dynamic analysis of the system and visualize it. Then we get to engage different parts of our brain, which helps us solve problems. In different. Ways than if we're just talking about it all the time.

[00:51:11] Allan Stewart: Yeah. Plus it's just fun to see.

[00:51:13] Dave Adsit: It's really fun to see. I like, I like being able to say, Hey, here's let's analyze this code base. Hey, look in this file, there's a class. And in this class, there's like all these things that work together and all those things that work together. Is this really, should this be one cohesive class or is it actually two classes trying to get away from each other? Right. Like we talk about everything in software development comes back to. The two fundamental concepts of coupling and cohesion, right? We want loose coupling and high cohesion. So build me a thing that is very much about a single concept. And then I can put them all together, like little Lego parts to build a loosely coupled system.

[00:51:56] Allan Stewart: Which incidentally, oftentimes informs how well these visualization tools, like how, how useful they can be. Cause if you have a big ball of mud that has. All the coupling and very poor cohesion, then it's probably going to look like a big blob that is not easy to understand, but that tells you something. And if you clean up your code, lower the coupling and improve the cohesion, then all of a sudden these tools can give you something that is much, much more useful in, in what it's visualizing.

[00:52:33] Dave Adsit: Right.

[00:52:33] Allan Stewart: So I also think about how, just like when we were talking about the code. And how it doesn't matter how much of a piece of trash it is, right? where you get people together and it can be incredibly revealing to see just how everybody is understanding the system by getting them to just sit down and draw it. I think I've been part of numerous really interesting activities and trainings and discussions with a team by having every member of the team draw their own picture of how does the system work and then comparing those and looking at them. And it's not even because some people have it right or wrong. In fact, it's very rarely the case that there's anything wrong with a diagram in those exercises, but they're revealing different aspects of the system and different things that different

[00:53:38] Dave Adsit: members of the team find important. Right. That's what I was going to say about the draw the system is that there's a lot of ways that you can do it. You can sit everybody down and say, here is a blank piece of paper and a pen or pencil or a collection of colored pencils, whatever you need, draw the system as you understand it. And then we will, you know, compare and contrast them and we'll see, oh, this person doesn't even know about the email subsystem or that person is only focused on the payment processing system and put tons and tons and tons of detail into the payment processing system that this other person drew as a single box. Right. You can, you can, you can find out where there's complexity in your system. Nobody knew about this one part or everybody focused a ton of effort on this one part or whatever. And, you know, another way that I've done this is instead of having everybody go off on their own and do it, you just give everybody a marker and you say, okay, get up and draw the next thing that you think is most important. And so everybody will on one whiteboard draw one part of the system at a time, until we've put all of the important parts of the system on the whiteboard and we can see how it kind of unfolds and, and what is, what is known about the system, I should say. I was all, I almost said what is true about the system, but that's not necessarily the case. Right. It's like, what do we know about the system that we're responsible for? And I've done this exercise with the same team quarter after quarter after quarter. And then we'll start with a block of paper and then we'll start with a blank page every time. And you see what has become important since the last time we drew it. And you can see it evolve. If you, you know, snap a picture of the whiteboard at the end and then erase it. Right. Cause it's, it's not about the, this is the case where producing the diagram is a communication tool, but the diagram itself may not be super useful long-term. And you certainly don't want it to necessarily influence how people are continuing to build the system.

[00:55:49] Allan Stewart: Yeah. Yeah. It goes back to that idea of understanding the purpose and the audience. I can remember multiple times. There's one that stands out in my memory where a team that I was working on, we were trying to design a part of the system and we were really leaning heavily into one architectural pattern and it was getting complicated and more and more complicated. And we came to you and we're like, Dave, what should we do? This is getting so complicated. And then on, on the whiteboard that we had been drawing and putting, you know, boxes and arrows and, and lines and, and all those, all those things we're doing with the thing there. And you came in and you drew one line that crossed through all of them. And we were doing like a bunch of dashed lines for asynchronous message processing. And you're like, no one solid line for a synchronous API call. And we're like, Oh, because it's about communication, right? Like, help people understand a, a concept. And in, in the, in that kind of a context that we're just all working on the whiteboard that gave us the information we needed to know so that we could move forward.

[00:56:59] Dave Adsit: Right. Well, and I remember the team that you were with was doing everything they could to do one way asynchronous communication for this particular thing. And you were going about it in a very logical and determined way. And I think that you were accidentally reinventing TCP. Either that or some kind of a saga pattern. Right. And so in this case, the most valuable thing for the system was to move forward. So we chose to just do a synchronous call and take the performance hit of waiting for that other service to make a decision and return a result before moving on. Yeah. But if you hadn't drawn it all up, I probably would have, if we had just been discussing it without the diagram, I probably would have still said, Oh, well now we need to just have a message for when this happens so that we can respond to it over here. And we would have probably just kept going in the same direction.

[00:57:59] Allan Stewart: Yeah, that might be. So one thing as we're talking about diagrams that I think has been very valuable for me is being consistent in the diagrams. Now this is less important if, if all you're doing again, thinking about your audience and the context of how this is going to be. But if you're, if you're writing down like written documentation or you're creating a diagram that is going to be shared or maybe stored with our other documentation, having some decent level of consistency in that diagram can be really helpful. I went to a presentation once where they talked about having gestalt in your diagram. So it's kind of like a togetherness, a cohesiveness of the diagram. That lets you convey it clearly because, so for example, if in your diagram, you've got a whole bunch of boxes and then all of a sudden a circle. Well, what does the circle mean? If you just arbitrarily chose circle for one of those things, but it's really the same as, as the squares, you're going to add confusion. Same thing with other things like color or spacing. Sometimes those things can really matter in a diagram because of what it. Implies or what it's accidentally communicating. So if you've got a legend on the side and you say, oh yeah, these are color coded according to which team owns which service that might be very useful. If you have color coded them randomly arbitrarily, where most of them are white and one of them is red, then people are going to wonder like, what's wrong with the red one? Is it, is that one broken? Is it, are we trying to get rid of that service? So having some kind of consideration. And consistency in the diagram can really do a lot to improve, improve the usability, even the readability of the diagram. I've definitely seen ones, architecture diagrams that look like some kind of abstract art. Maybe the kind that where paint gets thrown on the wall or, or possibly spaghetti is thrown onto the wall because I can't quite understand what it is. And it's just like this mess of color. And line that, uh, that doesn't necessarily convey anything, uh, strategic to me.

[01:00:25] Dave Adsit: Right. Well, and I think that something that you are communicating here is that every diagram is actually wrong. They are the map, not the territory. Yeah. So every diagram should be focused on some aspect of the system and should not be interpreted to be the entirety of the system. Yeah. So, you know, we, uh, a map is a useful tool because it's an abstraction. Maybe I have a roadmap that I can use to find my way from one part of the country to another. And that is useful in one way, but a map of the local transit system would be useful in an entirely different way and not necessarily interchangeable. Yeah. Right. And, and we've talked about, you know, other types of maps. Maybe I want a topographical map or I want a rainfall map, or if I'm, I don't know, we I've recently tried to get into gardening unsuccessfully. Maybe I need a map of the, uh, agricultural zones of the country that I ha and so I can find out where I am and what I should expect to succeed with. Planting in my area based on rainfall and temperature. Right. All of these maps are. A subset of the total totality of the terrain and are therefore wrong. Yeah.

[01:01:56] Allan Stewart: I think, I think that's really important to consider because it turns out that, you know, uh, a map that is accurate, perfectly accurate. If you could do that would just be the place it wouldn't. Right. And so you can't get utility out of the map by it being hyper accurate. Because it's just. Like, well, I want to see. The roads, you know, which roads should I take? What path should I take to get from this city to this city? Well, if I have to walk the map to get to that city, it's not very useful. Um, and. You know, in a lot of cases, I think it's fascinating that they will purposely abstract things. So like railways, um, or subways, their maps are often wrong. Or at least they are not. They're not accurately representing the positions of these places. It looks like all of these exist on a straight line out of this spoke. And that's not true. Sometimes the track curves to the left quite significantly along your journey. But it's roughly. Irrelevant. It's roughly straight because the track. You can go forward or backwards on the track and we'll just call that straight. And it makes it more useful. For somebody who is trying to get around a big city.

[01:03:16] Dave Adsit: Right. So. Being an architect is hard. Because. You're always asked to provide some kind of a deliverable. About the system. Or what does an architect even do? What would you say you do here? Is a question that comes up often, right? What does that architect who is a very highly paid member of the technical staff do? He doesn't write that much code. He's not delivering that much code in the production system. So what is he doing and why is it valuable? Right. And so. Again, it boils down. In my experience to these two things. Architects make decisions and they draw diagrams. And so if we can do those two things effectively. Then we can be effective in the role of architect.