diff --git a/abstractions/12-queues-and-topics.md b/abstractions/12-queues-and-topics.md new file mode 100644 index 0000000..c8a6502 --- /dev/null +++ b/abstractions/12-queues-and-topics.md @@ -0,0 +1,82 @@ +--- +layout: default +title: Queues and topics +parent: Abstractions +nav_order: 12 +permalink: /abstractions/queues-and-topics +--- + +# Queues and topics + +Imagine that we have a service-based architecture, where a number of services (A and B) produce messages, +and a number of other services (C & D) consume messages. +A common way to diagram such an architecture is like this: + +[![](/images/queues-and-topics/1.png)](/images/queues-and-topics/1.png) + +From some points of view, this diagram is a true and accurate representation of the architecture. +Services A and B are sending messages to an intermediary, which is forwarding those messages on to services C and D. +The problem is that the "hub and spoke" nature of this diagram tends to obscure the real story, where a +degree of coupling may exist between the message producers and consumers. + +## Point-to-point + +The problem here is being caused by us representing the message bus as a C4 container, which arguably isn't correct. +A better approach is to think about each separate queue and topic as being a "data store". +A message queue is essentially a data store - it's a bucket for storing data (messages), +with producers adding data, and consumers taking it away. +The implication here is that the queues and topics are C4 containers, rather than the message bus itself. + +[![](/images/queues-and-topics/2.png)](/images/queues-and-topics/2.png) + +In this example, we can clearly see there's a coupling between services A and C via a queue named X, +and there's a similar point-to-point coupling between services B and D via a queue named Y. + +Modelling queues and topics as C4 containers also provides a way to think about them independently of their +deployment topology. At development time, you might have all queues and topics running on a single instance of a message +bus to save resources on your laptop. The live deployment topology might see individual queues and topics deployed +to separate message buses, brokers, or clusters for performance, scalability, or security reasons. + +If you genuinely have a point-to-point coupling via a queue, you could further simplify this diagram by omitting +the queues, and moving the queue names to the arrows instead. + +[![](/images/queues-and-topics/3.png)](/images/queues-and-topics/3.png) + +The result is a visually simpler and less cluttered diagram, but the queues are no +longer as explicitly evident on the diagram. Since the C4 model is notation independent, you could additionally use a +different line style (solid vs dashed) or colour to highlight message-based relationships. +Neither version of the diagrams is "better" than the other, they are just telling the same story in a different way. +It's all trade-offs. + +## Pub/sub + +You'll notice that all diagrams have shown messages flowing from the left of the diagram to the right, +with the arrows labelled as "Sends messages to". Although this works well, particularly for point-to-point and +queue-based architectures, you can also change the arrow directions to better highlight something +more pub/sub or topic-based. + +[![](/images/queues-and-topics/4.png)](/images/queues-and-topics/4.png) + +This version of the diagram better shows the roles of the message publishers and subscribers. Again, it's just a +different way of telling the same story. + +## Summary + +There are a number of ways to diagram message-based architectures, with the latter two of the following being "correct": + +- Incorrect: Model the message bus as a C4 container. +- Correct: Explicitly model queues and topics as C4 containers. +- Correct: Implicitly model message-based interactions using a "via" notation on relationships. + +## Further considerations + +The examples presented have assumed that a single software system is comprised from a number of services communicating +via queues and topics. In other words, everything on the diagram sits "inside" the software system boundary, and is +"owned" by that single software system. + +If you've read the [microservices](/abstractions/microservices) recommendations and you're modelling each +service as a separate software system, you additionally need to consider who "owns" the queues and topics. +If service A (a software system) has a point-to-point relationship with service B (also a software system) +via a queue named X (a container) ... who owns the container? Does service A own the definition of the message format +and operation of the queue, or is it service B? Or perhaps it's jointly owned, or owned by somebody else entirely. +Ownership will impact the diagrams. \ No newline at end of file diff --git a/faq.md b/faq.md index 11fb508..2939a78 100644 --- a/faq.md +++ b/faq.md @@ -59,19 +59,6 @@ Although you can certainly use the C4 model in this way, this is not the intende C4 model is just a way to describe a software system, from different levels of abstraction, and it implies nothing about the process of delivering software. -## Should you include message buses, API gateways, service meshes, etc? - -If you have two services, A and B, that communicate by sending a message via a message bus (irrespective of topics, -queues, p2p, pub/sub, etc) or another intermediary (e.g. an API gateway or service mesh), you have a couple of options. - -1. The first option is to show service A sending a message to the intermediary, and the intermediary subsequently forwarding -that message to service B. While accurate, the "hub and spoke" nature of the diagram tends to obscure the notion that -there's coupling between the message producer and consumer. - -2. The other approach is to omit the intermediary, and instead use notation (e.g. a textual description, colour coding, -line style, etc) to signify that the interaction between service A and B happens via an intermediary. This approach -3. tends to result in diagrams that tell a clearer story. - ## Using C4 to describe libraries, frameworks and SDKs? The C4 model is really designed to model a software system, at various levels of abstraction. To document a library, diff --git a/images/queues-and-topics/1.png b/images/queues-and-topics/1.png new file mode 100644 index 0000000..6be69ca Binary files /dev/null and b/images/queues-and-topics/1.png differ diff --git a/images/queues-and-topics/2.png b/images/queues-and-topics/2.png new file mode 100644 index 0000000..9380949 Binary files /dev/null and b/images/queues-and-topics/2.png differ diff --git a/images/queues-and-topics/3.png b/images/queues-and-topics/3.png new file mode 100644 index 0000000..b53c504 Binary files /dev/null and b/images/queues-and-topics/3.png differ diff --git a/images/queues-and-topics/4.png b/images/queues-and-topics/4.png new file mode 100644 index 0000000..237d5f5 Binary files /dev/null and b/images/queues-and-topics/4.png differ