Skip to content

Commit

Permalink
Adds a page about queues and topics.
Browse files Browse the repository at this point in the history
  • Loading branch information
Simon Brown committed Sep 19, 2024
1 parent ce4f56a commit 7a4ecae
Show file tree
Hide file tree
Showing 6 changed files with 82 additions and 13 deletions.
82 changes: 82 additions & 0 deletions abstractions/12-queues-and-topics.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
---
layout: default
title: Queues and topics
parent: Abstractions
nav_order: 12
permalink: /abstractions/queues-and-topics
---

# Queues and topics

Imagine that we have a service-based architecture, where a number of services (A and B) produce messages,
and a number of other services (C & D) consume messages.
A common way to diagram such an architecture is like this:

[![](/images/queues-and-topics/1.png)](/images/queues-and-topics/1.png)

From some points of view, this diagram is a true and accurate representation of the architecture.
Services A and B are sending messages to an intermediary, which is forwarding those messages on to services C and D.
The problem is that the "hub and spoke" nature of this diagram tends to obscure the real story, where a
degree of coupling may exist between the message producers and consumers.

## Point-to-point

The problem here is being caused by us representing the message bus as a C4 container, which arguably isn't correct.
A better approach is to think about each separate queue and topic as being a "data store".
A message queue is essentially a data store - it's a bucket for storing data (messages),
with producers adding data, and consumers taking it away.
The implication here is that the queues and topics are C4 containers, rather than the message bus itself.

[![](/images/queues-and-topics/2.png)](/images/queues-and-topics/2.png)

In this example, we can clearly see there's a coupling between services A and C via a queue named X,
and there's a similar point-to-point coupling between services B and D via a queue named Y.

Modelling queues and topics as C4 containers also provides a way to think about them independently of their
deployment topology. At development time, you might have all queues and topics running on a single instance of a message
bus to save resources on your laptop. The live deployment topology might see individual queues and topics deployed
to separate message buses, brokers, or clusters for performance, scalability, or security reasons.

If you genuinely have a point-to-point coupling via a queue, you could further simplify this diagram by omitting
the queues, and moving the queue names to the arrows instead.

[![](/images/queues-and-topics/3.png)](/images/queues-and-topics/3.png)

The result is a visually simpler and less cluttered diagram, but the queues are no
longer as explicitly evident on the diagram. Since the C4 model is notation independent, you could additionally use a
different line style (solid vs dashed) or colour to highlight message-based relationships.
Neither version of the diagrams is "better" than the other, they are just telling the same story in a different way.
It's all trade-offs.

## Pub/sub

You'll notice that all diagrams have shown messages flowing from the left of the diagram to the right,
with the arrows labelled as "Sends messages to". Although this works well, particularly for point-to-point and
queue-based architectures, you can also change the arrow directions to better highlight something
more pub/sub or topic-based.

[![](/images/queues-and-topics/4.png)](/images/queues-and-topics/4.png)

This version of the diagram better shows the roles of the message publishers and subscribers. Again, it's just a
different way of telling the same story.

## Summary

There are a number of ways to diagram message-based architectures, with the latter two of the following being "correct":

- Incorrect: Model the message bus as a C4 container.
- Correct: Explicitly model queues and topics as C4 containers.
- Correct: Implicitly model message-based interactions using a "via" notation on relationships.

## Further considerations

The examples presented have assumed that a single software system is comprised from a number of services communicating
via queues and topics. In other words, everything on the diagram sits "inside" the software system boundary, and is
"owned" by that single software system.

If you've read the [microservices](/abstractions/microservices) recommendations and you're modelling each
service as a separate software system, you additionally need to consider who "owns" the queues and topics.
If service A (a software system) has a point-to-point relationship with service B (also a software system)
via a queue named X (a container) ... who owns the container? Does service A own the definition of the message format
and operation of the queue, or is it service B? Or perhaps it's jointly owned, or owned by somebody else entirely.
Ownership will impact the diagrams.
13 changes: 0 additions & 13 deletions faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,19 +59,6 @@ Although you can certainly use the C4 model in this way, this is not the intende
C4 model is just a way to describe a software system, from different levels of abstraction, and it implies nothing
about the process of delivering software.

## Should you include message buses, API gateways, service meshes, etc?

If you have two services, A and B, that communicate by sending a message via a message bus (irrespective of topics,
queues, p2p, pub/sub, etc) or another intermediary (e.g. an API gateway or service mesh), you have a couple of options.

1. The first option is to show service A sending a message to the intermediary, and the intermediary subsequently forwarding
that message to service B. While accurate, the "hub and spoke" nature of the diagram tends to obscure the notion that
there's coupling between the message producer and consumer.

2. The other approach is to omit the intermediary, and instead use notation (e.g. a textual description, colour coding,
line style, etc) to signify that the interaction between service A and B happens via an intermediary. This approach
3. tends to result in diagrams that tell a clearer story.

## Using C4 to describe libraries, frameworks and SDKs?

The C4 model is really designed to model a software system, at various levels of abstraction. To document a library,
Expand Down
Binary file added images/queues-and-topics/1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/queues-and-topics/2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/queues-and-topics/3.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/queues-and-topics/4.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 7a4ecae

Please sign in to comment.