Skip to content

Commit

Permalink
Add concepts around fire and forget
Browse files Browse the repository at this point in the history
  • Loading branch information
Martijn Dirkse committed Sep 16, 2024
1 parent 11fc96d commit 28bbad9
Show file tree
Hide file tree
Showing 3 changed files with 21 additions and 2 deletions.
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
*Under construction*

.. _advancedDevelopmentIntegrationPatternsFireForget:

Fire and Forget concepts
========================

This section turns to the fire and forget integration pattern introduced in the introduction :ref:`advancedDevelopmentIntegrationPatterns`. It means that the sender of a message does not wait until the receiver has processed the message. With the fire and forget integration pattern, the network may be set up so that the sender will never be informed about processing errors. This situation is supported by the Frank!Framework's *error store*. When processing of a message fails, it is stored in an error store so that an operator can resend the message from the Frank!Console. Instructions for the operator to do this are in :ref:`managingProcessedMessagesError`.

With the fire and forget integration pattern, messages are often put on a queue. Typically the sender gets a positive response when the request has been put on the queue. The intended recipient reads the request from the queue and processes it. When an error occurs, it may be desirable to roll back reading the request from the queue as well. Then the message can be read again later, hopefully when it can be processed successfully. This can be implemented using XA transactions, transactions that span across multiple data-processing systems, e.g. a queue and a database. Frank developers should also add an error store in this scenario, because processing the request may fail repeatedly. Operators should have a chance to correct errors by hand and then resend failed messages.

An alternative to using a real queue is using the database as a queue. This can be done by sending messages using a ``MessageStoreSender`` and by reading these messages using a ``MessageStoreListener``. The ``MessageStoreSender`` / ``MessageStoreListener`` pair uses database table ``IBISSTORE`` as the queue, which is also used by the ``<JdbcMessageLog>`` introduced in section :ref:`advancedDevelopmentIntegrationPatternsOnlyOnce`. This database-backed queue is insulated from other data in table ``IBISSTORE`` by providing a ``slotId``. All elements of the queue share the same value for the ``slotId`` field within table ``IBISSTORE`` while other data has a different value for this field.

When a The ``MessageStoreSender`` / ``MessageStoreListener`` pair is used, the Frank!Framework automatically adds an error store to the Frank!Console. Frank developers do not need to explicitly add an error store to their configuration in this case. Therefore, the examples that will be added in proceeding sections will cover a real queue with an explicit error store before turning to the ``MessageStoreSender`` and the ``MessageStoreListener``. This allows for clear examples to illustrate the concept of an error store.
Original file line number Diff line number Diff line change
Expand Up @@ -19,4 +19,5 @@ This section introduces concepts and features of the Frank!Framework to implemen
:maxdepth: 3

onlyOnce
transactions
transactions
fireAndForget
Original file line number Diff line number Diff line change
Expand Up @@ -23,12 +23,16 @@ Attribute ``checkForDuplicates="true"`` does the trick. The example expects the

Setting attribute ``processResultCacheSize="0"`` is a workaround for issue https://github.com/frankframework/frankframework/issues/7432. When this issue will have been solved, setting this attribute will not be necessary anymore.

The ``<JdbcMessageLog>`` configures the Frank!Framework to remember the incoming messages. They are stored in a database table named ``IBISSTORE``. The ``slotId`` attribute is needed to distinguish between the different ``<JdbcMessageLog>`` elements that can appear in a Frank application.
The ``<JdbcMessageLog>`` configures the Frank!Framework to remember the incoming messages. They are stored in a database table named ``IBISSTORE``. The ``slotId`` attribute is needed to distinguish between the different ``<JdbcMessageLog>`` elements that can appear in a Frank application. Message logs have another purpose in addition to remembering message ids already received - they act as audit logs as well. See :ref:`managingProcessedMessagesLog`.

.. WARNING::

Table ``IBISSTORE`` is only created if property ``jdbc.migrator.active`` is true and if this is configured as a system property or application property. Setting this within a configuration is not sufficient.

.. NOTE::

The fact that the ``<JdbcMessageLog>`` is backed by the database has an important consequence. When multiple instances of the application run in parallel, the message log still behaves as expected. This would not be possible if a message log would only keep its data in memory. If that would be the case, other instances would not know that some incoming message was seen already because in-memory information is not shared.

The receiver can be changed to expect a correlation id that is extracted from the incoming message, instead of a message id. The changes shown below modify ``Configuration.xml`` to process a message only if the extracted correlation id has not been seen before:

.. include:: ../../snippets/Frank2Transactions/v490/toCorrelationId.txt
Expand Down

0 comments on commit 28bbad9

Please sign in to comment.