Skip to content

Commit

Permalink
[nop] Clarify CHANGELOG, data longevity
Browse files Browse the repository at this point in the history
  • Loading branch information
ptaoussanis committed May 31, 2024
1 parent 9ce3f98 commit b91e2b3
Show file tree
Hide file tree
Showing 4 changed files with 82 additions and 10 deletions.
22 changes: 16 additions & 6 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,15 +7,25 @@ This project uses [**Break Versioning**](https://www.taoensso.com/break-versioni
> **Dep**: Nippy is [on Clojars](https://clojars.org/com.taoensso/nippy/versions/3.4.2).
> **Versioning**: Nippy uses [Break Versioning](https://www.taoensso.com/break-versioning).
This release updates some internal dependencies and is **recommended for all existing users**.
This release includes **important updates to internal dependencies** and is **recommended for all existing users**.

It should be a **non-breaking update** for almost all users of Nippy `v3.4.x`, `v3.3.x`, and `v3.2.x`.
It should be a **straight-forward and non-breaking update** for almost everyone:

Notes:
| Updating from Nippy version | API changes? | Changes to [byte output](https://github.com/taoensso/nippy/wiki/2-Operational-considerations#stability-of-byte-output)? | New types |
| :-------------------------- | :----------- | :---------------------------------------------------------------------------------------------------------------------- | :---------------------------------------------- |
| `v3.4.1` (2024-05-02) | - | - | - |
| `v3.4.0` (2024-04-30) | - | Yes | `MapEntry` |
| `v3.3.0` (2023-10-11) | - | - | `java.lang.ClassCastException`, `java.sql.Date` |
| `v3.2.0` (2022-07-18) | - | - | `org.joda.time.DateTime` |
| `v3.1.3` (2022-06-23) | - | - | Several `java.time.X` types |

- May produce **different serialized output** to `v3.4.0` and `v3.3.0`. Most users won't care about this, but you could be affected if you depend on specific serialized byte values (for example by comparing serialized output between different versions of Nippy).
- When using Nippy version **X** to thaw data frozen by Nippy version **Y>X**, there is necessarily a risk of the thaw throwing when encountering unfamiliar types. This **can affect rolling updates** and/or **limit your ability to revert** a Nippy update - **so please ensure adequate testing** in your environment before updating against production data!
- As always, **please report any unexpected problems** 🙏
If updating from older versions of Nippy, please see the relevant release notes.

As always:

- See [operational considerations](https://github.com/taoensso/nippy/wiki/2-Operational-considerations) for info on: **data compatibility**, **rolling updates**, **rollback support**, etc.
- It's always a good idea to **ensure adequate testing** in your environment before updating against production data!
- **Please report any unexpected problems** 🙏

\- [Peter Taoussanis](https://www.taoensso.com)

Expand Down
38 changes: 35 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,9 @@

Clojure's rich data types are awesome. And its [reader](https://clojure.org/reference/reader) allows you to take your data just about anywhere. But the reader can be painfully slow when you've got a lot of data to crunch (like when you're serializing to a database).

Nippy is an attempt to provide a reliable, high-performance **drop-in alternative to the reader**.
Nippy is a mature, high-performance **drop-in alternative to the reader**.

Used by [Carmine](https://www.taoensso.com/carmine), [Faraday](https://www.taoensso.com/faraday), [PigPen](https://github.com/Netflix/PigPen), [Onyx](https://github.com/onyx-platform/onyx),
[XTDB](https://github.com/xtdb/xtdb), [Datalevin](https://github.com/juji-io/datalevin), and others.
It is used at scale by [Carmine](https://www.taoensso.com/carmine), [Faraday](https://www.taoensso.com/faraday), [PigPen](https://github.com/Netflix/PigPen), [Onyx](https://github.com/onyx-platform/onyx), [XTDB](https://github.com/xtdb/xtdb), [Datalevin](https://github.com/juji-io/datalevin), and others.

## Latest release/s

Expand All @@ -35,6 +34,39 @@ See [here][GitHub releases] for earlier releases.
- [Tools](https://taoensso.github.io/nippy/taoensso.nippy.tools.html) for easy + robust **integration into 3rd-party libraries**, etc.
- Powerful [thaw transducer](https://taoensso.github.io/nippy/taoensso.nippy.html#var-*thaw-xform*) for flexible data inspection and transformation

## Operational considerations

### Data longevity

Nippy is widely used to store **long-lived** data and promises (as always) that **data serialized today should be readable by all future versions of Nippy**.

But please note that the **converse is not generally true**:

- Nippy `vX` **should** be able to read all data from Nippy `vY<=X` (backwards compatibility)
- Nippy `vX` **may/not** be able to read all data from Nippy `vY>X` (forwards compatibility)

### Rolling updates and rollback

From time to time, Nippy may introduce:

- Support for serializing **new types**
- Optimizations to the serialization of **pre-existing types**

To help ease **rolling updates** and to better support **rollback**, Nippy (since version v3.4) will always introduce such changes over **two version releases**:

- Release 1: to add **read support** for the new types
- Release 2: to add **write support** for the new types

Starting from v3.4, Nippy's release notes will **always clearly indicate** if a particular update sequence is recommended.

### Stability of byte output

It has **never been an objective** of Nippy to offer **predictable byte output**, and I'd generally **recommend against** depending on specific byte output.

However, I know that a small minority of users *do* have specialized needs in this area.

So starting with Nippy v3.4, Nippy's release notes will **always clearly indicate** if any changes to byte output are expected.

## Performance

Since its earliest versions, Nippy has consistently been the **fastest serialization library for Clojure** that I'm aware of. Latest [benchmark](../../blob/master/test/taoensso/nippy_benchmarks.clj) results:
Expand Down
30 changes: 30 additions & 0 deletions wiki/2 Operational-considerations.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# Data longevity

Nippy is widely used to store **long-lived** data and promises (as always) that **data serialized today should be readable by all future versions of Nippy**.

But please note that the **converse is not generally true**:

- Nippy `vX` **should** be able to read all data from Nippy `vY<=X` (backwards compatibility)
- Nippy `vX` **may/not** be able to read all data from Nippy `vY>X` (forwards compatibility)

# Rolling updates and rollback

From time to time, Nippy may introduce:

- Support for serializing **new types**
- Optimizations to the serialization of **pre-existing types**

To help ease **rolling updates** and to better support **rollback**, Nippy (since version v3.4.1) will always introduce such changes over **two version releases**:

- Release 1: to add **read support** for the new types
- Release 2: to add **write support** for the new types

Starting from v3.4.1, Nippy's release notes will **always clearly indicate** if a particular update sequence is recommended.

# Stability of byte output

It has **never been an objective** of Nippy to offer **predictable byte output**, and I'd generally **recommend against** depending on specific byte output.

However, I know that a small minority of users *do* have specialized needs in this area.

So starting with Nippy v3.4, Nippy's release notes will **always clearly indicate** if any changes to byte output are expected.
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
> This article was kindly contributed by a Nippy user (@Outrovurt)
> This article is **community content** kindly contributed by a Nippy user (@Outrovurt)
This article describes a number of use cases where you need to make changes to your code which will have some impact on data you have already frozen using Nippy, and how best to manage each specific case. We will also discuss custom freezing and thawing.

Expand Down

0 comments on commit b91e2b3

Please sign in to comment.