Skip to content

Commit

Permalink
[nop] Update project template
Browse files Browse the repository at this point in the history
  • Loading branch information
ptaoussanis committed Sep 25, 2023
1 parent e864294 commit df5a7df
Show file tree
Hide file tree
Showing 6 changed files with 467 additions and 24 deletions.
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,4 @@ pom.xml*
/target/
/checkouts/
/logs/
/wiki/.git
31 changes: 16 additions & 15 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
This project uses Break Versioning (https://www.taoensso.com/break-versioning)
This project uses [**Break Versioning**](https://www.taoensso.com/break-versioning).

[releases]: https://github.com/taoensso/nippy/releases
---

# `v3.3.0-RC1` (2023-08-02)

Expand Down Expand Up @@ -35,7 +35,7 @@ Please test carefully and report any unexpected problems, thank you! 🙏
```

> This is a non-breaking maintenance release.
> See [here](https://github.com/ptaoussanis/encore#recommended-steps-after-any-significant-dependency-update) for recommended steps when updating any Clojure/Script dependencies.
> See [here](https://github.com/taoensso/encore#recommended-steps-after-any-significant-dependency-update) for recommended steps when updating any Clojure/Script dependencies.
## New since `v3.1.3`

Expand All @@ -53,7 +53,7 @@ Please test carefully and report any unexpected problems, thank you! 🙏
* [#145] [Fix] Freezing custom types with munged field names

The boxed Boolean bug has been around since the first version of Nippy and is mostly
relevant to users doing Java interop. For more info see: https://github.com/ptaoussanis/nippy/commit/8909a32bdd654a136da385e0e09c9cc44416f964
relevant to users doing Java interop. For more info see: https://github.com/taoensso/nippy/commit/8909a32bdd654a136da385e0e09c9cc44416f964


# `v3.2.0-RC3` (2022-06-27)
Expand All @@ -63,7 +63,7 @@ relevant to users doing Java interop. For more info see: https://github.com/ptao
```

> This is a non-breaking maintenance release.
> See [here](https://github.com/ptaoussanis/encore#recommended-steps-after-any-significant-dependency-update) for recommended steps when updating any Clojure/Script dependencies.
> See [here](https://github.com/taoensso/encore#recommended-steps-after-any-significant-dependency-update) for recommended steps when updating any Clojure/Script dependencies.
## New since `v3.1.3`

Expand All @@ -81,7 +81,7 @@ relevant to users doing Java interop. For more info see: https://github.com/ptao
* [#145] [Fix] Freezing custom types with munged field names

The boxed Boolean bug has been around since the first version of Nippy and is mostly
relevant to users doing Java interop. For more info see: https://github.com/ptaoussanis/nippy/commit/8909a32bdd654a136da385e0e09c9cc44416f964
relevant to users doing Java interop. For more info see: https://github.com/taoensso/nippy/commit/8909a32bdd654a136da385e0e09c9cc44416f964


# `v3.2.0-RC2` (2022-06-23)
Expand All @@ -91,7 +91,7 @@ relevant to users doing Java interop. For more info see: https://github.com/ptao
```

> This is a non-breaking maintenance release.
> See [here](https://github.com/ptaoussanis/encore#recommended-steps-after-any-significant-dependency-update) for recommended steps when updating any Clojure/Script dependencies.
> See [here](https://github.com/taoensso/encore#recommended-steps-after-any-significant-dependency-update) for recommended steps when updating any Clojure/Script dependencies.
## New since `v3.1.3`

Expand All @@ -108,7 +108,7 @@ relevant to users doing Java interop. For more info see: https://github.com/ptao
* [#89 #150] [Fix] Boxed Booleans incorrectly freezing to primitive `true` (@RolT)

The boxed Boolean bug has been around since the first version of Nippy and is mostly
relevant to users doing Java interop. For more info see: https://github.com/ptaoussanis/nippy/commit/8909a32bdd654a136da385e0e09c9cc44416f964
relevant to users doing Java interop. For more info see: https://github.com/taoensso/nippy/commit/8909a32bdd654a136da385e0e09c9cc44416f964


# `v3.1.3` (2022-06-23)
Expand All @@ -118,15 +118,15 @@ relevant to users doing Java interop. For more info see: https://github.com/ptao
```

> This is a non-breaking, bugfix release.
> See [here](https://github.com/ptaoussanis/encore#recommended-steps-after-any-significant-dependency-update) for recommended steps when updating any Clojure/Script dependencies.
> See [here](https://github.com/taoensso/encore#recommended-steps-after-any-significant-dependency-update) for recommended steps when updating any Clojure/Script dependencies.
## Fixes since `v3.1.1`

* [#148] [Fix] `tools/freeze` should use `*freeze-opts*` even for unwrapped vals
* [#89 #150] [Fix] Boxed Booleans incorrectly freezing to primitive `true` (@RolT)

The boxed Boolean bug has been around since the first version of Nippy and is mostly
relevant to users doing Java interop. For more info see: https://github.com/ptaoussanis/nippy/commit/8909a32bdd654a136da385e0e09c9cc44416f964
relevant to users doing Java interop. For more info see: https://github.com/taoensso/nippy/commit/8909a32bdd654a136da385e0e09c9cc44416f964


# `v3.1.1` (2020-11-18)
Expand Down Expand Up @@ -184,7 +184,7 @@ relevant to users doing Java interop. For more info see: https://github.com/ptao

> This release is focused on smoothing out rough edges left by `CVE-2020-24164` [#130], and to **ease transition** from versions of Nippy < `v2.15.0 final`.
> See [here](https://github.com/ptaoussanis/encore#recommended-steps-after-any-significant-dependency-update) for recommended steps when updating any Clojure/Script dependencies.
> See [here](https://github.com/taoensso/encore#recommended-steps-after-any-significant-dependency-update) for recommended steps when updating any Clojure/Script dependencies.
Note that there's **separate details** below for upgrading from `v2.15` vs `v2.14`:

Expand All @@ -203,8 +203,8 @@ Usually a non-breaking drop-in replacement, but there's some changes you might l

- [#122] Option to disable freezing and/or thawing of metadata.
- `freeze` and `thaw` now support opts: `:serializable-allowlist`, `:incl-metadata?`.
- New `read-quarantined-serializable-object-unsafe!` util to read quarantined Serializable objects. See [API docs](http://ptaoussanis.github.io/nippy/taoensso.nippy.html#var-read-quarantined-serializable-object-unsafe.21) and/or [#130] for details.
- Add `allow-and-record-any-serializable-class-unsafe` util. See [API docs](http://ptaoussanis.github.io/nippy/taoensso.nippy.html#var-allow-and-record-any-serializable-class-unsafe) and/or [#130] for details.
- New `read-quarantined-serializable-object-unsafe!` util to read quarantined Serializable objects. See [API docs](http://taoensso.github.io/nippy/taoensso.nippy.html#var-read-quarantined-serializable-object-unsafe.21) and/or [#130] for details.
- Add `allow-and-record-any-serializable-class-unsafe` util. See [API docs](http://taoensso.github.io/nippy/taoensso.nippy.html#var-allow-and-record-any-serializable-class-unsafe) and/or [#130] for details.


## Upgrading from `v2.14` (may be BREAKING)
Expand All @@ -230,7 +230,8 @@ Likely breaking. Please see [#130] for **detailed upgrade instructions**.

- [#120] Update `freezable?` to cover `nil`


---

# Earlier releases

See [here][releases] for earlier releases.
See [here](https://github.com/taoensso/nippy/releases) for earlier releases.
20 changes: 11 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,14 @@

# Nippy

#### The fastest serialization library for Clojure
### The fastest serialization library for Clojure

Clojure's rich data types are awesome. And its [reader](https://clojure.org/reference/reader) allows you to take your data just about anywhere. But the reader can be painfully slow when you've got a lot of data to crunch (like when you're serializing to a database).

Nippy is an attempt to provide a reliable, high-performance **drop-in alternative to the reader**.

Used by [Carmine](https://github.com/taoensso/carmine), [Faraday](https://github.com/ptaoussanis/faraday), [PigPen](https://github.com/Netflix/PigPen), [Onyx](https://github.com/onyx-platform/onyx), [XTDB](https://github.com/xtdb/xtdb), and others.
Used by [Carmine](https://www.taoensso.com/carmine), [Faraday](https://www.taoensso.com/faraday), [PigPen](https://github.com/Netflix/PigPen), [Onyx](https://github.com/onyx-platform/onyx),
[XTDB](https://github.com/xtdb/xtdb), [Datalevin](https://github.com/juji-io/datalevin), and others.

## Latest release/s

Expand All @@ -25,14 +26,15 @@ See [here][GitHub releases] for earlier releases.

- Small, simple **all-Clojure** library
- **Terrific performance**: the [best](#performance) for Clojure that I'm aware of
- Comprehensive support for **all standard data types**
- Easily extendable to **custom data types**
- Comprehensive support for [all standard data types](../../wiki/1-Getting-started#deserializing)
- Easily extendable to [custom data types](../../wiki/1-Getting-started#custom-types)
- **Robust test suite**, incl. full coverage for every supported type
- Auto fallback to Java Serializable when available
- Auto fallback to Clojure Reader for all other types (including tagged literals
- Pluggable **compression** with built-in [LZ4](https://code.google.com/p/lz4/)
- Pluggable **encryption** with built-in AES128
- Tools for easy + robust **integration into 3rd-party libraries**, etc.
- Pluggable [encryption](../../wiki/1-Getting-started#encryption) with built-in AES128
- [Tools](https://taoensso.github.io/nippy/taoensso.nippy.tools.html) for easy + robust **integration into 3rd-party libraries**, etc.
- Powerful [thaw transducer](https://taoensso.github.io/nippy/taoensso.nippy.html#var-*thaw-xform*) for flexible data inspection and transformation

## Performance

Expand All @@ -41,9 +43,9 @@ Since its earliest versions, Nippy has consistently been the **fastest serializa
- Roundtrip times **>12x faster** than `tools.reader` with **60% smaller** data size.
- Roundtrip times **>2x faster** than `data.fressian` with **30% smaller** data size.

![benchmarks-png](benchmarks.png)
![benchmarks-png](../../raw/master/benchmarks.png)

The [benchmark code](https://github.com/taoensso/nippy/blob/master/src/taoensso/nippy/benchmarks.clj) can be easily run in your own environment.
The [benchmark code](../../blob/master/src/taoensso/nippy/benchmarks.clj) can be easily run in your own environment.

## Documentation

Expand All @@ -52,7 +54,7 @@ The [benchmark code](https://github.com/taoensso/nippy/blob/master/src/taoensso/

## Funding

You can [help support][sponsor] on this project, thank you!! 🙏
You can [help support][sponsor] continued work on this project, thank you!! 🙏

## License

Expand Down
153 changes: 153 additions & 0 deletions wiki/1 Getting-started.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,153 @@
# Setup

Add the [relevant dependency](../#latest-releases) to your project:

```clojure
Leiningen: [com.taoensso/nippy "x-y-z"] ; or
deps.edn: com.taoensso/nippy {:mvn/version "x-y-z"}
```

And setup your namespace imports:

```clojure
(ns my-app (:require [taoensso.nippy :as nippy]))
```

# De/serializing

As an example of what it can do, let's take a look at Nippy's own reference stress data:

```clojure
nippy/stress-data
=>
{:nil nil
:true true
:false false
:boxed-false (Boolean. false)

:char \ಬ
:str-short "ಬಾ ಇಲ್ಲಿ ಸಂಭವಿಸ"
:str-long (apply str (range 1000))
:kw :keyword
:kw-ns ::keyword
:kw-long (keyword
(apply str "kw" (range 1000))
(apply str "kw" (range 1000)))

:sym 'foo
:sym-ns 'foo/bar
:sym-long (symbol
(apply str "sym" (range 1000))
(apply str "sym" (range 1000)))

:regex #"^(https?:)?//(www\?|\?)?"

;;; Try reflect real-world data:
:many-small-numbers (vec (range 200))
:many-small-keywords (->> (java.util.Locale/getISOLanguages)
(mapv keyword))
:many-small-strings (->> (java.util.Locale/getISOCountries)
(mapv #(.getDisplayCountry (java.util.Locale. "en" %))))

:queue (enc/queue [:a :b :c :d :e :f :g])
:queue-empty (enc/queue)
:sorted-set (sorted-set 1 2 3 4 5)
:sorted-map (sorted-map :b 2 :a 1 :d 4 :c 3)

:list (list 1 2 3 4 5 (list 6 7 8 (list 9 10 '(()))))
:vector [1 2 3 4 5 [6 7 8 [9 10 [[]]]]]
:map {:a 1 :b 2 :c 3 :d {:e 4 :f {:g 5 :h 6 :i 7 :j {{} {}}}}}
:set #{1 2 3 4 5 #{6 7 8 #{9 10 #{#{}}}}}
:meta (with-meta {:a :A} {:metakey :metaval})
:nested [#{{1 [:a :b] 2 [:c :d] 3 [:e :f]} [#{{}}] #{:a :b}}
#{{1 [:a :b] 2 [:c :d] 3 [:e :f]} [#{{}}] #{:a :b}}
[1 [1 2 [1 2 3 [1 2 3 4 [1 2 3 4 5]]]]]]

:lazy-seq (repeatedly 1000 rand)
:lazy-seq-empty (map identity '())

:byte (byte 16)
:short (short 42)
:integer (int 3)
:long (long 3)
:bigint (bigint 31415926535897932384626433832795)

:float (float 3.14)
:double (double 3.14)
:bigdec (bigdec 3.1415926535897932384626433832795)

:ratio 22/7
:uri (java.net.URI. "https://clojure.org/reference/data_structures")
:uuid (java.util.UUID/randomUUID)
:util-date (java.util.Date.)
:sql-date (java.sql.Date/valueOf "2023-06-21")

;;; JVM 8+
:time-instant (enc/compile-if java.time.Instant (java.time.Instant/now) nil)
:time-duration (enc/compile-if java.time.Duration (java.time.Duration/ofSeconds 100 100) nil)
:time-period (enc/compile-if java.time.Period (java.time.Period/of 1 1 1) nil)

:bytes (byte-array [(byte 1) (byte 2) (byte 3)])
:objects (object-array [1 "two" {:data "data"}])

:stress-record (StressRecord. "data")
:stress-type (StressType. "data")

;; Serializable
:throwable (Throwable. "Yolo")
:exception (try (/ 1 0) (catch Exception e e))
:ex-info (ex-info "ExInfo" {:data "data"})}
```

Serialize it:

```clojure
(def frozen-stress-data (nippy/freeze nippy/stress-data))
=> #<byte[] [B@3253bcf3>
```

Deserialize it:

```clojure
(nippy/thaw frozen-stress-data)
=> {:bytes (byte-array [(byte 1) (byte 2) (byte 3)])
:nil nil
:boolean true
<...> }
```

Couldn't be simpler!

See also the lower-level [`freeze-to-out!`](https://taoensso.github.io/nippy/taoensso.nippy.html#var-freeze-to-out.21) and [`thaw-from-in!`](https://taoensso.github.io/nippy/taoensso.nippy.html#var-thaw-from-in.21) fns for operating on `DataOutput` and `DataInput` types directly.

# Encryption

> You may want to consider using Nippy with [Tempel](https://www.taoensso.com/tempel) for more comprehensive encryption options.

Nippy also gives you **dead simple data encryption**.
Add a single option to your usual freeze/thaw calls like so:

```clojure
(nippy/freeze nippy/stress-data {:password [:salted "my-password"]}) ; Encrypt
(nippy/thaw <encrypted-data> {:password [:salted "my-password"]}) ; Decrypt
```

There's two default forms of encryption on offer: `:salted` and `:cached`. Each of these makes carefully-chosen trade-offs and is suited to one of two common use cases. See [`aes128-encryptor`](https://taoensso.github.io/nippy/taoensso.nippy.html#var-aes128-encryptor) for a detailed explanation of why/when you'd want one or the other.

# Custom types

It's easy to extend Nippy to your own data types:

```clojure
(defrecord MyType [data])

(nippy/extend-freeze MyType :my-type/foo ; A unique (namespaced) type identifier
[x data-output]
(.writeUTF data-output (:data x)))

(nippy/extend-thaw :my-type/foo ; Same type id
[data-input]
(MyType. (.readUTF data-input)))

(nippy/thaw (nippy/freeze (MyType. "Joe"))) => #taoensso.nippy.MyType{:data "Joe"}
```
Loading

0 comments on commit df5a7df

Please sign in to comment.