Skip to content

Commit

Permalink
initial
Browse files Browse the repository at this point in the history
Signed-off-by: jkoberg <jkoberg@owncloud.com>
  • Loading branch information
kobergj committed Dec 1, 2023
1 parent 185a524 commit 78f1630
Show file tree
Hide file tree
Showing 11 changed files with 1,613 additions and 5 deletions.
10 changes: 6 additions & 4 deletions .golangci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -111,10 +111,11 @@ linters-settings:
# minimal occurrences count to trigger, 3 by default
min-occurrences: 3
depguard:
list-type: blacklist
# Packages listed here will reported as error if imported
packages:
- github.com/golang/protobuf/proto
rules:
main:
deny:
- pkg: "github.com/golang/protobuf/proto"
desc: not allowed
misspell:
# Correct spellings using locale preferences for US or UK.
# Default is to use a neutral variety of English.
Expand Down Expand Up @@ -192,6 +193,7 @@ linters:
- tagliatelle
- nosnakecase
- errchkjson
- forbidigo

# Can be considered to be enabled
- gochecknoinits
Expand Down
3 changes: 2 additions & 1 deletion go.work
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
go 1.18
go 1.21

use (
./v4/acme/certmagic
Expand Down Expand Up @@ -84,6 +84,7 @@ use (
./v4/store/memory
./v4/store/mysql
./v4/store/nats-js
./v4/store/nats-js-kv
./v4/store/redis
./v4/sync/consul
./v4/sync/etcd
Expand Down
79 changes: 79 additions & 0 deletions v4/store/nats-js-kv/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
# NATS JetStream Store Plugin

> **WARNING**: NATS Object Storage is still marked as an experimental preview.
This plugin uses the NATS JetStream [Object Store](https://docs.nats.io/nats-concepts/jetstream/obj_store) to implement the Go-Micro store interface.

This enables a key/value store with unlimited message size, allowing you to store files or data of any size under a single key.

You can use this plugin like any other store plugin.
To start a local NATS JetStream server run `nats-server -js`.

To manually create a new storage object call:

```go
natsjs.NewStore(opts ...store.Option)
```

The Go-Micro store interface uses databases and tables to store keys. These translate
to buckets (object stores) and key prefixes. If no database (bucket name) is provided, "default" will be used.

You can call `Write` with any arbitrary database name, and if a bucket with that name does not exist yet,
it will be automatically created.

If a table name is provided, it will use it to prefix the key as `<table>_<key>`.

To delete a bucket, and all the key/value pairs in it, pass the `DeleteBucket` option to the `Delete`
method, then they key name will be interpreted as a bucket name, and the bucket will be deleted.

Next to the default store options, a few NATS specific options are available:


```go
// NatsOptions accepts nats.Options
NatsOptions(opts nats.Options)

// JetStreamOptions accepts multiple nats.JSOpt
JetStreamOptions(opts ...nats.JSOpt)

// ObjectStoreOptions accepts multiple nats.ObjectStoreConfigs
// This will create buckets with the provided configs at initialization.
//
// type ObjectStoreConfig struct {
// Bucket string
// Description string
// TTL time.Duration
// MaxBytes int64
// Storage StorageType (either memory or file storage)
// Replicas int
// Placement *Placement
// }
//
ObjectStoreOptions(cfg ...*nats.ObjectStoreConfig)

// DefaultTTL sets the default TTL to use for new buckets
// By default no TTL is set.
//
// TTL ON INDIVIDUAL WRITE CALLS IS NOT SUPPORTED, only bucket wide TTL.
// Either set a default TTL with this option or provide bucket specific options
// with ObjectStoreOptions
DefaultTTL(ttl time.Duration)

// DefaultMemory sets the default storage type to memory only.
//
// The default is file storage, persisting storage between service restarts.
// Be aware that the default storage location of NATS the /tmp dir is, and thus
// won't persist reboots.
DefaultMemory()

// DefaultDescription sets the default description to use when creating new
// buckets. The default is "Store managed by go-micro"
DefaultDescription(text string)

// DeleteBucket will use the key passed to Delete as a bucket (database) name,
// and delete the bucket.
// This option should not be combined with the store.DeleteFrom option, as
// that will overwrite the delete action.
DeleteBucket()
```

18 changes: 18 additions & 0 deletions v4/store/nats-js-kv/context.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
package natsjskv

import (
"context"

"go-micro.dev/v4/store"
)

// setStoreOption returns a function to setup a context with given value.
func setStoreOption(k, v interface{}) store.Option {
return func(o *store.Options) {
if o.Context == nil {
o.Context = context.Background()
}

o.Context = context.WithValue(o.Context, k, v)
}
}
66 changes: 66 additions & 0 deletions v4/store/nats-js-kv/go.mod
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
module github.com/go-micro/plugins/v4/store/nats-js-kv

go 1.21

require (
github.com/cornelk/hashmap v1.0.8
github.com/nats-io/nats-server/v2 v2.8.4
)

require (
github.com/Microsoft/go-winio v0.5.2 // indirect
github.com/ProtonMail/go-crypto v0.0.0-20220824120805-4b6e5c587895 // indirect
github.com/acomagu/bufpipe v1.0.3 // indirect
github.com/bitly/go-simplejson v0.5.0 // indirect
github.com/cloudflare/circl v1.2.0 // indirect
github.com/cpuguy83/go-md2man/v2 v2.0.2 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/emirpasic/gods v1.18.1 // indirect
github.com/fsnotify/fsnotify v1.5.4 // indirect
github.com/go-git/gcfg v1.5.0 // indirect
github.com/go-git/go-billy/v5 v5.3.1 // indirect
github.com/go-git/go-git/v5 v5.4.2 // indirect
github.com/golang/protobuf v1.5.2 // indirect
github.com/google/go-cmp v0.5.6 // indirect
github.com/imdario/mergo v0.3.13 // indirect
github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99 // indirect
github.com/kevinburke/ssh_config v1.2.0 // indirect
github.com/klauspost/compress v1.17.0 // indirect
github.com/minio/highwayhash v1.0.2 // indirect
github.com/mitchellh/go-homedir v1.1.0 // indirect
github.com/nats-io/jwt/v2 v2.2.1-0.20220330180145-442af02fd36a // indirect
github.com/nats-io/nkeys v0.4.5 // indirect
github.com/nats-io/nuid v1.0.1 // indirect
github.com/nxadm/tail v1.4.8 // indirect
github.com/oxtoacart/bpool v0.0.0-20190530202638-03653db5a59c // indirect
github.com/patrickmn/go-cache v2.1.0+incompatible // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/russross/blackfriday/v2 v2.1.0 // indirect
github.com/sergi/go-diff v1.2.0 // indirect
github.com/test-go/testify v1.1.4 // indirect
github.com/urfave/cli/v2 v2.14.0 // indirect
github.com/xanzy/ssh-agent v0.3.2 // indirect
github.com/xrash/smetrics v0.0.0-20201216005158-039620a65673 // indirect
golang.org/x/mod v0.8.0 // indirect
golang.org/x/sys v0.5.0 // indirect
golang.org/x/text v0.13.0 // indirect
golang.org/x/time v0.0.0-20211116232009-f0f3c7e86c11 // indirect
golang.org/x/tools v0.6.0 // indirect
golang.org/x/xerrors v0.0.0-20220609144429-65e65417b02f // indirect
gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 // indirect
gopkg.in/warnings.v0 v0.1.2 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
)

require (
github.com/google/uuid v1.3.0
github.com/miekg/dns v1.1.50 // indirect
github.com/nats-io/nats.go v1.31.0
github.com/pkg/errors v0.9.1
github.com/stretchr/testify v1.7.1
go-micro.dev/v4 v4.9.0
golang.org/x/crypto v0.6.0 // indirect
golang.org/x/net v0.6.0 // indirect
golang.org/x/sync v0.1.0 // indirect
google.golang.org/protobuf v1.28.1 // indirect
)
Loading

0 comments on commit 78f1630

Please sign in to comment.