Skip to content

An log structured merge tree varient of nitro storage engine

License

Notifications You must be signed in to change notification settings

t3rm1n4l/nitro-lsm

Repository files navigation

nitro

A high performance in-memory key-value item storage engine written in golang. The storage engine is based lock-free data structures and scales well with multicore CPUs.

Build Status Go Report Card GoDoc

Features

  • Operations: insert, delete, iterator (lookup, range queries)
  • Supports multiple concurrent readers and writers which scales almost linearly
  • Database snapshots which facilitates stable and repeatable scans
  • Lock-free data structures ensures that concurrent readers and writers doesn't block each other
  • The memory overhead of metadata of an item is 64 bytes
  • Fast snapshotting: Minimal overhead snapshots and they can be created frequently (eg,. every 10ms)
  • Optional memory manager based on jemalloc to avoid golang garbage collector for higher performance
  • Custom key comparator
  • Fast backup and restore on disk

Example usage

// Create a nitro instance with default config
db := nitro.New()
defer db.Close()

// Create a writer
// A writer should be created for every concurrent thread
w := db.NewWriter()
for i := 0; i < 100; i++ {
	itm := []byte(fmt.Sprintf("item-%02d", i))
	w.Put(itm)
}

// Create an immutable snapshot
snap1, _ := db.NewSnapshot()

for i := 0; i < 100; i++ {
	if i%2 == 0 {
		itm := []byte(fmt.Sprintf("item-%02d", i))
		w.Delete(itm)
	}
}

// Create an immutable snapshot
snap2, _ := db.NewSnapshot()

// Create an iterator for a snapshot
it1 := snap1.NewIterator()
count1 := 0
for it1.SeekFirst(); it1.Valid(); it1.Next() {
	fmt.Println("snap-1", string(it1.Get()))
	count1++
}

// Close snapshot and iterator once you have finished using them
it1.Close()
snap1.Close()

// Create an iterator for a snapshot
it2 := snap2.NewIterator()
count2 := 0
for it2.SeekFirst(); it2.Valid(); it2.Next() {
	fmt.Println("snap-2", string(it2.Get()))
	count2++
}

// Close snapshot and iterator once you have finished using them
it2.Close()
snap2.Close()

fmt.Println(count2 == count1/2)

License

Apache 2.0

About

An log structured merge tree varient of nitro storage engine

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published