FAQ.md

FAQ

• How do I create an index?
• How do I get my data into search-index?
  • Export/Import an index
  • Add documents to an index
• Can I use another backend like MySQL or Redis?
• How do I get out entire documents and not just document IDs?
• How do I search on specific fields?
• How do I compose queries?
• How do I perform a simple aggregation on a field?
  • Get a set of document ids per unique field value
  • Get counts per unique field value
  • Define custom "buckets"
  • Combine an aggregation with a search
• How do I make a simple typeahead / autosuggest / matcher

How do I create an index?

const si = import si from 'si'

// ...

const index = await si() // or si().then(index => ...)

There is also a file in the dist folder called search-index.x.x.x.js (replace 'x.x.x' with version number) that can be referenced from a script tag:

<script type='text/javascript' src='search-index.x.x.x.js'></script>
<script type='text/javascript'>
  import si from search-index
  // ...
</script>

How do I get my data into search-index?

You can either import an index, or add documents:

Export/Import an index

// EXPORT an index
const exportFile = await index1.EXPORT()

// IMPORT an index
index2.IMPORT(exportFile)

NOTE: IMPORTing an index completely overwrites any existing index, so if you had an existing index that contained several thousand documents into which you imported an external index that contained one document, then your existing index would now contain one document.

Add documents to an index

// then somewhere else in the code, being aware of asynchronousity
PUT([ /* my array of objects */ ]).then(doStuff)

Can I use another backend like MySQL or Redis?

Yes you can! Because search-index is built on top of levelup its possible to use another backend by passing the appropriate abstract-leveldown when initialising. Many datastores are supported. Use the db initialisation option. (see tests for a working example).

Example:

const memdown = require('memdown')
const si = require('search-index')

const memdownIndex = await si({
  db: memdown,
  name: indexName
})

How do I get out entire documents and not just document IDs?

Use { DOCUMENTS: true }.

Query that returns document IDs:

SEARCH([ 'search', 'terms' ])

Query that returns documents:

SEARCH([ 'search', 'terms' ], { DOCUMENTS: true })

How do I search on specific fields?

To return hits for all documents containing 'orange' in the title field you would do something like this:

QUERY({
  AND: [ 'title:orange' ]
})

// can also be expressed as:
QUERY({
  AND: [{
    FIELD: [ 'title' ],
    VALUE: 'orange'
  }]
})

// or even
QUERY({
  AND: [{
    FIELD: [ 'title' ],
    VALUE: {
      GTE: 'orange',
      LTE: 'orange'
    }
  }]
})

How do I compose queries?

Queries can be composed by nesting SEARCH, AND, NOT and OR clauses as deeply as required. For example:

QUERY({
  OR: [
    {
      AND: [ 'brand:volvo', 'manufacturer:tesla' ]
    },
    'make:bmw'
  ]
})

How do I perform a simple aggregation on a field?

Get a set of document ids per unique field value

const facets = await FACETS({ FIELD: name })

Get counts per unique field value

const facets = await FACETS({ FIELD: name })
  .then(fcts => fcts.map(
    f => ({
      FIELD: f.FIELD,
      VALUE: f.VALUE,
      count: f._id.length
    })
  ))

Define custom "buckets"

const buckets = await BUCKETS ([
  token1,
  token2,
  token3
])

Combine an aggregation with a search

By using QUERY with the appropriate options, you can create aggregations on the set of documents returned by the QUERY.

// also works with BUCKETS
const buckets = QUERY(q, {
  FACETS: [ { FIELD: name } ]
})

How do I make a simple typeahead / autosuggest / matcher

There are of course many ways to do this, but if you just want a simple "begins with" autosuggest, then you can simply use the DICTIONARY function:

const results = DICTIONARY('b')   // [ 'bananas','branch','brunch' ]
const results = DICTIONARY('br')  // [ 'branch','brunch' ]
const results = DICTIONARY('bra') // [ 'branch' ]

Alternatively you can use DICTIONARY to extract all terms from the index and then feed them into some third-party matcher logic.

import FuzzySet from 'fuzzyset'
// ...
const dict = await DICTIONARY()
const fs = FuzzySet()
dict.forEach(d => fs.add(d))
// ...
// fuzzy matching
fs.get(searchTerm)