Icelastic

Rack middleware that provides Cool URIs for Elasticsearch-based web services.

Features

Real-world examples

Icelastic is used in production at the Norwegian Polar Institute APIs, below are some real-world example queries

• Searching ?q=
• Filtering filter-{variable}=value
• Faceting facets={variable[,varible2]}
• Range-faceting (aka. bucketing)

Multiple response formats

The default Icelastic response format is a JSON feed modeled after Atom/OpenSearch.

• JSON format=json
• CSV format=csv
• GeoJSON format=geojson
• JSON array - format=json&variant=array
• HAL - format=hal
• raw - format=raw

Use

Simple

# config.ru
require 'icelastic'

use Rack::Icelastic, {
  :url => "http://localhost:9200", :index => "example", :type => "test" }
}

Advanced

Example of injecting a custom CDL response writer, and setting various configuraton options

writers = Icelastic::Default.writers
writers << My::ElasticsearchCDLWriter

use ::Rack::Icelastic, {
  :url => "http://localhost:9200",
  :index => "oceanography",
  :log => false, # Logging in elasticsearch-ruby ?
  :type => "point",
  :params => {
    "facets" => "station,cruise,ctd,collection,mooring,serialnumber",
    "date-year" => "measured", # Date facets with a year interval
    "limit" => 10, # Items per page
    "size-facet" => 100 # Number of facet items
  },
  :writers => writers
}

module My
  class ElasticsearchCDLWriter

    def self.format
      "cdl"
    end

    def self.type
      "text/plain"
    end

    def initialize(request, elasticsearch_response_hash)
      @feed = Icelastic::ResponseWriter::Feed.new(request, elasticsearch_response_hash)
    end

    def build
      @feed.build do |feed|
        "netcdf {}" # build response here
      end
    end
  end
end

URI reference

Search

  "?q=<value>" # Regular query
  "?q-<field>=<value>" # Field query
  "?q-<field1>,<field2>=<value>" # Multi-field query

Global parameters (paging, sorting, limiting, scoring)

  "?start=10" # Results are shown from the 10th row
  "?limit=50" # Show 50 rows per result page


  "?sort=<field>" # Sort ascending on field
  "?sort=-<field>" # Sort descending on field

  "?fields=<field1>,<field2>,<field3>" # Only show fields 1,2,3 in the response rows
  "?highlight=true" # Enable term highlighting. Injects a highlight key with the relevant sections into the entry
  "?score" # Include relevance scoring in result

Filtering

  "?filter-<field>=<value>" # Basic filter

  "?filter-<field>=<value1>,<value2>" # AND filter
  "?filter-<field>=<value1>|<value2>" # OR filter
  "?not-<field>=<value>" # NOT filter (starts with not instead of filter)

  "?filter-<field>=<value1>..<value2>" # Range filter
  "?filter-<field>=<value>.." # Range filter (greater or equal then)
  "?filter-<field>=..<value>" # Range filter (less or equal then)

Faceting

  "?facets=<field1>,<field2>" # Facet on field1 and field2
  "?facet-<name>=<field>" # Labeled facet (generates a facet with a specific name)

  "?date-<interval>=<field1>,<field2>" # Generate a date facet with the specified interval (year|month|day)

  "?size-facet=<number>" # Specify the number of facets to return
  "?facet.variant=<variant>" # Available variants: object,tuple, term

Aggregations

  "?date-<interval>=<field>[<field1>:<field2>]" # Specify a temporal aggregation

  "?rangefacet-<field>=<interval>" # Range facet with interval

Formats

  "?format=raw" # Returns the raw elasticsearch response (application/json)

  "?format=geojson" # Return a GeoJSON featureCollection containing point features
  "?format=geojson&geometry=linestring" # Return a GeoJSON featureCollection containing a linestring feature
  "?format=geojson&geometry=multipoint" # Return a GeoJSON featureCollection containing a multipoint feature

  "?format=csv" # Return results as csv (Only basic support)
  "?format=csv&fields=<field1>" # For the best results with csv specify the fields you want in the results
  "?format=csv&fields=<alias>:<field>" # Header fields can be renamed with an alias

Installation

Add this line to your application's Gemfile:

gem 'icelastic', :git => "git://github.com/npolar/icelastic.git"

And then execute:

$ bundle

Rangefaceting requires copying scripts/range.groovy to your elasticsearch scripts folder, ie. elasticsearch/config/scripts/range.groovy. See also: Elasticsearch Scripting

Testing

The tests need a real Elasticsearch server to run

  export ICELASTIC_ELASTICSEARCH_COMMAND=tmp/elasticsearch/bin/elasticsearch
  mkdir -p tmp/elasticsearch && wget -O - https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-1.3.4.tar.gz | tar xz --directory=tmp/elasticsearch/ --strip-components=1

Run tests

  bundle exec rspec

Contributing

1. Fork it
2. Create your feature branch (git checkout -b my-new-feature)
3. Commit your changes (git commit -am 'Add some feature')
4. Push to the branch (git push origin my-new-feature)
5. Create new Pull Request

rubydoc