Contributing sources

Glad you’d like to add another tentacle source to Picky!
(to use for your index)

How to do it

1. Check the available Sources if the source already exists.
2. If yes, use (and improve) it :)
3. If not, fork the repository and follow the instructions below.
4. Add a new load statement to lib/picky/loader.rb

Every source derives from the base class Sources::Base, which contains 3 methods. These methods are all called by the indexer:

• connect_backend # Connect to the backend here if your source needs that.
• take_snapshot(type) # If you have fast moving data, you might want to get a snapshot here (into a hash, or a temp table etc.)
• harvest(type, category) # This is the mandatory method. Yield each id and data for that id.

By default, the methods are empty. So if you don’t need to connect to a backend, just don’t override the method.

Example

This is how the CSV source implements the harvest method. It’s currently not optimized.

  def harvest _, category # We don't care about the type, just about the category.
    index = field_names.index category.name # Get the right column from the data.
    ::CSV.foreach file_name do |ary| # A helper method that reads a line from the csv file.
      indexed_id = ary.shift
      text = ary[index]
      next unless text
      yield indexed_id, text # <- the most important part, yields the indexed_id and the corresponding text to the indexer. 
    end
  end