Skip to content

Jobs

Jump to bottom
Adam Wead edited this page May 10, 2017 · 7 revisions

Jobs, jobs, jobs, jobs! Who wants a job?

ActiveJob

Based on an existing framework in Rails

http://edgeguides.rubyonrails.org/active_job_basics.html

Classes that do one thing: perform

class GoodJob < ActiveJob::Base

  def perform
    # do your job
  end

end

Let's Do Our Job

git fetch
git checkout day3/jobs/start 
bundle install

But first, let's make sure:

ps -ax | grep spring | awk '{ print $1 }' | xargs kill -s KILL
killall fsevent_watch

Is fits giving you the fits?

which fits.sh

If the above is not returning anything, you need to edit config.fits_path in config/initializers/hyrax.rb

We need to start with a blank slate. Make sure your Rails server is stopped.

bundle exec rake dev:clean

In a new terminal from the your ahc root:

redis-cli

Then:

> keys *
(empty list or set)

And leave it open. Meanwhile... in another terminal window far, far away:

bundle exec resque-pool -E development

Now, go back to your redis-cli terminal and list the keys again. Resque has created the queues in Redis.

Restart your rails application. Once that's done, open two browser tabs:

Create a new work. You'll need to re-register a user account. BUT WAIT THERE'S MORE! For a limited time you can see the job queue process occur before your very eyes. Here's what to do:

  • enter the metadata
  • upload a file
  • DON'T CLICK SAVE!
  • check the agreement
  • wait....
  • are you ready?
  • Ok, click Save, but then IMMEDIATELY go to the resque web tab
  • hit refresh
  • keep hitting refresh

Ok, so what? Hopefully, you should have a new work with a thumbnail and you will have seen the jobs run through their queues in resque web.

Now checkout redis:

> keys *

Wut? Redis queues the jobs and resque reads them off, performing each one.

Take This Job and Shove It

How do we override jobs? When creating a new work, Hyrax actors kick off some jobs:

  1. https://github.com/projecthydra-labs/hyrax/blob/master/app/actors/hyrax/actors/create_with_files_actor.rb#L42
  2. https://github.com/projecthydra-labs/hyrax/blob/master/app/jobs/attach_files_to_work_job.rb
  3. https://github.com/projecthydra-labs/hyrax/blob/master/app/actors/hyrax/actors/file_set_actor.rb#L36
  4. https://github.com/projecthydra-labs/hyrax/blob/master/app/actors/hyrax/actors/file_actor.rb#L30
  5. https://github.com/projecthydra-labs/hyrax/blob/master/app/jobs/ingest_file_job.rb
  6. https://github.com/projecthydra-labs/hyrax/blob/master/app/jobs/characterize_job.rb
  7. https://github.com/projecthydra-labs/hyrax/blob/master/app/jobs/create_derivatives_job.rb

I want a new characterize job.... but first, we must test.

mkdir app/jobs
mkdir spec/jobs
touch spec/jobs/characterize_job_spec.rb

Write a test: https://github.com/awead/hydra-rspec-templates/blob/master/spec/jobs/sample_job_spec.rb

Write a job:

touch app/jobs/characterize_job_spec.rb

You're overriding https://github.com/projecthydra-labs/hyrax/blob/master/app/jobs/characterize_job.rb

class CharacterizeJob < ActiveJob::Base
  queue_as Hyrax.config.ingest_queue_name

  # @param [FileSet] file_set
  # @param [String] file_id identifier for a Hydra::PCDM::File
  # @param [String, NilClass] filepath the cached file within the Hyrax.config.working_path
  # @return my resignation
  def perform(file_set, file_id, filepath = nil)
    raise StandardError, "I quit."
  end
end

Restart resque-pool and Rails. Create a new work and watch the queue in resque web.

Backends

ActiveJob supports different backends so you can choose accordingly. In Rails 4, the default is inline but in Rails 5 it is now async which mimics a production environment. You can read more about it here

However, as of this writing there is still a problem with reporting errors, see https://github.com/rails/rails/issues/26848

TL;DR Use Rails 5. In development mode, use async but know that errors will not be surfaced!

Clone this wiki locally