GitHub - kevinpet/jobcontrol: Fork of Hadoop JobControl classes, working on some changes I'd like to see

kevinpet / jobcontrol Public

Notifications You must be signed in to change notification settings
Fork 1
Star 1

Fork of Hadoop JobControl classes, working on some changes I'd like to see

Notifications

Name		Name	Last commit message	Last commit date
Latest commit History 8 Commits
src/kdp		src/kdp
.gitignore		.gitignore
README		README
build.xml		build.xml

Repository files navigation

JobControl fork

Extending Hadoop JobControl to handle more complex tasks.

Hadoop Enhanced JobControl
==========================

Extend Hadoop JobControl to handle more complex tasks.

- Allow file system operations (move, delete) as dependencies.
- Restart failed workflows.
- Allow composition of workflows (i.e a workflow as a dependency).
- Expose a well-defined extension point for adding Pig, Cascading, etc.
- Simplify job creation using a Scala-based DSL.

Examples
--------

Sample project (three stage extraction of a representative keyword
from a piece of text) to use JobControl.