Calling Scalding from inside your application

Feel free to add this doc.

See the code here:

We've talked about making it easier to weave scalding into frameworks, but that will mostly be better docs.

Summingbird does what you want to do. See here:

The basic approach is this:

You create a FlowDef (mutable state of the job DAG) and Mode (wraps the config). Those should be either made implicit or passed explicitly to any read/write calls.

After you have prepared the FlowDef (scalding is just mutating that state). Finally call:

mode.newFlowConnector(config).connect(flowDef).complete

(taken from Job.scala, buildFlow/run).

And you will have run this FlowDef run for this Mode.