Regularly poll executors to track their utilization#613

Open

squito wants to merge 12 commits intomesos:masterfrom squito:executor_info

+187-7

Commits on May 15, 2013

add simple ExecutorStatus
Imran Rashid
committed
plumb ExecutorStatus through system. (still missing polling)
Imran Rashid
committed
regularly poll the executors for their status (just standalone cluster for now)
Imran Rashid
committed
add StageStarted event
Imran Rashid
committed
StatsReportListener summarizes executor utilization at end of every stage
Imran Rashid
committed
setup executor status polling in local scheduler
Imran Rashid
committed
stage started events when waiting stages start
Imran Rashid
committed
b/c of initialization order, dag scheduler may be null
Imran Rashid
committed

Commits on Jun 20, 2013

change ExecutorStatusPoller to use a concurrent executor with a fixed rate scheduler
Imran Rashid
committed
combine allExecutors and totalCoreCount
Imran Rashid
committed
shutdown the statusPoller
Imran Rashid
committed
rename fields in ExecutorActivitySummary to make it more obvious that it aggregates many samples
Imran Rashid
committed