job reporting

The JobReport API

Easy Batch records several metrics during batch processing and provides a complete report at the end of execution. This report is an instance of the JobReport class and contains the following information:

• The job start and end times
• The job status
• The number of read records
• The number of written records
• The number of filtered records
• The number of errors

Contributing custom metrics

It is possible to contribute custom metrics by adding them to JobMetrics with addMetric method. In order to get access to JobMetrics, you need to implement the JobListener interface. The following example is a listener that calculates record processing time average and add it to the job report as a custom metric:

public class RecordProcessingTimeCalculator implements PipelineListener, JobListener {

    private long startTime;
    private long nbRecords;
    private long recordProcessingTimesSum;

    @Override
    public Record beforeRecordProcessing(Record record) {
        nbRecords++;
        startTime = System.currentTimeMillis();
        return record;
    }

    @Override
    public void afterRecordProcessing(Record input, Record output) {
        recordProcessingTimesSum +=  System.currentTimeMillis() - startTime;
    }

    @Override
    public void onRecordProcessingException(Record record, Throwable throwable) {
        recordProcessingTimesSum += System.currentTimeMillis() - startTime;
    }

    @Override
    public void beforeJobStart(JobParameters jobParameters) {
        // no op
    }

    @Override
    public void afterJobEnd(JobReport jobReport) {
        jobReport.getMetrics().addMetric(
            "Record processing time average (in ms)",
             (double)recordProcessingTimesSum / (double)nbRecords);
    }
}

Format a job report

You can format reports using the JobReportFormatter interface.

The HtmlJobReportFormatter implements this interface to format reports in HTML:

String htmlReport = new HtmlJobReportFormatter().formatReport(report);

The following is an example of HTML report:

images/report.png

To use the HtmlJobReportFormatter, you should add the easybatch-tools module to your dependencies:

<dependency>
    <groupId>org.easybatch</groupId>
    <artifactId>easybatch-tools</artifactId>
    <version>{revnumber}</version>
</dependency>

You can also provide your own implementation of the JobReportFormatter interface to format reports to custom formats.

Merge job reports

When you run multiple jobs to process a data source in parallel, each job will generate a partial report for the data partition it has processed. You may want to merge partial reports into a consolidated one. This is where the JobReportMerger comes to the rescue:

The merged report is defined as follows:

• The start time is the minimum of start times in partial reports
• The end time is the maximum of end times in partial reports
• The total read records is the sum of total read records in partial reports
• The total written records is the sum of total written records in partial reports
• The total filtered records is the sum of total filtered records in partial reports
• The total error records is the sum of total error records in partial reports
• The final status is COMPLETED (if all partials are completed) or FAILED (if one of partials has failed).
• The final name is the concatenation of partial job names.

To use the job report merger, first you need to add the easybatch-tools module to your dependencies:

<dependency>
    <groupId>org.easybatch</groupId>
    <artifactId>easybatch-tools</artifactId>
    <version>{revnumber}</version>
</dependency>

Then you can use the following snippet to get a merged report:

JobReportMerger reportMerger = new DefaultJobReportMerger();
JobReport finalReport = reportMerger.mergeReports(report1, report2);