Scatter estimate upsample_and_fit_scatter_estimate scale factors should be saved and or written to file

[robbietuk]

Scatter estimation relies on the tails estimation via upsample_and_fit_scatter_estimate. Currently there is no easy methodology to determine if the scatter estimate tail estimates converge or are within the clamped ranges without inspection of the console log, e.g. by scrolling through the log and finding.

INFO: upsample_and_fit_scatter_estimate: Interpolating scatter estimate to size of emission data

INFO: upsample_and_fit_scatter_estimate: scale factors before thresholding:
{{0.309695, 0.28541, 0.300074, 0.302864, 0.313234, 0.31094, 0.314898, 0.308822, 0.304241, 0.299309, 0.291353, 0.297305,
0.292876, 0.290333, 0.294117, 0.294381, 0.295705, 0.29346, 0.291988, 0.294252, 0.294026, 0.297229, 0.295458, 0.296193, 0
.297299, 0.299582, 0.303539, 0.30171, 0.308032, 0.307231, 0.312167, 0.316669, 0.318669, 0.323363, 0.329573, 0.327603, 0.
333274, 0.333633, 0.34919, 0.351366, 0.357016, 0.370945, 0.367954, 0.35421, 0.35202, 0.340191, 0.346707}
}


INFO: upsample_and_fit_scatter_estimate: scale factors after thresholding:
{{0.309695, 0.28541, 0.300074, 0.302864, 0.313234, 0.31094, 0.314898, 0.308822, 0.304241, 0.299309, 0.291353, 0.297305,
0.292876, 0.290333, 0.294117, 0.294381, 0.295705, 0.29346, 0.291988, 0.294252, 0.294026, 0.297229, 0.295458, 0.296193, 0
.297299, 0.299582, 0.303539, 0.30171, 0.308032, 0.307231, 0.312167, 0.316669, 0.318669, 0.323363, 0.329573, 0.327603, 0.
333274, 0.333633, 0.34919, 0.351366, 0.357016, 0.370945, 0.367954, 0.35421, 0.35202, 0.340191, 0.346707}
}


INFO: upsample_and_fit_scatter_estimate: scale factors after filtering:
{{0.303876, 0.304381, 0.304559, 0.305302, 0.305178, 0.307868, 0.307758, 0.306114, 0.303838, 0.301258, 0.297748, 0.295648
, 0.294239, 0.293724, 0.294025, 0.293266, 0.293462, 0.29399, 0.294434, 0.294588, 0.294658, 0.295206, 0.296291, 0.297618,
 0.298716, 0.300259, 0.301941, 0.304223, 0.30699, 0.309717, 0.312549, 0.316529, 0.319325, 0.323046, 0.326112, 0.330758,
0.335429, 0.340236, 0.346147, 0.351911, 0.354902, 0.357529, 0.356243, 0.355578, 0.354105, 0.350642, 0.347607}
}

#1280 (comment) gives an example of how to use sed to extract the values.

This makes it hard to validate the scatter modelling of new data.

---

Instead, ScatterEstimation could retain each Array<2, float> scale_factors at each step, which can be access after the simulation. All relevant logic is here:

STIR/src/scatter_buildblock/upsample_and_fit_scatter_estimate.cxx

Lines 111 to 123 in b754fa7

	info("upsample_and_fit_scatter_estimate: Finding scale factors by sinogram", 3);
	scale_factors = get_scale_factors_per_sinogram(emission_proj_data, interpolated_scatter, weights_proj_data);
	info(boost::format("upsample_and_fit_scatter_estimate: scale factors before thresholding:\n%1%") % scale_factors, 2);
	threshold_lower(scale_factors.begin_all(), scale_factors.end_all(), min_scale_factor);
	threshold_upper(scale_factors.begin_all(), scale_factors.end_all(), max_scale_factor);
	info(boost::format("upsample_and_fit_scatter_estimate: scale factors after thresholding:\n%1%") % scale_factors, 2);
	VectorWithOffset<float> kernel(-static_cast<int>(half_filter_width), half_filter_width);
	kernel.fill(1.F / (2 * half_filter_width + 1));
	ArrayFilter1DUsingConvolution<float> lowpass_filter(kernel, BoundaryConditions::constant);
	std::for_each(scale_factors.begin(), scale_factors.end(), lowpass_filter);
	info(boost::format("upsample_and_fit_scatter_estimate: scale factors after filtering:\n%1%") % scale_factors, 2);

---

Object requirements:

• Store each scatter estimate iteration
• Store the scale_factors array at each step (before and after thresholding, after filtering) values
• Determine if thresholding was applied (compare values before and after thresholding?)
• Possible class names: ScaleFactorRecords?
• Probably store the min_scale_factor and max_scale_factor values

Questions:

• How to handle the case of min_scale_factor == max_scale_factor

Additional:

• Allow for saving of the data to a csv (or alternative data format)

[KrisThielemans]

This would be nice to have

• Store each scatter estimate iteration

what do you mean?

• Store the scale_factors array at each step (before and after thresholding, after filtering) values

agreed

• Determine if thresholding was applied (compare values before and after thresholding?)

a user can do that, I don't see why we would.

• Possible class names: ScaleFactorRecords?

Probably store the min_scale_factor and max_scale_factor values

they are already in the ScatterEstimation class. I am not so sure if it's practical/useful to have a separate "records" class concentrating on a the scaling only. (It'd need to store far more parameters anyway (masks, etc)).

This fails a bit into a much more general topic of keeping track of parameters and making a record of that. All ParsingObject classes have a record of all input parameters, so these can be queried (and it'd be possible to use the idea behind parameter_info() to extend something else that is more useful). That doesn't cover "output parameters" though. Might be worth thinking about those in a more general context.

Questions:

• How to handle the case of min_scale_factor == max_scale_factor

what's special about it? It'll just set them to that value. done.

[robbietuk]

A modification; storing each scatter iteration is already done as each iteration can be written to file. I mostly want access to the upsample_and_fit_scatter_estimate: scale factors, which can be done via some kind of record keeping class.