local_tensors in aggregator function

[dskhanirfan]

In the aggregator function (e.g weighted_average_aggregation), what does local_tensors contain other than tensor_values and weight_values? and what is the difference in tensor_values and weight_values in local_tensors? How are the weight_values computed? Are the weight_values dependent on the dataset size of the collaborator?

[alexey-khorkin]

@dskhanirfan, Thanks for the interest in OpenFL.
LocalTensor contains next fields:

LocalTensor = namedtuple('LocalTensor', ['col_name', 'tensor', 'weight'])

So, it has:

• col_name - name of collaborator
• tensor - numpy array from collaborator, with model weights
• weight - this is float number, required to calculate weighted average tensor (aggregated)

How are the weight_values computed?

It happens here: https://github.com/intel/openfl/blob/031f0ce39e9cb96cbc205a163675f8ca213c5f7c/openfl/component/aggregator/aggregator.py#L756

        collaborator_weights_unnormalized = {
            c: self.collaborator_task_weight[TaskResultKey(task_name, c, self.round_number)]
            for c in collaborators_for_task}
        weight_total = sum(collaborator_weights_unnormalized.values())
        collaborator_weight_dict = {
            k: v / weight_total
            for k, v in collaborator_weights_unnormalized.items()
        }

and self.collaborator_task_weight contains dataset size (# samples) for each task. So, as you can see, we normalize collaborator weights according to the dataset size (on each collaborator).
@dskhanirfan, Is this the answer to your question?

[dskhanirfan]

@alexey-khorkin Is the dataset size for each participating collaborator available in the aggregator function in every federation round?

[alexey-khorkin]

@dskhanirfan, According to AggregationFunction call method: https://github.com/intel/openfl/blob/031f0ce39e9cb96cbc205a163675f8ca213c5f7c/openfl/component/aggregation_functions/interface.py#L54
We have local_tensors variable with normalized weights, so the dataset size is not available in the aggregator function.