Benchmarking results

[otcostsubmission]

Thanks a lot for the great repo. I wanted to know what your benchmark results looked like. I'm especially interested in Per-FedAvg (HF). I have found that performance is highly variable and wanted to know if its an implementation issue or you found the same thing?

[KarhouTam]

Hi, @otcostsubmission. First, thanks for your appreciation.

I observed the similar things from my experiment of Per-FedAvg. Seems Per-FedAvg is sensitive to hyperparameters and model backbone. I got the source code of Per-FedAvg from the author and can ensure that my implementation is correct.