请问有哪些将LM或LLM与联邦学习相结合的工作？

[youngfish42]

求推荐相关的论文列表，清单和代码，欢迎讨论

也欢迎分享和介绍相关的优秀论文和项目~

[youngfish42]

@youngfish42 曾经在24年1月做过一项调研，如下是查到的相关论文清单。先抛砖引玉一下~

• Che, Tianshi, Ji Liu, Yang Zhou, Jiaxiang Ren, Jiwen Zhou, Victor Sheng, Huaiyu Dai, and Dejing Dou. “Federated Learning of Large Language Models with Parameter-Efficient Prompt Tuning and Adaptive Optimization.” In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, edited by Houda Bouamor, Juan Pino, and Kalika Bali, 7871–88. Singapore: Association for Computational Linguistics, 2023. https://aclanthology.org/2023.emnlp-main.488.
• Chen, Chaochao, Xiaohua Feng, Jun Zhou, Jianwei Yin, and Xiaolin Zheng. “Federated Large Language Model: A Position Paper.” arXiv, July 17, 2023. https://doi.org/10.48550/arXiv.2307.08925.
• Collins, Liam, Shanshan Wu, Sewoong Oh, and Khe Chai Sim. “Profit: Benchmarking Personalization and Robustness Trade-off in Federated Prompt Tuning.” arXiv, October 6, 2023. https://doi.org/10.48550/arXiv.2310.04627.
• “Profit: Benchmarking Personalization and Robustness Trade-off in Federated Prompt Tuning,” 2023. https://openreview.net/forum?id=5JsO2DClwk.
• Dong, Chenhe, Yuexiang Xie, Bolin Ding, Ying Shen, and Yaliang Li. “Tunable Soft Prompts Are Messengers in Federated Learning.” In Findings of the Association for Computational Linguistics: EMNLP 2023, edited by Houda Bouamor, Juan Pino, and Kalika Bali, 14665–75. Singapore: Association for Computational Linguistics, 2023. https://aclanthology.org/2023.findings-emnlp.976.
• Han, Shanshan, Baturalp Buyukates, Zijian Hu, Han Jin, Weizhao Jin, Lichao Sun, Xiaoyang Wang, et al. “FedMLSecurity: A Benchmark for Attacks and Defenses in Federated Learning and Federated LLMs.” arXiv, October 6, 2023. https://doi.org/10.48550/arXiv.2306.04959.
• Hilmkil, Agrin, Sebastian Callh, Matteo Barbieri, Leon René Sütfeld, Edvin Listo Zec, and Olof Mogren. “Scaling Federated Learning for Fine-Tuning of Large Language Models.” In Natural Language Processing and Information Systems, edited by Elisabeth Métais, Farid Meziane, Helmut Horacek, and Epaminondas Kapetanios, 15–23. Lecture Notes in Computer Science. Cham: Springer International Publishing, 2021. https://doi.org/10.1007/978-3-030-80599-9_2.
• Hou, Charlie, Hongyuan Zhan, Akshat Shrivastava, Sid Wang, Aleksandr Livshits, Giulia Fanti, and Daniel Lazar. “Privately Customizing Prefinetuning to Better Match User Data in Federated Learning.” arXiv, February 22, 2023. https://doi.org/10.48550/arXiv.2302.09042.
• Jiang, Jingang, Xiangyang Liu, and Chenyou Fan. “Low-Parameter Federated Learning with Large Language Models.” arXiv, July 25, 2023. https://doi.org/10.48550/arXiv.2307.13896.
• Kandpal, Nikhil, Krishna Pillutla, Alina Oprea, Peter Kairouz, Christopher A. Choquette-Choo, and Zheng Xu. “User Inference Attacks on Large Language Models,” 2023. https://openreview.net/forum?id=4uyyLG4KCH.
• Kuang, Weirui, Bingchen Qian, Zitao Li, Daoyuan Chen, Dawei Gao, Xuchen Pan, Yuexiang Xie, Yaliang Li, Bolin Ding, and Jingren Zhou. “FederatedScope-LLM: A Comprehensive Package for Fine-Tuning Large Language Models in Federated Learning.” arXiv, September 1, 2023. https://doi.org/10.48550/arXiv.2309.00363.
• Li, Jianwei, Sheng Liu, and Qi Lei. “Beyond Gradient and Priors in Privacy Attacks: Leveraging Pooler Layer Inputs of Language Models in Federated Learning,” 2023. https://openreview.net/forum?id=H0inHCV05c.
• Liu, Xiangyang, Tianqi Pang, and Chenyou Fan. “Federated Prompting and Chain-of-Thought Reasoning for Improving LLMs Answering.” In Knowledge Science, Engineering and Management, edited by Zhi Jin, Yuncheng Jiang, Robert Andrei Buchmann, Yaxin Bi, Ana-Maria Ghiran, and Wenjun Ma, 3–11. Lecture Notes in Computer Science. Cham: Springer Nature Switzerland, 2023. https://doi.org/10.1007/978-3-031-40292-0_1.
• Recasens, Pol G., Jordi Torres, Josep Lluis Berral, Søren Hauberg, and Pablo Moreno-Muñoz. “Beyond Parameter Averaging in Model Aggregation,” 2023. https://openreview.net/forum?id=sPtEDSVD4K.
• Shen, Yifei, Jiawei Shao, Xinjie Zhang, Zehong Lin, Hao Pan, Dongsheng Li, Jun Zhang, and Khaled B. Letaief. “Large Language Models Empowered Autonomous Edge AI for Connected Intelligence.” arXiv, July 6, 2023. https://doi.org/10.48550/arXiv.2307.02779.
• Sun, Jingwei, Ziyue Xu, Hongxu Yin, Dong Yang, Daguang Xu, Yiran Chen, and Holger R. Roth. “FedBPT: Efficient Federated Black-Box Prompt Tuning for Large Language Models.” arXiv, October 2, 2023. https://doi.org/10.48550/arXiv.2310.01467.
• Wang, Boxin, Yibo Jacky Zhang, Yuan Cao, Bo Li, H. Brendan McMahan, Sewoong Oh, Zheng Xu, and Manzil Zaheer. “Can Public Large Language Models Help Private Cross-Device Federated Learning?” arXiv, May 20, 2023. https://doi.org/10.48550/arXiv.2305.12132.
• Woisetschläger, Herbert, Alexander Isenko, Shiqiang Wang, Ruben Mayer, and Hans-Arno Jacobsen. “Federated Fine-Tuning of LLMs on the Very Edge: The Good, the Bad, the Ugly.” arXiv, October 4, 2023. https://doi.org/10.48550/arXiv.2310.03150.
• Xing, Pengwei, Songtao Lu, and Han Yu. “FedLogic: Interpretable Federated Multi-Domain Chain-of-Thought Prompt Selection for Large Language Models.” arXiv, August 29, 2023. https://doi.org/10.48550/arXiv.2308.15324.
• Xu, Mengwei, Yaozong Wu, Dongqi Cai, Xiang Li, and Shangguang Wang. “Federated Fine-Tuning of Billion-Sized Language Models across Mobile Devices.” arXiv, August 26, 2023. https://doi.org/10.48550/arXiv.2308.13894.
• Yu, Sixing, J. Pablo Muñoz, and Ali Jannesari. “Federated Foundation Models: Privacy-Preserving and Collaborative Learning for Large Models.” arXiv, May 18, 2023. https://doi.org/10.48550/arXiv.2305.11414.
• Yue, Linan, Qi Liu, Yichao Du, Weibo Gao, Ye Liu, and Fangzhou Yao. “FedJudge: Federated Legal Large Language Model.” arXiv, September 15, 2023. https://doi.org/10.48550/arXiv.2309.08173.
• Zhang, Jianyi, Saeed Vahidian, Martin Kuo, Chunyuan Li, Ruiyi Zhang, Guoyin Wang, and Yiran Chen. “Towards Building the Federated GPT: Federated Instruction Tuning.” arXiv, May 9, 2023. https://doi.org/10.48550/arXiv.2305.05644.
• Zhang, Jianyi, Saeed Vahidian, Martin Kuo, Chunyuan Li, Ruiyi Zhang, Tong Yu, Guoyin Wang, and Yiran Chen. “Towards Building the FederatedGPT: Federated Instruction Tuning,” 2023. https://openreview.net/forum?id=TaDiklyVps.
• Zhang, Liang, Kiran Koshy Thekumparampil, Sewoong Oh, and Niao He. “DPZero: Dimension-Independent and Differentially Private Zeroth-Order Optimization,” 2023. https://openreview.net/forum?id=s7hquGszME.
• Zhao, Jun. “Privacy-Preserving Fine-Tuning of Artificial Intelligence (AI) Foundation Models with Federated Learning, Differential Privacy, Offsite Tuning, and Parameter-Efficient Fine-Tuning (PEFT).” TechRxiv, October 2, 2023. https://doi.org/10.36227/techrxiv.24191886.v1.
• Zhao, Wanru, Yihong Chen, Royson Lee, Xinchi Qiu, Yan Gao, Hongxiang Fan, and Nicholas Donald Lane. “Breaking Physical and Linguistic Borders: Multilingual Federated Prompt Tuning for Low-Resource Languages,” 2023. https://openreview.net/forum?id=HyRwexERAo.

其中，有代码的论文如下

• Tunable Soft Prompts are Messengers in Federated Learning https://github.com/alibaba/FederatedScope/tree/fedsp/federatedscope/nlp/fedsp
• Federated Learning of Large Language Models with Parameter-Efficient Prompt Tuning and Adaptive Optimization https://github.com/llm-eff/FedPepTAO
• Towards Building the Federated GPT: Federated Instruction Tuning https://github.com/JayZhang42/FederatedGPT-Shepherd
• FederatedScope-LLM: A Comprehensive Package for Fine-tuning Large Language Models in Federated Learning https://github.com/alibaba/FederatedScope/tree/llm

[youngfish42]

另附上海交通大学团队 @rui-ye 的工作，如下是他们的宣传语

"OpenFedLLM: Training Large Language Models on Decentralized Private Data via Federated Learning"。我们在这个工作里提供了一个联邦学习（FL）训练大语言模型（LLM）的代码框架OpenFedLLM（代码简单，科研友好），并基于这个框架进行了一系列实验研究（单张3090八小时内可完成训练！），最后指出了一系列这个新场景下的新研究方向。

这个代码框架里有什么？

• 从FL的角度，我们实现了7种代表性的FL算法
• 从LLM的角度，我们实现了指令微调和价值对齐2个重要训练过程
• 从实验的角度，我们提供了超过8种训练数据集和超过30种评测指标

这个paper里有什么？

• 基础性的介绍，一系列未来的研究方向
• 联邦指令微调在通用、金融、医疗、代码以及混合场景下的性能展示。在金融场景下，可实现联邦训练的模型 > GPT-4 > 本地训练的模型，说明联邦训练的必要性
• 联邦价值对齐在有用性偏好和无害性偏好场景下的性能展示

文章链接：https://arxiv.org/abs/2402.06954
代码链接：https://github.com/rui-ye/OpenFedLLM

[yankang18]

我们去年写了一个survey叫做Grounding Foundation Models through Federated Transfer Learning: A General Framework，https://arxiv.org/abs/2311.17431。 从知识迁移的角度对LM和FL结合的工作梳理了下。

[youngfish42]

感谢分享，我去学习一下~

[yankang18]

你的分享挺好，有几篇，我都漏掉了。

[youngfish42]

不客气~ 我比较擅长查文献哈哈哈

[UmikaMoss]

关于联邦学习和大模型结合的轻量化以及PEFT微调方法有一篇survey，A Survey on Efficient Federated Learning Methods for Foundation Model Training，https://arxiv.org/abs/2401.04472

[youngfish42]

感谢分享~ （这个好像是在我上次调研后才放出来的论文，最近三个月应该还有蛮多新工作~）