deepspeed support

[WentseChen]

Description

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)

Checklist

• I have ensured make test pass (required).
• I have checked the code using make format (required).

[codecov]

Codecov Report

Attention: 216 lines in your changes are missing coverage. Please review.

Comparison is base (38da73a) 74.49% compared to head (fc02030) 73.38%.
Report is 3 commits behind head on main.

Files	Patch %	Lines
openrl/modules/networks/policy_network_gpt.py	0.00%	112 Missing ⚠️
openrl/modules/networks/value_network_gpt.py	0.00%	72 Missing ⚠️
openrl/envs/nlp/rewards/intent.py	50.00%	10 Missing ⚠️
openrl/envs/nlp/rewards/kl_penalty.py	68.96%	9 Missing ⚠️
openrl/utils/logger.py	18.18%	9 Missing ⚠️
openrl/algorithms/ppo.py	55.55%	4 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #282      +/-   ##
==========================================
- Coverage   74.49%   73.38%   -1.12%     
==========================================
  Files         252      254       +2     
  Lines       13035    13256     +221     
==========================================
+ Hits         9711     9728      +17     
- Misses       3324     3528     +204     

Flag	Coverage Δ
unittests	73.38% <16.92%> (-1.12%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.