Skip to content

Commit

Permalink
Update QR-DQN optimizer to only use q_net parameters (#252)
Browse files Browse the repository at this point in the history
* Updated QR-DQN optimizer input to only include quantile_net parameters

* Fix QR-DQN paper link in docs and update changelog
  • Loading branch information
corentinlger authored Jul 6, 2024
1 parent dc25cc6 commit 5c81398
Show file tree
Hide file tree
Showing 3 changed files with 5 additions and 3 deletions.
4 changes: 3 additions & 1 deletion docs/misc/changelog.rst
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@ New Features:

Bug Fixes:
^^^^^^^^^^
- Updated QR-DQN optimizer input to only include quantile_net parameters (@corentinlger)
- Updated QR-DQN paper link in docs (@corentinlger)

Deprecations:
^^^^^^^^^^^^^
Expand Down Expand Up @@ -580,4 +582,4 @@ Contributors:
-------------

@ku2482 @guyk1971 @minhlong94 @ayeright @kronion @glmcdona @cyprienc @sgillen @Gregwar @rnederstigt @qgallouedec
@mlodel @CppMaster @burakdmb @honglu2875 @ZikangXiong @AlexPasqua @jonasreiher @icheered @Armandpl
@mlodel @CppMaster @burakdmb @honglu2875 @ZikangXiong @AlexPasqua @jonasreiher @icheered @Armandpl @corentinlger
2 changes: 1 addition & 1 deletion docs/modules/qrdqn.rst
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ instead of predicting the mean return (DQN).
Notes
-----

- Original paper: https://arxiv.org/abs/1710.100442
- Original paper: https://arxiv.org/abs/1710.10044
- Distributional RL (C51): https://arxiv.org/abs/1707.06887
- Further reference: https://github.com/amy12xx/ml_notes_and_reports/blob/master/distributional_rl/QRDQN.pdf

Expand Down
2 changes: 1 addition & 1 deletion sb3_contrib/qrdqn/policies.py
Original file line number Diff line number Diff line change
Expand Up @@ -171,7 +171,7 @@ def _build(self, lr_schedule: Schedule) -> None:

# Setup optimizer with initial learning rate
self.optimizer = self.optimizer_class( # type: ignore[call-arg]
self.parameters(),
self.quantile_net.parameters(),
lr=lr_schedule(1),
**self.optimizer_kwargs,
)
Expand Down

0 comments on commit 5c81398

Please sign in to comment.