Define a algorithm-specific sample() method

[zoctipus]

Hi thanks for such a great repository,

I have a proposal for interfacing algorithms with a generic sample() method
some algorithms like ppo, rpo, a2c, trpo, uses memory.sample_all() with mini_batch as arguments
some other algorithms like sac, td3. dqn, ddpg, uses memory.sample() with batch_size as arguments

if I want to incorporates some new loss, new procedure that requires calculating memory samples in my training application, and wants to test it against all available algorithms (this very reason why I like skrl), it will be very flexible if I can just write a patch that modifies post_interation() to Agent class, so that all algorithms all inherits this new features. While solution like #171 will work, but that requires adding new loss term in each possible algorithms.

but that requires a uniform sample method across all specific algorithms, if some require sample_all with mini_batch variable and others requires sample with batch variable it is not as so easy to work with.

I think this suggestion can greatly enhance workflow that requires making edit your code base with creative algorithm designing, testing efficiently with existing algorithm.

Let me know what is your thought on this! If you like the idea, I can take extra caution when writing the code and open a pull request to help!

Thanks for your effort!

[Toni-SM]

Hi @zoctipus

Your proposal makes perfect sense.
I think the way to go is to create a new memory class (e.g.: SequentialMemory) that allows for sequential/consecutive memory sampling (via the .sample() method) and use this memory (and replace .sample_all()) in on-policy algorithm like PPO, RPO, A2C and TRPO...

If you feel like it, you can open a PR.
If you have another implementation idea for this case, please feel free to discuss here :)