Should predicted observations be used for computing intrinsic value term in the likelihood AIF?

[sai-prasanna]

I noticed that you use preferred observations for computing the intrinsic value term in the likelihood AIF. But from what I understand the preferred observations should be used only for the extrinsic value term.

contrastive-aif/agents.py

Line 326 in 980e386

_, posterior_states = self.wm.posterior(obs_embed=self.wm.obs_encoder(preferred_obs).expand(batch_b*batch_t, self.wm.obs_encoder.embed_size), prev_action=None, prev_state=init_states, is_init=True)

[mazpie]

Hi @sai-prasanna,

thanks for noting this bug in the public version of the code! I checked the original repo, and the code looks like this:

# compute intrinsic value
embed = self.obs_encoder(predicted_obs)
_, posterior_states = self.posterior(embed, actions, prior_states, is_init=True)

I currently have no time to re-test this version with the change myself, so I may do it later in the future.
If you are currently working with the repo, it would be great if you could test it and contribute the fix yourself!