Feature Loss not decreasing

[RAJA-PARIKSHAT]

I am training on CLIP features in a different scene, but the feature loss is not decreasing. I checked all the gradients, is anybody facing such an issue?

[ywyue]

Hi @RAJA-PARIKSHAT, which scene are you using? Did you use the feature rendering code in this codebase? Is the color loss decreasing?

[RAJA-PARIKSHAT]

I am training on human's data. The number of cameras are 100+, what I have seen with 2-3 cameras the network is able to overfit but if the number of views are increased, features are not getting learned correctly. Initializing the set learning rate was too low, so overfitting on 2 views was also taking long, but I corrected the learning rate.

[ywyue]

Hi @RAJA-PARIKSHAT, sorry for the late response. Our pipeline should be applicable to general multi-view dataset (either object/human level or scene-level). I have no specific idea about your issue because I didn't work on your dataset before. Did you use both color loss and feature loss? In my experience, only using feature loss will not work. The color loss is essential for the Gaussian to learn correct geometry in the meantime.