LMC Variational Strategy vs LMC Kernel; trying to understand mean functions.

[fjaraavilaa]

Hello. I am a new user and I am wondering one specific thing. I am not sure if I am, even theoretically, thinking things wrong.
I have a deterministic function for a Multi-Task mean.

import gpytorch
import torch
from mean_functions.deterministic_function import Multi_Task_Deterministic_Function


class Multi_Task_GP_Deterministic(gpytorch.models.ExactGP):
    def __init__(self, train_x, train_y, inducing_points, num_tasks, latent_rank, likelihood):
        super(Sparse_Multi_Task_GP_Power_Curve, self).__init__(train_x, train_y, likelihood)
        self.mean_module = Multi_Task_Deterministic_Function()
        self.base_covar_module = gpytorch.kernels.LCMKernel([gpytorch.kernels.RBFKernel() for _ in range(num_latents)], 
                                                            num_tasks = 44, rank = latent_rank)
        self.covar_module = gpytorch.kernels.InducingPointKernel(self.base_covar_module, 
                                                                 inducing_points, likelihood=likelihood)
    def forward(self, x):
        mean_x = self.mean_module(x)
        covar_x = self.covar_module(x)
        return gpytorch.distributions.MultitaskMultivariateNormal(mean_x, covar_x)

This deterministic function is per task and it is sort of dependent on the task index. Nonetheless, I am not sure if I use a variational strategy how to proceed, given that the example here: https://docs.gpytorch.ai/en/v1.12/examples/04_Variational_and_Approximate_GPs/SVGP_Multitask_GP_Regression.html uses pre-made functions. I would also like to do something like this: http://bayesiandeeplearning.org/2019/papers/27.pdf . I am not sure how feasible it ends up being.

Thanks!

[fjaraavilaa]

I see someone tried to already solve this problem:
https://github.com/QWERTY6191/projected-lmc

[fjaraavilaa]

I was also trying to find a solution via the examples with DenseNet and the example seems quite non-reproducible on a colab notebook and locally. I tried replicating it with some DenseNets I found online but they were incompatible