Advice for GP functional data analysis with spectral mixture kernels

[tloredo]

Hello, GPyTorch experts-

I've used GPyTorch in the past for GP regression of individual time series, using spectral mixture kernel(SMKs). My team got it working pretty quickly; thank you for making this straightforward with GPyTorch!

Now we'd like to do functional data analysis (FDA) with GPs, hopefully with SMKs. For FDA, we will be jointly analyzing many time series, considered as a collection of sample paths drawn from a common GP (right now just one GP, but in the future, in FDA regression settings, perhaps from a family of GPs that depend on covariates). I think I see how to do this with GPyTorch—I'd just add the (marginal) likelihoods for each time series, and optimize that joint likelihood function. (If that doesn't sound right, please advise!)

What's unclear to me in regard to doing this with SMKs is how to initialize the optimizer. We are analyzing data where the individual time series are fairly sparse (from ~10 to a few hundred points, perhaps up to ~1000 points), and, importantly, where they are irregularly and asychronously sampled. Were they sampled uniformly, I suppose one idea would be to adapt initialize_from_data_empspect, which uses the empirical spectrum. I could replace the spectrum with the sum (or average?) of the spectra of the various sample paths. (We'll have tens to thousands of sample paths in our applications.)

But we don't have uniformly, synchronously sampled data. For a single time series, we use initialize_from_data. But it's unclear to me exactly how the algorithm there was motivated, and so it's not obvious to me how to generalize it to the multiple sample path setting.

One idea is to use a Lomb-Scargle periodogram (LSP, a generalization of the periodogram, for irregularly sampled data) in a way similar to how initialize_from_data_empspect works, though that would introduce some overhead for computing the LSP for every time series.

We'd appreciate any suggestions or advice!