Clarification on preprocessing direction

[DaeunGu]

I'm working with spectral data where each sample is a row and each column is an absorbance value at different wavenumbers.
I know that in some scalers in sklearn, data transformations are done along the columns by default.
I'm wondering how it works in the chemotools package for methods like MSC and SNV. Do these preprocess the data along rows or columns? And is there an option to change the direction of preprocessing if needed?

Thank you for your help.

[paucablop]

Hi @DaeunGu 😄

Both methods are performed row-wise, as they are intended to remove the physical variability in the spectra due to scatter. The SNV operates entirely at a row level, and no information about other spectra in is used to correct the target one. In the MSC you need to select a reference spectrum to make the correction, you can choose to use the mean/median spectrum from your training set as reference. In such case you would calculate the mean/median column wise, and then perform the transformation row-wise for each spectrum. If you would like to find more information about this, you can check this paper, it provides a good overview :)

https://www.sciencedirect.com/science/article/pii/S0165993609001629

I hope this helps 🚀

[DaeunGu]

Thank you for your kind explanation!

[paucablop]

Glad I could help :)