Calculation of Rotation Matrix is a bit misleading...

[kadir-gunel]

function mapOrthogonal(X::T, Y::T; λ::Float32=Float32(1)) where {T}
    F = CUDA.CUBLAS.svd(X * Y')
    W = permutedims(F.U * F.Vt * cuinv((X * X') + λ .* CuMatrix{Float32}(I, 300, 300)))
    return W, F.S
end

The above code snippet is kind of misleading. Thinking in theory lead me to think that the matrix multiplication inside the decomposition operation X * Y' is wrong. Because what we do is that we rotate the source space towards the target space where as this operation leads to opposite direction!

We should calculate the rotation operation based on X and not Y!

To do so :

1. Just transpose the operation : X * Y' to Y * X'.
2. Probably, we need to shift the Vt and U operations.
3. Debug for verification.

[kadir-gunel]

Probably the correct operation would be as follows :

function mapOrthogonal(X::T, Y::T; λ::Float32=Float32(1)) where {T}
    F = CUDA.CUBLAS.svd(Y * X')
    W = permutedims(F.U' * F.V * cuinv((X * X') + λ .* CuMatrix{Float32}(I, 300, 300)))
    return W, F.S
end

[kadir-gunel]

I have changed the code as the following :

function mapOrthogonal(X::CuMatrix, Y::CuMatrix)
    d, n = size(X)

    XY = CuMatrix{Float32}(undef, d, d)
    W = CuMatrix{Float32}(undef, d, d)

    gemm!('N', 'T', cu(Float32(1)), X, Y, cu(Float32(0)), XY)

    F = svd(XY)

    gemm!('N', 'N', cu(Float32(1)), F.U, F.Vt, cu(Float32(0)), W)

    return permutedims(W)
end

I directly call the cuBLAS methods, additionally there is no regularization.