Migrate from RAFT to CUVS

[tarang-jain]

Remove the dependency on raft::compiled and modify GPU implementations to use cuVS backend in place of RAFT.

A deeper insight into the dependency:
FAISS gets the ANN algorithm implementations such as IVF-Flat and IVF-PQ from cuVS. RAFT is meant to be a lightweight C++ header-only template library that cuVS relies on for the more fundamental / low-level utilities. Some examples of these are RAFT's device mdarray and mdspan objects; the RAFT resource object (raft::resource) that takes care of the stream ordering of device functions; linear algebra functions such as mapping, reduction, BLAS routines etc. A lot of the cuVS functions take the RAFT mdspan objects as arguments (for example raft::device_matrix_view). Therefore FAISS relies on both cuVS and RAFT. FAISS gets RAFT headers through cuVS and uses them to create the function arguments that can be consumed by cuVS. Note that we are not explicitly linking FAISS against raft::raft or raft::compiled. Only the required headers are included and compiled rather than compiling the whole RAFT shared library. This is the reason we still see mentions of raft in FAISS.

[tarang-jain]

@asadoughi I was able to resolve the seg fault. It was related to some bugs in the cuvs parts of bfKnn.

[mfoerste4]

@tarang-jain , as discussed offline I added the changes required to get rid of additional overhead and achieve comparable performance of faiss+cuvs vs plain cuvs.

[facebook-github-bot]

@asadoughi has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

[asadoughi]

Is RAFT still needed for compilation or we need to install rmm explicitly?

https://github.com/facebookresearch/faiss/actions/runs/10889999572/job/30218385769?pr=3549

The link interface of target "cuvs::cuvs" contains:

    rmm::rmm

  but the target was not found.  Possible reasons include:

[tarang-jain]

@asadoughi I tried linking raft::raft but now I see another failure in ci https://github.com/facebookresearch/faiss/actions/runs/10908718713/job/30275337629?pr=3549

[tarang-jain]

rmm and raft should ideally be available through cuvs. But I have also tried explicitly linking raft::raft.

[asadoughi]

It looks like one test is still failing TestTorchUtilsGPU.test_train_add_with_ids. Have you had a chance to troubleshoot?

[tarang-jain]

@asadoughi yes I have been looking into it. It is quite strange. The following order of operations gives erroneous results:
create a GPU torch tensor --> train GPU IVF-Flat Index --> search the index --> reset the index --> train again with the same tensor but this time have the data on CPU --> train the GPU IVF-Flat index --> search the index.

However, if both the tensors are on the same device (either CPU or GPU), there is no failure and it works fine.