knn-troch

Code works as you run knn.py, pre-commit / code coverage / pytest not properly configured yet

Example

# Testing knn-torch
torch.device("cuda" if torch.cuda.is_available() else "cpu")
knn = KNN(noisy=True, n_train=10000, n_test=2000, plotting_reso=200, plot_flag=False)
knn._plot_generate_grid_hsv(knn.plotting_reso, 3)
time = datetime.now()
for i in range(10, 110, 10):
    acc = knn.classify_and_evaluate(k=i)
    print(f"for k = {i}, acc: {acc}")
t_torch = datetime.now() - time
print("Time taken to run 10 trials with knn-torch: ", t_torch)

# Testing sk-learn's knn
from sklearn.neighbors import KNeighborsClassifier
time = datetime.now()
for i in range(10, 110, 10):
    neigh = KNeighborsClassifier(n_neighbors=i)
    neigh.fit(knn.data_train['X'], knn.data_train['Y'])
    preds = neigh.predict(knn.data_test['X'])
    acc = (preds == knn.data_test['Y']).sum()/len(preds)
    print(f"for k = {i}, acc: {acc}")
t_sklearn = datetime.now() - time
print("Time taken to run 10 trials with sklearn: ", t_sklearn)

print("Speedup: ", t_sklearn/t_torch, " times")

# Rerun with plotting for visualization cases
torch.device("cuda" if torch.cuda.is_available() else "cpu")
knn = KNN(noisy=True, n_train=10000, n_test=2000, plotting_reso=200, plot_flag=True)
knn._plot_generate_grid_hsv(knn.plotting_reso, 3)
acc = knn.classify_and_evaluate(k=10)
print(f"for k = {i}, acc: {acc}")

Output

for k = 10, acc: 0.8619999885559082

...

for k = 210, acc: 0.8115000128746033

Time taken to run 20 trials with knn-torch: 0:00:00.034190

for k = 10, acc: 0.8619999885559082

...

for k = 210, acc: 0.8149999976158142

Time taken to run 20 trials with sklearn: 0:00:02.614075

Speedup: 76.45729745539631 times

Plots

Philosophy

Pre-compute the distances between all points within a distribution first with torch mat multiplications (significantly faster than numpy's loops), then choose the distances of how up to k-neighbours to use for voting.