Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Google Colab Error with CIFAR-10 example: "ValueError: The number of threads must be between 1 and 2" #382

Open
AntonioMacaronio opened this issue Jul 17, 2024 · 1 comment

Comments

@AntonioMacaronio
Copy link

I'm trying to follow the example of FFCV on CIFAR-10 here, except I'm just copy-pasting the code onto google colab. Here is the cell I'm running (it's just copy paste from the docs):
image

Here is the error I ran into: (I'm connected to a T4 runtime)

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
[<ipython-input-16-1b60a993f99a>](https://localhost:8080/#) in <cell line: 8>()
     26 
     27     # Create loaders
---> 28     loaders[name] = Loader(f'/tmp/cifar_{name}.beton',
     29                             batch_size=BATCH_SIZE,
     30                             num_workers=8,

3 frames
[/usr/local/lib/python3.10/dist-packages/ffcv/loader/loader.py](https://localhost:8080/#) in __init__(self, fname, batch_size, num_workers, os_cache, order, distributed, seed, indices, pipelines, custom_fields, drop_last, batches_ahead, recompile)
    142             self.num_workers = cpu_count()
    143 
--> 144         Compiler.set_num_threads(self.num_workers)
    145 
    146         if indices is None:

[/usr/local/lib/python3.10/dist-packages/ffcv/pipeline/compiler.py](https://localhost:8080/#) in set_num_threads(cls, n)
     18             n = cpu_count()
     19         cls.num_threads = n
---> 20         set_num_threads(n)
     21         ch.set_num_threads(n)
     22 

[/usr/local/lib/python3.10/dist-packages/numba/np/ufunc/parallel.py](https://localhost:8080/#) in set_num_threads(n)
    606     if not isinstance(n, (int, np.integer)):
    607         raise TypeError("The number of threads specified must be an integer")
--> 608     snt_check(n)
    609     _set_num_threads(n)
    610 

[/usr/local/lib/python3.10/dist-packages/numba/np/ufunc/parallel.py](https://localhost:8080/#) in snt_check(n)
    568     def snt_check(n):
    569         if n > NUMBA_NUM_THREADS or n < 1:
--> 570             raise ValueError(msg)
    571     return snt_check
    572 

ValueError: The number of threads must be between 1 and 2```
@levongh
Copy link

levongh commented Jul 26, 2024

@AntonioMacaronio try to change the ffcv version, from the backtrace it seems the ffcv version is pretty old. the code is changed by the following commit 352d1ef

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants