Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

加载模型报错ValueError: prefetch_factor option could only be specified in multiprocessing.let num_workers > 0 to enable multiprocessing, otherwise set prefetch_factor to None. #453

Open
licamla opened this issue Mar 28, 2023 · 2 comments

Comments

@licamla
Copy link

licamla commented Mar 28, 2023

Describe the bug

from fastHan import FastHan

model = FastHan()
model.set_cws_style('cityu')

model('加载模型报错')

加载模型报错

  File "/lib/python3.10/site-packages/fastNLP/core/dataloaders/torch_dataloader/fdl.py", line 152, in __init__
    super().__init__(**dl_kwargs)
  File "/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 245, in __init__
    raise ValueError('prefetch_factor option could only be specified in multiprocessing.'
ValueError: prefetch_factor option could only be specified in multiprocessing.let num_workers > 0 to enable multiprocessing, otherwise set prefetch_factor to None.

Desktop (please complete the following information):

torch 2.0.0+cpu
FastNLP 1.0.1

Additional context
torch的DataLoader参数num_workers=0时,prefetch_factor需要设置成None。
而fastNLP里的子类TorchDataLoader的默认num_workers=0,prefetch_factor=2。按道理来说TorchDataLoader应该像DataLoader默认prefetch_factor=None

@licamla licamla changed the title 下载模型报错ValueError: prefetch_factor option could only be specified in multiprocessing.let num_workers > 0 to enable multiprocessing, otherwise set prefetch_factor to None. 加载模型报错ValueError: prefetch_factor option could only be specified in multiprocessing.let num_workers > 0 to enable multiprocessing, otherwise set prefetch_factor to None. Mar 28, 2023
@JustinPeanut
Copy link

您好,我使用了0.6.0版本的fastnlp,但是在下载bert预训练模型的时候,http报502,服务器是停止维护了吗?

@xiaoyingwang055
Copy link

解决了吗,我也报错了

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants