fix multiprocess issue (RuntimeError An attempt has been made to start a new process before the current process has finished its bootstrapping phase) #62
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fix
RuntimeError
in #28 once and for all by protecting main code withif __name__ != '__main__': return
Background:
awq
package was imported intrain.py
outside of a function. This was due topeft
importingawq
if installed, causing problems for users who had autoawq installed (me). The previous PR #30 solved by moving the peft imports inside the functions where the modules are needed.multiprocess
package is causing the problem. It appears that importing multiprocess changes the way child processes are created, similar to the behavior on Windows (see PyTorch documentation).train.py
does not import multiprocess directly, the HuggingFacedatasets
package does. If a user modifies the code to import datasets, for example withfrom datasets import load_dataset
, the error occurs.train.py
withif __name__ != '__main__': return
, similar to how it was recommended in the error message.Error:
This change should prevent the RuntimeError from occurring, regardless of whether the user imports packages that affect process creation.