We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
In TemporaryCheckpointsJSON(https://github.com/huggingface/transformers-bloom-inference/blob/main/inference_server/models/ds_inference.py#L80) ,
When use glob.glob(f"{self.model_path}/*.bin"), files path in the list will all contain model_path prefix (eg: modelname is bigscience/bloom ).
glob.glob(f"{self.model_path}/*.bin")
model_path
bigscience/bloom
{"type": "BLOOM", "checkpoints": ["bigscience/bloom/pytorch_model.bin"], "version": 1.0}
While set it as root_dir (glob.glob("*.bin", root_dir=self.model_path)) will not:
root_dir
glob.glob("*.bin", root_dir=self.model_path)
{"type": "BLOOM", "checkpoints": ["pytorch_model.bin"], "version": 1.0}
And it will align to DeepSpeed's loading way (replace_module.py). Because when loading, it will add root_dir again:
sd = [ torch.load(os.path.join(base_dir1, checkpoint[i]), map_location='cpu') ]
So, current dump way will duplicate model_path.
And I raised a PR: #71 .
The text was updated successfully, but these errors were encountered:
No branches or pull requests
In TemporaryCheckpointsJSON(https://github.com/huggingface/transformers-bloom-inference/blob/main/inference_server/models/ds_inference.py#L80) ,
![image](https://user-images.githubusercontent.com/5948851/233960651-2b64a5f8-2d8a-4982-88fe-50852381f635.png)
When use
glob.glob(f"{self.model_path}/*.bin")
, files path in the list will all containmodel_path
prefix (eg: modelname isbigscience/bloom
).While set it as
root_dir
(glob.glob("*.bin", root_dir=self.model_path)
) will not:And it will align to DeepSpeed's loading way (replace_module.py). Because when loading, it will add
root_dir
again:So, current dump way will duplicate
model_path
.And I raised a PR: #71 .
The text was updated successfully, but these errors were encountered: