Dev quant squeeze #941
Dev quant squeeze #941
Conversation
|
have formatted the code and added a quick fix @Giuseppe5 |
|
Hi @Giuseppe5, I've updated the code to be compatible with PyTorch < 2.0, and it should now pass the CI checks. In PyTorch < 2.0, torch.squeeze() doesn't support multiple dims.
To make Brevitas flexible, QuantTensor.squeeze() always supports multiple dims, even when using PyTorch < 2.0. The method seems more complex than other meta-data sensitive methods because it needs to guarantee that the value tensor and the meta-data tensors choose the same dims to squeeze. |
| tensor_meta = { | ||
| 'scale': self.scale, 'zero_point': self.zero_point, 'bit_width': self.bit_width} | ||
| for k, tm in tensor_meta.items(): | ||
| if tm is not None and len(value.shape) == len(tm.shape) - len(sorted_target_dims): |
There was a problem hiding this comment.
metadatas cannot be None anymore
| @@ -46,6 +46,21 @@ def transpose_handler(inp, *args, **kwargs): | |||
| return inp.transpose(*args, **kwargs) | |||
|
|
|||
|
|
|||
| @implements(torch.permute) | |||
| def permute_handler(inp, *args, **kwargs): | |||
| return inp.permute(*args, **kwargs) | |||
There was a problem hiding this comment.
Don't we need a permute implementation to permute metadata of quant tensor?
There was a problem hiding this comment.
Sorry for the late reply. I will update my code next week : )
|
As you can see, in the meantime we merged another PR to add support for FloatQuantTensor. The extension to that should be trivial but I don't want to expand the scope too much. |
The PR was submitted to add the Squeeze/Unsqueeze operation to QuantTensor. Issue: #891
Changes
See more discussions in #891 #728.