-
Notifications
You must be signed in to change notification settings - Fork 9
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Make casting negative numbers more predictable. #96
Comments
To cut a long story short, it's because internally the conversion is using unsigned shift operations (i.e. 'logical' shifting is being forced rather than 'arithmetic' shifting). I can't remember whether this was intentional or not, but it quite possibly was because technically signed right shift is 'implementation defined' prior to C++20, whilst unsigned shifting is well defined. The fix would be fairly simple, it would just be a matter of removing a few casts and consequently some unneeded local type aliases. However, even with that the result won't be quite perfect because of the inherant loss of precision involved. |
A reminder for myself: |
Brief Description
Casting between different signed types can have unintuitive (incorrect?) results. Some examples of how to do this correctly would be welcomed.
Observed Behaviour
When casting negative Q1.14 values to Q3.12, the results are very unexpected. The fractional bit of the number seems to be within ~1 of the true value, but the integer part can be quite a bit off. What should be -1 ends up being 3. Or positive 2 instead of -2.
Expected Behaviour
Casting a fixed point number from one format to another should maintain a roughly equivalent value.
Source Code
Output:
Extra Information
The text was updated successfully, but these errors were encountered: