Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Potential eager cache False Positive hits due to floating point strings #448

Open
farazkh80 opened this issue Dec 16, 2024 · 0 comments
Open

Comments

@farazkh80
Copy link
Collaborator

In current eager cache implementation for tripy, we normalize the Trace of a tensor and take the string representation of it as a part of the key for cache inserts/lookups to obtain cache executables.

However, if some storage ops that are not lifted in the trace as inputs (i.e. output storage op placeholder for trace outputs), have float values with high precision, there is a non-zero probability that the early trimming/rounding of the string representation of the floating point values in the trace string (part of the key) would lead to incorrect cache behaviour (false positives).

Even though the odds of this happening is low for now, this issue can be addressed by

  1. hashing the trace instead of taking its str representation (need to ensure all children of trace at all levels are hashable (i.e all lists need to be turned into hashableList type)
  2. lifting all storage ops as inputs which will first require to solve shape of shape inference in MLIR-TRT.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant