-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: Make TPCH dbgen text buffer size consistent with Presto Java #12169
fix: Make TPCH dbgen text buffer size consistent with Presto Java #12169
Conversation
✅ Deploy Preview for meta-velox canceled.
|
@minhancao : Please fix the PR title fix: Make TPCH dbgen text buffer size consistent with Presto Java |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @minhancao
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@minhancao : Please can you fix the test
[ RUN ] TpchConnectorTest.simple
/__w/velox/velox/velox/vector/tests/utils/VectorTestBase.cpp:151: Failure
Value of: expected->equalValueAt(actual.get(), i, i)
Actual: false
Expected: true
at 0: expected {0, ALGERIA, 0, furiously regular requests. platelets affix furious}, but got {0, ALGERIA, 0, haggle. carefully final deposits detect slyly agai}
[ FAILED ] TpchConnectorTest.simple (1422 ms)
Just tested with java and c++, checksum on o_comment matches with this change.
|
e42165f
to
a6a95e2
Compare
@aditi-pandit Fixed the test by updating the expected vector for n_comment column. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @minhancao
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good.
@minhancao Can you rebase against latest ? Not sure why the fuzzer build jobs are failing. |
…gen's text buffer size
a6a95e2
to
aee5149
Compare
@kgpai Rebased to latest main branch |
@xiaoxmeng has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
@xiaoxmeng merged this pull request in dea4758. |
Conbench analyzed the 0 benchmark runs that triggered this notification. None of the specified runs were found on the Conbench server. The full Conbench report has more details. |
Changed text buffer size to be 300 MB for Velox's dbgen to match with Java Presto TPCH dbgen's text buffer size. The text buffer size is used in randomly generating offset and length to grab a chunk from the overall text for each row. This fixed the difference in the comment column for the tables in TPCH.
Java:
https://github.com/trinodb/tpch/blob/master/src/main/java/io/trino/tpch/TextPool.java#L35
C++:
https://github.com/facebookincubator/velox/blob/main/velox/tpch/gen/DBGenIterator.cpp#L40
Resolves: prestodb/presto#24011