Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: pushdown filters for some information_schema tables #3091

Conversation

killme2008
Copy link
Contributor

@killme2008 killme2008 commented Jan 3, 2024

I hereby agree to the terms of the GreptimeDB CLA

What's changed and what's your intention?

Some tables in information_schema will be large in the production when there are many created tables, such as columns and tables etc.
This PR supports pushing down query filters for some information_schema tables, it would reduce the memory consumption while constructing the result stream and speed up the query.

Currently it supports following filters:

  • Eq
  • NotEq
  • InList
  • expr AND expr
  • expr OR expr
  • NOT expr
  • expr LIKE pattern

We can support more types in the future. Looks like datafusion doesn't push down like expression currently. @waynexia

Checklist

  • I have written the necessary rustdoc comments.
  • I have added the necessary unit tests and integration tests.
  • This PR does not require documentation updates.

Refer to a related PR or issue link (optional)

#2931

@github-actions github-actions bot added docs-required This change requires docs update. Size: M labels Jan 3, 2024
@killme2008 killme2008 added docs-not-required This change does not impact docs. and removed docs-required This change requires docs update. labels Jan 3, 2024
@github-actions github-actions bot added Size: L and removed Size: M labels Jan 3, 2024
@killme2008 killme2008 marked this pull request as ready for review January 4, 2024 02:31
Copy link

codecov bot commented Jan 4, 2024

Codecov Report

Attention: 71 lines in your changes are missing coverage. Please review.

Comparison is base (ec43b91) 85.61% compared to head (452ad9d) 85.19%.
Report is 10 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3091      +/-   ##
==========================================
- Coverage   85.61%   85.19%   -0.43%     
==========================================
  Files         804      807       +3     
  Lines      131031   132101    +1070     
==========================================
+ Hits       112180   112541     +361     
- Misses      18851    19560     +709     

@killme2008 killme2008 force-pushed the feature/information-schema-pushdown-filter branch from 6c49e04 to b8f5a73 Compare January 4, 2024 10:17
@killme2008
Copy link
Contributor Author

@zhongzc PTAL

Copy link
Contributor

@zhongzc zhongzc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@killme2008 killme2008 mentioned this pull request Jan 5, 2024
14 tasks
@killme2008 killme2008 added this pull request to the merge queue Jan 5, 2024
Merged via the queue into GreptimeTeam:main with commit c2db970 Jan 5, 2024
16 checks passed
@killme2008 killme2008 deleted the feature/information-schema-pushdown-filter branch January 5, 2024 07:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
docs-not-required This change does not impact docs.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants