Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add option to automatically try all parse modes #228

Open
wants to merge 1 commit into
base: 15-latest
Choose a base branch
from

Conversation

seanlinsley
Copy link
Member

@seanlinsley seanlinsley commented Dec 21, 2023

This adds a PG_QUERY_PARSE_ALL parse mode to automatically try every parse mode, which is helpful if you're parsing a query and don't know ahead of time which parse mode is valid for the query.

Additionally, this adds pg_query_normalize_opts which was missed by #216.

Note that the parse tree and fingerprints are different for some queries because multiple parse modes are able to parse the query. The NEW.name and NEW.author.first_name parse trees are actually incorrect so we may need to change the order in which the parse modes are tried.

# parse_opts
INVALID result for "v3.c1 := 4" with 6 mode
expected: {"version":150001,"stmts":[{"stmt":{"PLAssignStmt":{"name":"v3","indirection":[{"String":{"sval":"c1"}}],"nnames":2,"val":{"targetList":[{"ResTarget":{"val":{"A_Const":{"ival":{"ival":4},"location":9}},"location":9}}],"limitOption":"LIMIT_OPTION_DEFAULT","op":"SETOP_NONE"}}}}]}
  actual: {"version":150001,"stmts":[{"stmt":{"PLAssignStmt":{"name":"v3","indirection":[{"String":{"sval":"c1"}}],"nnames":1,"val":{"targetList":[{"ResTarget":{"val":{"A_Const":{"ival":{"ival":4},"location":9}},"location":9}}],"limitOption":"LIMIT_OPTION_DEFAULT","op":"SETOP_NONE"}}}}]}
INVALID result for "NEW.name = upper(cleanString(NEW.name))" with 6 mode
expected: {"version":150001,"stmts":[{"stmt":{"PLAssignStmt":{"name":"new","indirection":[{"String":{"sval":"name"}}],"nnames":2,"val":{"targetList":[{"ResTarget":{"val":{"FuncCall":{"funcname":[{"String":{"sval":"upper"}}],"args":[{"FuncCall":{"funcname":[{"String":{"sval":"cleanstring"}}],"args":[{"ColumnRef":{"fields":[{"String":{"sval":"new"}},{"String":{"sval":"name"}}],"location":29}}],"funcformat":"COERCE_EXPLICIT_CALL","location":17}}],"funcformat":"COERCE_EXPLICIT_CALL","location":11}},"location":11}}],"limitOption":"LIMIT_OPTION_DEFAULT","op":"SETOP_NONE"}}}}]}
  actual: {"version":150001,"stmts":[{"stmt":{"SelectStmt":{"targetList":[{"ResTarget":{"val":{"A_Expr":{"kind":"AEXPR_OP","name":[{"String":{"sval":"="}}],"lexpr":{"ColumnRef":{"fields":[{"String":{"sval":"new"}},{"String":{"sval":"name"}}]}},"rexpr":{"FuncCall":{"funcname":[{"String":{"sval":"upper"}}],"args":[{"FuncCall":{"funcname":[{"String":{"sval":"cleanstring"}}],"args":[{"ColumnRef":{"fields":[{"String":{"sval":"new"}},{"String":{"sval":"name"}}],"location":29}}],"funcformat":"COERCE_EXPLICIT_CALL","location":17}}],"funcformat":"COERCE_EXPLICIT_CALL","location":11}},"location":9}}}}],"limitOption":"LIMIT_OPTION_DEFAULT","op":"SETOP_NONE"}}}]}
INVALID result for "NEW.author.first_name = upper(cleanString(NEW.author.first_name))" with 6 mode
expected: {"version":150001,"stmts":[{"stmt":{"PLAssignStmt":{"name":"new","indirection":[{"String":{"sval":"author"}},{"String":{"sval":"first_name"}}],"nnames":3,"val":{"targetList":[{"ResTarget":{"val":{"FuncCall":{"funcname":[{"String":{"sval":"upper"}}],"args":[{"FuncCall":{"funcname":[{"String":{"sval":"cleanstring"}}],"args":[{"ColumnRef":{"fields":[{"String":{"sval":"new"}},{"String":{"sval":"author"}},{"String":{"sval":"first_name"}}],"location":42}}],"funcformat":"COERCE_EXPLICIT_CALL","location":30}}],"funcformat":"COERCE_EXPLICIT_CALL","location":24}},"location":24}}],"limitOption":"LIMIT_OPTION_DEFAULT","op":"SETOP_NONE"}}}}]}
  actual: {"version":150001,"stmts":[{"stmt":{"SelectStmt":{"targetList":[{"ResTarget":{"val":{"A_Expr":{"kind":"AEXPR_OP","name":[{"String":{"sval":"="}}],"lexpr":{"ColumnRef":{"fields":[{"String":{"sval":"new"}},{"String":{"sval":"author"}},{"String":{"sval":"first_name"}}]}},"rexpr":{"FuncCall":{"funcname":[{"String":{"sval":"upper"}}],"args":[{"FuncCall":{"funcname":[{"String":{"sval":"cleanstring"}}],"args":[{"ColumnRef":{"fields":[{"String":{"sval":"new"}},{"String":{"sval":"author"}},{"String":{"sval":"first_name"}}],"location":42}}],"funcformat":"COERCE_EXPLICIT_CALL","location":30}}],"funcformat":"COERCE_EXPLICIT_CALL","location":24}},"location":22}}}}],"limitOption":"LIMIT_OPTION_DEFAULT","op":"SETOP_NONE"}}}]}

# fingerprint_opts
INVALID result for "v3.c1 := 4" with 6 mode
expected: "a8c86658ce26a653"
actual: "7f65202632663ba0"
actual tokens: INVALID result for "NEW.name = upper(cleanString(NEW.name))" with 6 mode
expected: "bb52450e4d46f7a1"
actual: "184cff1a84037010"
actual tokens: INVALID result for "NEW.author.first_name = upper(cleanString(NEW.author.first_name))" with 6 mode
expected: "a148e3f78b53c252"
actual: "b7de0b0d4f636a16"

Todo:

  • if possible, fix the incorrect parse trees
  • update normalize tests to include literal values in every example to ensure they become bind params

Per 1/9 meeting:

  • verify if pg_stat_statements includes these parse mode variants
  • verify if auto_explain gives us enough information to know exactly which parse mode variant is needed

@seanlinsley seanlinsley requested a review from a team December 21, 2023 17:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant