v3.32.0

A BREAKING change introduced to enforce an allowed list of CORS origins for websockets to address cross-site websocket hijacking vulnerability

What's Changed

• enhancement: make left sidebar of datadoc resizable by @adamstruck in #1346
• fix: add native sum column analyzer by @rchandnaeg in #1418
• feat: Auto-load built-in notifiers if configured by @baumandm in #1420
• fix: Metastore ACL support for wildcard prefix match by @baumandm in #1423
• fix: enable websocket cors for production by @jczhong84 in #1425

Full Changelog: v3.31.2...v3.32.0

v3.31.2

A stored XSS vulnerability was observed and patched in this version.

What's Changed

• fix: Support serialized JSON environment variables by @baumandm in #1415
• fix: Fix DataDoc cell move down button by @baumandm in #1410
• [Snyk] Security upgrade jinja2 from 3.1.2 to 3.1.3 by @devinlundberg in #1394
• fix: schema filtering in table search modal by @adamstruck in #1348

Full Changelog: v3.31.1...v3.31.2

Querybook 3.31.1

What's Changed

A XSS vulnerability was observed and patched in this version. Checkout GHSA-fh6g-gvvp-587f for more details.

• chore: Fix typo in add_ai_assistant.md by @baumandm in #1328
• fix: Fix DataDoc contents overflow in non-Chrome browsers by @baumandm in #1332
• fix: some small changes and fixes for AI assistant by @jczhong84 in #1333
• Revert "fix: Fix DataDoc contents overflow in non-Chrome browsers (#1332)" by @jczhong84 in #1336
• chore(deps): bump uwsgi from 2.0.20 to 2.0.22 in /requirements by @dependabot in #1338
• Add column name suggestions to presto validator by @kgopal492 in #1330
• feat: ask users to confirm tables before query generation by @jczhong84 in #1339
• fix: Update docker-compose.yml - added shielding by @YuriyGavrilov in #1340
• chore: add query to ai assistant context logging by @jczhong84 in #1342
• fix: hotfix for worker starting up by @czgu in #1344
• Use gevent for celery worker, update timeout to GeventTimeout by @czgu in #1345
• fix: store the task type value rather than enum when creating a task by @adamstruck in #1329
• Refactor validation decorators by @kgopal492 in #1354
• Add query execution links to exporter results by @kgopal492 in #1364
• fix: Update runservice worker scripts to disable unneeded features by @baumandm in #1371
• Add custom message to validation decorator by @kgopal492 in #1373
• fix: cast table max upload size to int by @rchandnaeg in #1374
• ui: add surveys to querybook by @czgu in #1375
• fix: s3 table upload location by @jczhong84 in #1376
• fix: Add scroll bar to Query Engine Status popover by @baumandm in #1353
• Bigquery sqlalchemy metastore by @seuf in #1352
• Fix lint, package version, and survey config by @czgu in #1379
• ui: update survey bg by @meowcodes in #1381
• fix: possible unauthenticated SQL injection when login by @jczhong84 in #1383
• [chore] upgrade docs website to latest, add build test by @czgu in #1380
• fix: Store group description in public_info by @baumandm in #1384
• feat: update column stats ui, add sort by usage to columns by @kgopal492 in #1389
• feat: added ability to omit clean_up jobs from general db_clean task and fixed clean_up_archived_data_doc by @jij1949 in #1387
• fix: add a clear_sheet parameter for google sheet exporter by @jczhong84 in #1401
• fix: modify trigger for query_authoring survey by @jczhong84 in #1400
• Add table column statistics to text2sql query info by @kgopal492 in #1402
• fix: Update unique_table_ownership constraint to allow owner per type by @baumandm in #1403
• chore: Update pip install command by @baumandm in #1407
• fix: Fix create task schedule task_type by @baumandm in #1408
• fix: survey for table modal is blinking by @czgu in #1396
• feat: upgrade langchain version by @jczhong84 in #1406
• feat: enable impression logging of survey events by @czgu in #1411
• fix: XSS injection with Querybook RichTextEditor by @czgu in #1412

New Contributors

• @YuriyGavrilov made their first contribution in #1340
• @adamstruck made their first contribution in #1329
• @rchandnaeg made their first contribution in #1374
• @seuf made their first contribution in #1352

Full Changelog: v3.28.0...v3.31.1

Querybook 3.28.0

Please check here for highlight features

What's Changed

• docs: add security advisory info by @czgu in #1094
• fix: notify_user need to accept user not uid by @czgu in #1095
• fix: sidebar search should use relevance instead of alphabetical by @czgu in #1096
• fix: remove python-dev from dockerfile build by @czgu in #1098
• fix: boost table score for exact match when searching table by @jczhong84 in #1097
• Tune exact match boost score of table search by @jczhong84 in #1099
• fix: scheduled docs with latest_partition fail to run by @jczhong84 in #1101
• fix: hasOwnProperty is not a function issue from json-big by @jczhong84 in #1103
• feat: add support of running all cells of a data doc by @jczhong84 in #1102
• feat: add logout event hook by @czgu in #1104
• Add 2022 change logs by @jczhong84 in #1105
• feat: Make exact table search result auto show up by @czgu in #1106
• feat: Add shortcut for toggle ToC, update datadoc short cuts by @czgu in #1107
• fix: Chart date axis bug by @czgu in #1108
• feat: auto add quotes for table/columns by @czgu in #1109
• feat: add websocket logging by @jczhong84 in #1110
• fix: update event_log table schema by @jczhong84 in #1111
• feat: add drag and drop for templated variables by @czgu in #1112
• feat: a bunch of small ui fixes and improvements by @czgu in #1114
• feat: add frontend context logging by @jczhong84 in #1115
• move change log images to docs_website by @jczhong84 in #1117
• feat: validation now works for templated query by @czgu in #1119
• Add some frontend view/click events by @jczhong84 in #1118
• fix: Add disabled indicators to Schedules list by @baumandm in #1122
• feat: Visualize complex Hive column types v2 by @baumandm in #1091
• fix: Extend DataTableColumn type column length by @baumandm in #1121
• chore: update yaml-loader by @jczhong84 in #1124
• [Snyk] Security upgrade qs from 6.10.1 to 6.10.3 by @devinlundberg in #1092
• [Snyk] Security upgrade setuptools from 39.0.1 to 65.5.1 by @devinlundberg in #1113
• fix: remove duplicate volume mount by @jczhong84 in #1126
• chore: version upgrade a few packages by @czgu in #1123
• chore: update richtext editor to hooks by @czgu in #1127
• fix: json-bigint hasOwnProperty undefined issue by @jczhong84 in #1129
• fix: update /event_log api path by @jczhong84 in #1130
• fix: react-table version by @jczhong84 in #1132
• fix: axios error display by @czgu in #1133
• fix: remove timeout for presto explain by @czgu in #1135
• feat: add metastore loader config by @jczhong84 in #1134
• Feat: add enable and list filters for Scheds by @alexVasylenko in #1076
• fix: stay on current env after deleting doc by @lilyli9 in #1138
• chore: refactor DataTableView to hooks by @czgu in #1137
• add axios request timeout option by @jczhong84 in #1139
• chore: Move QueryViewNavigator to hooks by @czgu in #1140
• feat: query engine search scroll by @lilyli9 in #1136
• feat: add search box to hide columns popup by @lilyli9 in #1128
• fix: add safe suppress to sqlformat error by @czgu in #1141
• ui: Update searchbar css by @czgu in #1142
• fix: clicking a readonly text cell causes querybook crash by @jczhong84 in #1148
• fix: Fix Scheduled DataDocs Only toggle by @baumandm in #1149
• feat: add user group db schema support by @jczhong84 in #1144
• fix: sql-formatter by @jczhong84 in #1152
• fix: event logger when current user is None by @jczhong84 in #1153
• fix: number type variable with value 0 by @jczhong84 in #1154
• chore: not notify on query linting exceptions by @jczhong84 in #1155
• feat: support for a general validation message in query editor by @suryabhaskarkarri in #1156
• feat: add message when linter failed to run by @czgu in #1157
• feat: add estimated time for query execution by @czgu in #1158
• fix: unify status code and move user error to 400 by @czgu in #1161
• fix: filter null store values by @lilyli9 in #1163
• feat: toggle run all notifications by @lilyli9 in #1162
• feat: Add ability to cancel dead queries by @czgu in #1159
• fix: Revert "feat: add estimated time for query execution (#1158)" by @czgu in #1166
• refactor: progress ui by @czgu in #1168
• add more configurations for ldap integration by @okayhooni in #1145
• update querybook helm charts by @okayhooni in #1147
• ui: fix overflow for long usernames by @czgu in #1173
• chore: display error message from sql-formatter by @jczhong84 in #1175
• chore: upgrade sql-formatter by @jczhong84 in #1177
• fix: Enable mssql transpiling by @baumandm in #1178
• feat: add descriptive error msgs by @lilyli9 in #1150
• feat: make struct form fields ordered by @czgu in #1181
• feat: add more metadata support by @jczhong84 in #1182
• feat: find the closed color in ColorPalette by @jczhong84 in #1184
• fix: find nearest color by @jczhong84 in #1185
• feat: add button to sort columns in view table UI by @lilyli9 in #1187
• feat: add new metadata type - data element by @jczhong84 in #1191
• fix: add tooltip length by @jczhong84 in #1192
• feat: create data element along with syncing table by @jczhong84 in #1193
• docs: add helm deployment guide by @yongchand in #1183
• feat: display data element description as column description by @jczhong84 in #1195
• fix: check nullish of table property value by @jczhong84 in #1197
• fix: some bug fix related to tag and data element by @jczhong84 in #1199
• fix: remove broadcast from socketio.emit by @jczhong84 in #1202
• docs: update the step by step guide of connecting to a query engine by @jczhong84 in #1201
• fix: column name overlaps with type by @jczhong84 in #1203
• feat: add stats logging by @jczhong84 in #1204
• feat: add configurable timeout for Presto to querybook by @czgu in #1207
• feat: update stats metrics names and add tags by @jczhong84 in #1208
• fix: update data_element table charset to utf8 by @jczhong84 in #1209
• fix: add get_schema_location to metastore loader by @jczhong84 in #1214
• feat: add data elements to ES index by @jczhong84 in https://github.com/pinterest/qu...

Querybook 3.14.2

What's Changed

• Add DataHub Query Result Type Detector/Stats Analyzer/Display Transformer by @czgu in #290
• Add isNumeric and make type check more general by @czgu in #293
• UI Fix for user menu by @czgu in #295
• Ensure s3 chunk read accounts for utf8 length by @czgu in #296
• Ignore contents inside bracket of a query if it does not start keywords like SELECT by @czgu in #297
• Show DataHub version in UI by @czgu in #298
• Update filter... placeholder for nav bars by @czgu in #299
• Add force form export button by @czgu in #300
• Ensure header is sticky in full screen mode by @czgu in #301
• Add documentation for query result transform by @czgu in #303
• Fix multiple charting bugs by @czgu in #306
• Fixed wrong execution picking behaivor for chart cell by @czgu in #307
• Add snowflake support for DH by @czgu in #313
• Version Bump Styled Components by @czgu in #314
• Add typescript checking to unit test, fixed all errors by @czgu in #315
• Bump dot-prop from 4.2.0 to 4.2.1 in /docs_website by @dependabot in #316
• [BUG] CI breaks when trying to set env var, updating how env vars are set by @darapuk in #318
• Use ESLint instead of TSLint by @czgu in #317
• Add Board View by @meowcodes in #319
• Fix auto complete in codemirror due to eslint update by @czgu in #320
• Bump version to resolve alerts by @czgu in #321
• Bump Python & Flask version by @czgu in #322
• Add snowflake syntax error parsing by @czgu in #323
• Custom homepage plugin by @czgu in #324
• Make source query only show the last entry by @czgu in #325
• Only show query engine from in the environment for samples by @czgu in #326
• Add new documentation site for DataHub by @czgu in #327
• Add Change note for Nov 2020 Update by @czgu in #328
• Add Google form for waitlist by @czgu in #329
• [UI] Update Lush Theme Colors & Execution List Alignment by @meowcodes in #331
• [Fix] Classname Typo by @meowcodes in #332
• Fix snapshot tests by @czgu in #333
• Fix issue with table view not available in full view by @czgu in #335
• Add docker to docs site by @czgu in #337
• Update front page styling of the docs site by @czgu in #336
• Rename DataHub to Querybook for docs and docs site by @czgu in #339
• [Board] Add title and description to board view by @meowcodes in #340
• Fix selection behavior by @czgu in #341
• Use Gifs for Docs site example by @czgu in #342
• Pin EngineIO version to fix socketio issue by @czgu in #344
• Update docs website message, add alt to img by @czgu in #343
• Fixed gradient text with wrong class by @czgu in #345
• Make clone DataDoc public/private config same as create new DataDoc by @czgu in #346
• Update lexer to not categorize distinct from as table by @czgu in #347
• Update docs formatting by @czgu in #348
• Delegate markdown rendering to the frontend by @czgu in #349
• Add force show functionality to meta info by @czgu in #351
• Make sure border radius is shown for docs image modal by @czgu in #352
• Update the html plugin guide by @czgu in #353
• Add correct cascade behavior for sqlalchemy by @czgu in #354
• Add search filters for table search by @czgu in #356
• Add foreign key name in the latest migration by @czgu in #358
• Update doc site url to querybook.com by @czgu in #359
• Update Favicon for Querybook 🐘 by @czgu in #360
• Only show my lists on the sidebar by @czgu in #361
• Make Markdown use external links, Make adhoc query persists in indexDB by @czgu in #362
• Add the ability to order board in doc nav by @czgu in #363
• add icon titles for left+right sidebars by @czgu in #365
• Allow query title to be multiline by @czgu in #364
• Use content for markdown formatting by @czgu in #367
• Decentralized storybook, update to v6 by @czgu in #366
• Manual bump axios to fix security alert by @czgu in #368
• Replace notification with toast by @czgu in #369
• Add multi export functionality to scheduled DataDoc by @czgu in #372
• Remove dead code / unused references by @czgu in #370
• Simplified api check for uid by @czgu in #371
• Use HTML to store rich text content by @czgu in #373
• Remove license file from repo by @czgu in #374
• Rename DataHub to Querybook by @czgu in #375
• Remove unused files by @czgu in #378
• Fix Embedded Query Page by @czgu in #377
• Bump Jest version to fix security issue by @czgu in #381
• Ensure scheduler works with old configs by @czgu in #380
• Fix & Update Tutorials by @czgu in #382
• Add the ability to customize app name by @czgu in #379
• Add the official doc site in readme by @czgu in #383
• Bump ini from 1.3.5 to 1.3.8 by @dependabot in #391
• Fix scheduler export by @czgu in #392
• Bump immer version by @czgu in #390
• Add gtag tracking to docs website by @czgu in #393
• Update README url by @czgu in #394
• Add babel styled components plugin by @czgu in #397
• Refactor run_query into multiple modules by @czgu in #396
• Autofocus adhoc query editor at start by @czgu in #395
• Update organization of documentation site by @czgu in #398
• Make webpack hot reload in docker compose by @czgu in #401
• Fixed home page to docs url by @czgu in #402
• Fix alignment of logo by @czgu in #403
• Add code coverage with python by @czgu in #404
• Updated auth documentation, link login warning to doc by @czgu in #407
• Add OKTA auth integration by @czgu in #408
• Fix sort cell behavior for query results by @czgu in #410
• ci: use nondocker to run pytest and jest by @czgu in #413
• chore: Add automatic version bumping by @czgu in #412
• feat: Update query engine archive behavior by @czgu in #411
• chore: Add eye icon for password field by @czgu in #409
• docs:Document querybook dev experience on windows by @czgu in #415
• ci: update version bump & document tag by @czgu in https://github.com/pinterest/querybook/pull/...

Querybook 2.4

Features

Sidebar 2.0! A new much more powerful sidebar with lists, recent, and favorites all in a single view, with drag and drop support.
Table Tags! You can now easily add tags to tables and retrieve all tables with a certain tag through search.
Table Ownership Editing! With a single click, you can add yourself as the owner of a table.
GDoc Style Access Requests! You can now request and grant access to a given DataDoc directly in the UI.
Table Stats! Use it to populate any auto-generated statistics for all tables.
Column Search! Now you can narrow down your search to only columns, descriptions, or table names.
Sample Queries! Quickly see sample queries of a given table to kickstart your analysis.
Table Warnings! Found a given table has wrong data? Easily add table warnings and errors that will then show up in the query editor.
Frequent query users! See at a glance the top users of a given table, reach out to them for questions and doubts.

v2.3.0: Fix unable to set enable for schedule (#82)

Major Features

• Scheduling with Exporting: Automate export results to Google Sheets and get notified via Slack or email with the new DataDoc scheduling UI #60
• DataDoc Search/Replace: Navigate around your doc easily with the new search and replace UI! Also available for ad hoc query! #30
• Copy/Cut/Paste: You can now move your Query Cells to another DataDoc & retain the same execution history. #38
• Improved Charting: We added custom x-axis sorting and the option to show values in charts. #29 #59
• Advanced Templating: You can now nest templated variable definitions and leverage the full potentials of templating with Jinja2. #45
• Table Relevance: Autocomplete suggestions and table search are now ranked by impressions and usage of the table. #36

Minor Fixes

• You can now collapse the query inside a Query Cell.
• Sharing query execution in DataDoc will link to the query execution page directly.
• Clicking on a cell automatically updates the url to point to the cell for easy sharing
• Query completion message is rephrased to show the status first.
• Added editor settings to let users configure autocompletion, tab space, and font size.
• Running a CREATE as INSERT or insert query updates the table lineage automatically
• URLs will be auto-detected and converted to links in Text Cells
• Creating a DataDoc will include an empty Query Cell
• You can now convert an ad-hoc query to a DataDoc with a single click
• Exported google sheets are now editable and owned by the user
• DataDoc scheduling now records failures if errors occurred during scheduled runs