Static type checking #39

wangpatrick57 · 2024-09-02T21:42:21Z

Summary: added mypy --strict to CI and fixed all type errors (~900 total) found by it. This also found and fixed some logical errors in the code.

Demo:

Passing CI

Details:

Readability is much better now that everything is typed.
mypy found that we were using a feature from torch 2.4.0 so I upgraded torch from 2.0.0 -> 2.4.0.
Some type errors were because of dead code. These were removed.
Some type errors showed places where we were using the wrong type.
Many asserts were added for Optional[...] types.
Fixed a previously confusing situation around mixing psycopg.Connection and sqlalchemy.Connection in pg.py:create_conn().

lmwnshn

Rubber stamp.

wangpatrick57 added 30 commits September 1, 2024 22:34

added __init__.py files to all folders

eae129d

fixed some simple type errors

5eaf4bb

fixed some more errors

5b94622

fixed tune/protox/env/logger.py

3500ba8

fixed task.py

4bf44ad

fixed tune/cli.py

36eb67a

fixed tune/protox/cli.py

2c32b4f

fixed tune/protox/agent/cli.py

06e1e40

fixed tune/protox/agent/tune.py

3081e75

fixed tune/protox/agent/hpo.py

5b5d50b

fixed tune/protox/agent/replay.py

6a0468a

added mypy to req.txt

9318b8c

fixed a few scattered errors

955216b

fixed tune/protox/embedding/train_args.py

e054e82

fixed scripts/read_parquet.py

0c81612

fixed util/shell.py

8fc0af5

fixed tune/protox/env/space/primitive/index.py

b626bb4

fixed tune/protox/embedding/vae.py

b0fbba7

fixed misc/utils.py

0eeccc5

fixed tune/protox/agent/hpo.py

921ac5d

fixed tune/protox/agent/replay.py

a6f2240

fixed tune/protox/agent/build_trial.py

64ddbd9

fixed tune/protox/embedding/cli.py

d9d6996

fixed tune/protox/embedding/train.py

70c2d19

now ignoring errors in embedding/

5f52029

fixed util/pg.py

6228088

fixed tune/protox/env/mqo/mqo_wrapper.py

96338bc

fixed tune/protox/env/pg_env.py

6996e2c

fixed tune/protox/agent/replay.py

20d83bf

fixed tune/protox/tests/test_index_space.py

8c59fde

wangpatrick57 added 26 commits September 2, 2024 20:48

fixed tune/protox/env/space/holon_space.py

3303148

fixed tune/protox/agent/off_policy_algorithm.py

04aae88

fixed tune/protox/env/util/pg_conn.py

bb1043f

fixed tune/protox/env/logger.py

8d7d109

fixed tune/protox/tests/test_workload_utils.py

cb2d068

fixed tune/protox/env/util/workload_analysis.py

47c762e

fixed tune/protox/tests/test_primitive.py

2f68f7d

fixed benchmark/tpch/cli.py

09bdf39

fixed dbms/postgres/cli.py

c80f814

fixed manage/tests/test_clean.py

e39c151

fixed benchmark/tpch/load_info.py

b93d3f4

fixed manage/cli.py

fcf6894

replaced List/Dict/Set with list/dict/set

4273cd8

reformatted

e62f612

added mypy to CI

bada733

small type error

e094409

fixed issues around psycopg and sqlalchemy conn

5ec9ff6

made a few fixes to select.py

632d2a2

fixed tune/protox/embedding/datagen.py

181c6e9

fixed tune/protox/embedding/train_all.py

fff33e1

fixed tune/protox/embedding/select.py

64189a5

fixed tune/protox/embedding/analyze.py

31d4330

fixed other mypy bugs

10cbfe1

format

c509893

fixed create_sqlalchemy_conn using the psycopg connstr

ba9f27c

step_post_execute() now returns an Optional[float] for reward

67d2cf5

wangpatrick57 requested review from lmwnshn and 17zhangw September 3, 2024 17:50

lmwnshn approved these changes Sep 5, 2024

View reviewed changes

wangpatrick57 merged commit ac849f8 into cmu-db:main Sep 5, 2024
1 check passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Static type checking #39

Static type checking #39

wangpatrick57 commented Sep 2, 2024 •

edited

Loading

lmwnshn left a comment

Static type checking #39

Static type checking #39

Conversation

wangpatrick57 commented Sep 2, 2024 • edited Loading

lmwnshn left a comment

Choose a reason for hiding this comment

wangpatrick57 commented Sep 2, 2024 •

edited

Loading