Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move process imports to occur at runtime #84

Closed
LukeWeidenwalker opened this issue Mar 13, 2023 · 1 comment
Closed

Move process imports to occur at runtime #84

LukeWeidenwalker opened this issue Mar 13, 2023 · 1 comment

Comments

@LukeWeidenwalker
Copy link
Contributor

Since the process imports are defined as top-level imports, the dependencies are imported at the time the implementation is registered into the process registry. So even for a super simple process graph, all the packages are imported. This slows down the execution of the CLI in the executor by quite a bit (~5-10 seconds locally), so would be a good change delay these imports until the process is actually called. Not sure exactly how to best do this. The naive approach would be to move import statements within the function definitions, but I also don't want to duplicate lines like import numpy as np all over the place. Maybe there's a smart solution that has the same effect. Alternatively we can just do this for the heavier imports (xgboost, scipy, etc.).

@LukeWeidenwalker
Copy link
Contributor Author

Closed by #135

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant