You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Since the process imports are defined as top-level imports, the dependencies are imported at the time the implementation is registered into the process registry. So even for a super simple process graph, all the packages are imported. This slows down the execution of the CLI in the executor by quite a bit (~5-10 seconds locally), so would be a good change delay these imports until the process is actually called. Not sure exactly how to best do this. The naive approach would be to move import statements within the function definitions, but I also don't want to duplicate lines like import numpy as np all over the place. Maybe there's a smart solution that has the same effect. Alternatively we can just do this for the heavier imports (xgboost, scipy, etc.).
The text was updated successfully, but these errors were encountered:
Since the process imports are defined as top-level imports, the dependencies are imported at the time the implementation is registered into the process registry. So even for a super simple process graph, all the packages are imported. This slows down the execution of the CLI in the executor by quite a bit (~5-10 seconds locally), so would be a good change delay these imports until the process is actually called. Not sure exactly how to best do this. The naive approach would be to move import statements within the function definitions, but I also don't want to duplicate lines like
import numpy as np
all over the place. Maybe there's a smart solution that has the same effect. Alternatively we can just do this for the heavier imports (xgboost, scipy, etc.).The text was updated successfully, but these errors were encountered: