Refactor utility modules and test_utility.py

[alyssadai]

We have a growing number of *_utils.py modules for organizing utility functions shared across or specific to CLI commands. Currently, all these utility modules live in the same top-level directory as the main app (cli.py).

Meanwhile, we have a single test suite for all util functions, which is getting pretty big.

To better organize these util functions and their tests, we should:

• create a utilities package to store all *_utils.py files
• rename the shared utility.py to model_utils.py for more specificity
  • also, import as a namespace instead of importing functions directly, for consistency w/ other modules
• split test_utility.py into util-specific test modules
• remove test test_generate_context() since it replicates logic of function being tested and is hard to maintain
• refactor out merging of context + final dataset instance for output .jsonld file, which overlaps across commands (Originally posted by @surchs in [ENH] Add derivatives command and pipeline-catalog submodule #349 (comment))

[alyssadai]

@surchs, does the issue description make sense to you?

[surchs]

Looks good @alyssadai. So the goal would be to have a single module util to import from, so that any util function could be imported like so from util import my_util_fun, yes?

[alyssadai]

So the goal would be to have a single module util to import from, so that any util function could be imported like so from util import my_util_fun, yes?

Actually, I would prefer to still import each set of utilities separately, for better organization and clarity on where functions are defined.

At the moment, utility functions live in different .py files based on what CLI command they are used in/relevant for: pheno_utils.py (for functions used by bagel pheno), derivatives_utils.py, etc. And then there are two sets of utilities shared across commands, utility.py (to be renamed model_utils.py for clarity) and file_utils.py.

I would have a directory structure like:

bagel/
   utils/
     __init__.py
     pheno_utils.py
     bids_utils.py
     ...
   ...

and then import them like:

from utils import pheno_utils as putils

since usually all the utilities used by a CLI command are used together. Importing utility functions one at a time would be unnecessarily verbose, I think, and make it harder to see from a given part of code where the function is defined.

[neurobagel-bot]

🚀 Issue was released in v0.3.4 🚀