manage multiple settings

[yw-fang]

Hi, all,

While we can use a ~/.quacc.yaml to set up the global settings for most calculations, is it possible to overwrite the settings for a specific project?
I tried several ways to make it (including redecorate, change_settings, etc), but all of them did not work.

Here is my .quacc.yalm used for user-worker split mode:

SCRATCH_DIR: /scratch/
CREATE_UNIQUE_DIR: false 
WORKFLOW_ENGINE: jobflow 

Now, in a specific project, I don't want to use any WORKFLOW_ENGINE. In addition, I'd like to run the quacc directly in the remote HPC and want to save the output of quacc to a particular database named "mydb"

from quacc import settings, change_settings
from quacc.recipes.vasp.core import static_job

atoms = bulk("Cu")
my_settings = {"WORKFLOW_ENGINE": None, "STORE": {"MongoStore": {"database": "mydb", "collection_name": "outputs", "username": "", "password": "", "host": "localhost", "port": 27017}}}
with change_settings(my_settings):
    result = static_job(atoms, kpts=[2,2,2])
    print(result)

I executed this script in the remote HPC, and it didn't run the vasp calculations, and only prints a uuid for a job:
Job(name='static_job', uuid='09853b2e-xxxx-42c5-xxxx-xxxxxxxxxx')
which indicated that the jobflow was still used.
Does anyone know how to change the settings properly? Thank you!

[Andrew-S-Rosen]

@yw-fang: Thank you for your question.

In general, the recommended approach to change settings for a given project would be to use environment variables, as noted in the settings documentation. For instance, if using a cluster, you would ensure that your SLURM script would have export QUACC_WORKFLOW_ENGINE=None in it before launching the calculation. This is important because the environment variables are being defined before the Python code is ever launched.

Or in your case, there is a simpler approach: strip the decorator.

from quacc import change_settings, strip_decorator
from quacc.recipes.vasp.core import static_job
from ase.build import bulk

atoms = bulk("Cu")
my_settings = {"STORE": {"MongoStore": {"database": "mydb", "collection_name": "outputs", "username": "", "password": "", "host": "localhost", "port": 27017}}}
static_job_ = strip_decorator(static_job)

with change_settings(my_settings):
    result = static_job_(atoms, kpts=[2,2,2])
    print(result)

Another approach would be to have multiple YAML files, and specify which YAML file you want to use using export QUACC_CONFIG_FILE.

Basically, the issue you are running into is that the redecorate and change_settings functions cannot change the workflow engine dynamically. This is because the workflow engine gets defined and used at import time. There is no way to get rid of that unless you strip the decorator off the Job in the first place --- after all, what you are trying to do here is run a simple Python function with no decorator logic at all.

I will open a PR that raises a clear error if a user puts WORKFLOW_ENGINE in the dict passed to change_settings or similar.

[yw-fang]

Hi, Andrew, thanks very much for the explanation unambiguously. I used "export QUACC_CONFIG_FILE" to get the expected settings