diff --git a/buildstockbatch/__version__.py b/buildstockbatch/__version__.py index 552e75f2..d4410152 100644 --- a/buildstockbatch/__version__.py +++ b/buildstockbatch/__version__.py @@ -4,9 +4,9 @@ __title__ = "buildstockbatch" __description__ = "Executing BuildStock projects on batch infrastructure." __url__ = "http://github.com/NREL/buildstockbatch" -__version__ = "2023.11.0" -__schema_version__ = "0.3" -__author__ = "Noel Merket" -__author_email__ = "noel.merket@nrel.gov" +__version__ = "2024.11.0" +__schema_version__ = "0.4" +__author__ = "Noel Merket, Rajendra Adhikari" +__author_email__ = "noel.merket@nrel.gov, rajendra.adhikari@nrel.gov" __license__ = "BSD-3" __copyright__ = "Copyright {} The Alliance for Sustainable Energy".format(dt.date.today().year) diff --git a/docs/changelog/changelog_2024_11_0.rst b/docs/changelog/changelog_2024_11_0.rst new file mode 100644 index 00000000..fdfe0bd7 --- /dev/null +++ b/docs/changelog/changelog_2024_11_0.rst @@ -0,0 +1,129 @@ +==================== +v2024.11.0 Changelog +==================== + +.. changelog:: + :version: v2024.11.0 + :released: 2024-11-05 + + + .. change:: + :tags: general + :pullreq: 421 + + Refactor docker_base to use inversion of control so that it can more strongly and easily ensure consistency + between various implementations (GCP implementation to come). This also includes teasing apart the several batch + prep steps (weather, assets, and jobs) into their own methods so they can each be more easily understood, + shared, and maintained. + + .. change:: + :tags: general + :pullreq: 422 + + Refactor AWS code so it can be shared by the upcoming GCP implementation. + + .. change:: + :tags: general, bugfix + :pullreq: 426 + + A bugfix for gracefully handling empty data_point_out.json files. + + .. change:: + :tags: aws, feature + :pullreq: 345 + + Major update to get AWS Batch run environment working. + + .. change:: + :tags: general + :pullreq: 435 + + Add helper to log a summary of how many simulations succeeded and failed at the end of a job. + + .. change:: + :tags: general, feature + :pullreq: 437 + + Add a ``step_failures`` section to json results files with error messages from OpenStudio simulations. + + .. change:: + :tags: general + :pullreq: 436 + + Clean up handling of weather files in GCP/AWS implementations: only upload files that are required, + and fail with clearer messaging if any files are missing. + + .. change:: + :tags: general, feature + :pullreq: 423 + + Add GCP implementation. + + .. change:: + :tags: bugfix, schema + :pullreq: 450 + + Requires ``os_version`` and ``os_sha`` in the project file. + + .. change:: + :tags: general + :pullreq: 456 + + Refactor WorkflowGenerator. + + .. change:: + :tags: general, feature + :pullreq: 458 + + Add a new version (2024.07.19) of the Residential HPXML Workflow Generator that + changes UpgradeCosts from reporting measure to a regular measure. ReportHPXMLOutput + is no longer called. This feature is created to support the corresponding update in + ResStock (https://github.com/NREL/resstock/pull/1253) + + To facilitate the creation of new version, the workflow generator code base is refactored + to have one folder for each version. New yaml schema (v0.4) is created that defines a + `version` key for the `workflow_generator` block. The base class loads the appropriate + version of the workflow_generator based on the version key. If version key is missing + (when using older schema), it is defaulted to the oldest available version (2024.07.18). + + .. change:: + :tags: general, feature + :pullreq: 461 + + Add a new version (2024.07.20) of the Residential HPXML Workflow Generator that + exposes optional ``include_annual_bills`` (defaults to true) and + ``include_monthly_bills`` (defaults to false) arguments for reporting annual + and monthly utility bill outputs, respectively. + + .. change:: + :tags: general, bugfix + :pullreq: 464 + + Stop creating dask _metadata files for the timeseries parquet files since it crashes the + postprocessing. + + .. change:: + :tags: bugfix, hpc + :pullreq: 467 + + Updates the bind mount for apptainer to include the whole resources + directory instead of just the hpxml-measures directory. Makes it work + with newer versions of ResStock. + + .. change:: + :tags: general + :pullreq: 472 + + Remove support for Python 3.8, 3.9, and 3.10. Add support for Python 3.12. + + .. change:: + :tags: general + :pullreq: 472 + + Remove eagle related code and references since eagle is retired. + + .. change:: + :tags: general + :pullreq: 465 + + Append schedules to the timeseries files. \ No newline at end of file diff --git a/docs/changelog/changelog_dev.rst b/docs/changelog/changelog_dev.rst index 006de449..75eeab3d 100644 --- a/docs/changelog/changelog_dev.rst +++ b/docs/changelog/changelog_dev.rst @@ -14,128 +14,3 @@ Development Changelog This is an example change. Please copy and paste it - for valid tags please refer to ``conf.py`` in the docs directory. ``pullreq`` should be set to the appropriate pull request number and ``tickets`` to any related github issues. These will be automatically linked in the documentation. - - .. change:: - :tags: general - :pullreq: 421 - - Refactor docker_base to use inversion of control so that it can more strongly and easily ensure consistency - between various implementations (GCP implementation to come). This also includes teasing apart the several batch - prep steps (weather, assets, and jobs) into their own methods so they can each be more easily understood, - shared, and maintained. - - .. change:: - :tags: general - :pullreq: 422 - - Refactor AWS code so it can be shared by the upcoming GCP implementation. - - .. change:: - :tags: general, bugfix - :pullreq: 426 - - A bugfix for gracefully handling empty data_point_out.json files. - - .. change:: - :tags: aws, feature - :pullreq: 345 - - Major update to get AWS Batch run environment working. - - .. change:: - :tags: general - :pullreq: 435 - - Add helper to log a summary of how many simulations succeeded and failed at the end of a job. - - .. change:: - :tags: general, feature - :pullreq: 437 - - Add a ``step_failures`` section to json results files with error messages from OpenStudio simulations. - - .. change:: - :tags: general - :pullreq: 436 - - Clean up handling of weather files in GCP/AWS implementations: only upload files that are required, - and fail with clearer messaging if any files are missing. - - .. change:: - :tags: general, feature - :pullreq: 423 - - Add GCP implementation. - - .. change:: - :tags: bugfix, schema - :pullreq: 450 - - Requires ``os_version`` and ``os_sha`` in the project file. - - .. change:: - :tags: general - :pullreq: 456 - - Refactor WorkflowGenerator. - - .. change:: - :tags: general, feature - :pullreq: 458 - - Add a new version (2024.07.19) of the Residential HPXML Workflow Generator that - changes UpgradeCosts from reporting measure to a regular measure. ReportHPXMLOutput - is no longer called. This feature is created to support the corresponding update in - ResStock (https://github.com/NREL/resstock/pull/1253) - - To facilitate the creation of new version, the workflow generator code base is refactored - to have one folder for each version. New yaml schema (v0.4) is created that defines a - `version` key for the `workflow_generator` block. The base class loads the appropriate - version of the workflow_generator based on the version key. If version key is missing - (when using older schema), it is defaulted to the oldest available version (2024.07.18). - - .. change:: - :tags: general, feature - :pullreq: 461 - - Add a new version (2024.07.20) of the Residential HPXML Workflow Generator that - exposes optional ``include_annual_bills`` (defaults to true) and - ``include_monthly_bills`` (defaults to false) arguments for reporting annual - and monthly utility bill outputs, respectively. - - .. change:: - :tags: general, bugfix - :pullreq: 464 - - Stop creating dask _metadata files for the timeseries parquet files since it crashes the - postprocessing. - - .. change:: - :tags: general, bugfix - :pullreq: 466 - - Add a flag to allow only uploading the missing files to s3. - :tags: bugfix, hpc - :pullreq: 467 - - Updates the bind mount for apptainer to include the whole resources - directory instead of just the hpxml-measures directory. Makes it work - with newer versions of ResStock. - - .. change:: - :tags: general - :pullreq: 472 - - Remove support for Python 3.8, 3.9, and 3.10. Add support for Python 3.12. - - .. change:: - :tags: general - :pullreq: 472 - - Remove eagle related code and references since eagle is retired. - - .. change:: - :tags: general - :pullreq: 465 - - Append schedules to the timeseries files. diff --git a/docs/changelog/index.rst b/docs/changelog/index.rst index 6a13c3fd..ff27f447 100644 --- a/docs/changelog/index.rst +++ b/docs/changelog/index.rst @@ -11,6 +11,7 @@ Current Migration Guide .. toctree:: :titlesonly: + migration_2024_11_0 migration_2023_11_0 Changelogs @@ -19,6 +20,7 @@ Changelogs .. toctree:: :titlesonly: + changelog_2024_11_0 changelog_2023_11_0 changelog_2023_10_0 changelog_2023_06_0 diff --git a/docs/changelog/migration_2024_11_0.rst b/docs/changelog/migration_2024_11_0.rst new file mode 100644 index 00000000..fb91e68a --- /dev/null +++ b/docs/changelog/migration_2024_11_0.rst @@ -0,0 +1,54 @@ +.. |version| replace:: v2024.11.0 + +======================================= +What's new in buildstockbatch |version| +======================================= + +.. admonition:: About this Document + + This document describes changes between buildstockbatch version 2023.11.0 and + buildstockbatch version |version| + +Major Changes +============= + +Below are the major changes in this release. + +Eagle is retired +---------------- + +This version should be backwards compatible with previous versions of +buildstockbatch except for being able to run in Eagle. Since Eagle is retired, BuildStockBatch no +longer supports buildstock_eagle command. If you are just comming upto speed with Kestrel, you can +use your existing yaml file by replacing 'eagle' with 'kestrel' and using buildstock_kestrel +command instead of buildstock_eagle command. See how to run in Kestrel :ref:`kestrel-run`. + +BuildStockBatch uses Python 3.11 +-------------------------------- + +BuildStockBatch now uses Python 3.11. If you are using a version of Python +older than 3.11 in your local buildstockbatch environment, you will need to upgrade the python +version or create a new environment with python 3.11 (or 3.12). + +Timeseries files include schedules +---------------------------------- + +In ResStock runs, the schedules are now appended to the timeseries files. + +WorkflowGenerator is versioned +------------------------------ + +`The workflow generator needs a version specification. `_ +Older yaml files without version specification will default to version 2024.07.18 (the assigned version before versioning was introduced). +If you are using a custom branch of ResStock, you should use the same version of the workflow generator as that in the ResStock branch +it is based on. + +Google Cloud Platform (GCP) support +----------------------------------- + +BuildStockBatch now supports GCP. See :ref:`gcp-config` and :ref:`gcp-run`. + + +See :doc:`changelog_2024_11_0` for full details of what changed. + + diff --git a/docs/run_sims.rst b/docs/run_sims.rst index 281a9c75..436fb52f 100644 --- a/docs/run_sims.rst +++ b/docs/run_sims.rst @@ -20,8 +20,10 @@ Running a project file is straightforward. Call the ``buildstock_local`` command Running the simulation with ``--postprocessonly`` when there is already postprocessed results from previous run will overwrite those results. -NREL HPC (Kestrel) -~~~~~~~~~~~~~~~~~~~ +.. _kestrel-run: + +Running on NREL HPC (Kestrel) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ After you have :ref:`activated the appropriate conda environment on Kestrel `, you can submit a project file to be simulated by passing it to the ``buildstock_kestrel`` command. @@ -72,8 +74,10 @@ fail. If most of the jobs succeeded, rather than rerun everything you can resubmit just the jobs that failed with the ``--rerun_failed`` command line argument. This will also clear out and rerun the postprocessing. -Amazon Web Services -~~~~~~~~~~~~~~~~~~~ +.. _aws-run: + +Running on Amazon Web Services +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Running a batch on AWS is done by calling the ``buildstock_aws`` command line tool. @@ -124,8 +128,10 @@ This will clean up all the AWS resources that were created on your behalf to run the simulations. Your results will still be on S3 and queryable in Athena. -Google Cloud Platform -~~~~~~~~~~~~~~~~~~~~~ +.. _gcp-run: + +Running on Google Cloud Platform +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Run a project on GCP by calling the ``buildstock_gcp`` command line tool.