From b29d8005e33aadd4e9e57178101bc3ef9d0ca9bc Mon Sep 17 00:00:00 2001 From: Adam Nichols Date: Thu, 8 Aug 2024 16:38:38 -0400 Subject: [PATCH] WX-1798 Unbreak build by removing `swagger2markup` (#7488) --- .gitattributes | 1 - CHANGELOG.md | 42 +- docs/CommandLine.md | 4 +- docs/Imports.md | 3 +- docs/Modes.md | 2 +- docs/api/ErrorHandling.md | 55 - docs/api/RESTAPI.md | 1394 ----------------- docs/cromwell_features/Labels.md | 6 +- docs/developers/Contribute.md | 15 - docs/execution/ExecutionTwists.md | 2 +- docs/tutorials/ServerMode.md | 7 +- docs/wf_options/Overview.md | 2 +- .../src/main/resources/swagger/cromwell.yaml | 4 - mkdocs.yml | 3 - project/GenerateRestApiDocs.scala | 167 -- project/Publishing.scala | 2 +- project/Settings.scala | 2 - project/swagger2markup.sbt | 40 - src/ci/bin/testCheckPublish.sh | 2 +- 19 files changed, 39 insertions(+), 1714 deletions(-) delete mode 100644 docs/api/ErrorHandling.md delete mode 100644 docs/api/RESTAPI.md delete mode 100644 project/GenerateRestApiDocs.scala delete mode 100644 project/swagger2markup.sbt diff --git a/.gitattributes b/.gitattributes index d4b49d9edc9..9fcf38aa0a7 100644 --- a/.gitattributes +++ b/.gitattributes @@ -3,4 +3,3 @@ *.MD text *.java text *.html text -docs/api/RESTAPI.md linguist-generated=true diff --git a/CHANGELOG.md b/CHANGELOG.md index 8a01cc843d6..b5ab914e692 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,21 +7,9 @@ As of this version, a distribution of Java 17 is required to run Cromwell. Cromwell is developed, tested, and containerized using [Eclipse Temurin](https://adoptium.net/temurin/releases/?version=17). -### Fixed Optional and String Concatenation Bug - -As outlined in the [WDL Spec](https://github.com/openwdl/wdl/blob/main/versions/1.0/SPEC.md#prepending-a-string-to-an-optional-parameter), concatenating a string with an empty optional now correctly evaluates to the empty string. - ### Improved status reporting behavior -When Cromwell restarts during a workflow that is failing, it no longer reports pending tasks as a reason for that failure. - -### Removed Docker Hub health check - -Cromwell's healthcheck requests to Docker Hub were not authenticated, and thus became subject to rate limiting. To eliminate these false alarms, this functionality has been removed. - -The config key `services.HealthMonitor.config.check-dockerhub` is therefore obsolete. - -There is no change to any other usage of Docker Hub. +When Cromwell restarts during a workflow that is failing, it no longer reports pending tasks as a reason for that failure. ### Optional docker soft links @@ -41,16 +29,36 @@ Users reported cases where Life Sciences jobs failed due to insufficient quota, quota is available (which is the expected behavior). Cromwell will now retry under these conditions, which present with errors such as "PAPI error code 9", "no available zones", and/or "quota too low". -### Improved `size()` function performance on arrays - -Resolved a hotspot in Cromwell to make the `size()` engine function perform much faster on file arrays. Common examples of file arrays could include globs or scatter-gather results. This enhancement applies only to WDL 1.0 and later, because that's when `size()` added [support for arrays](https://github.com/openwdl/wdl/blob/main/versions/1.0/SPEC.md#acceptable-compound-input-types). - ### Database migration The `IX_WORKFLOW_STORE_ENTRY_WS` index is removed from `WORKFLOW_STORE_ENTRY`. The index had low cardinality and workflow pickup is faster without it. Migration time depends on workflow store size, but should be very fast for most installations. Terminal workflows are removed from the workflow store, so only running workflows contribute to the cost. +### Bug fixes + +#### Improved `size()` function performance on arrays + +Resolved a hotspot in Cromwell to make the `size()` engine function perform much faster on file arrays. Common examples of file arrays could include globs or scatter-gather results. This enhancement applies only to WDL 1.0 and later, because that's when `size()` added [support for arrays](https://github.com/openwdl/wdl/blob/main/versions/1.0/SPEC.md#acceptable-compound-input-types). + +#### Fixed Optional and String Concatenation Bug + +As outlined in the [WDL Spec](https://github.com/openwdl/wdl/blob/main/versions/1.0/SPEC.md#prepending-a-string-to-an-optional-parameter), concatenating a string with an empty optional now correctly evaluates to the empty string. + +### Removals + +#### `RESTAPI.md` docs discontinued + +Due to deprecation of the underlying library, Markdown docs will no longer be generated from the Cromwell API Swagger. The recommended alternative is starting a server and viewing the Swagger directly. + +#### Removed Docker Hub health check + +Cromwell's healthcheck requests to Docker Hub were not authenticated, and thus became subject to rate limiting. To eliminate these false alarms, this functionality has been removed. + +The config key `services.HealthMonitor.config.check-dockerhub` is therefore obsolete. + +There is no change to any other usage of Docker Hub. + ## 87 Release Notes ### GCP Batch diff --git a/docs/CommandLine.md b/docs/CommandLine.md index 932ac1503f5..a72767f50ac 100644 --- a/docs/CommandLine.md +++ b/docs/CommandLine.md @@ -34,7 +34,7 @@ The Cromwell jar file can be built as described in [Building](Building). ## `server` -`server` mode accepts no arguments and runs Cromwell as a web server that accepts REST requests. The default mode for most applications of Cromwell, suitable for production use. See the documentation for [Cromwell's REST endpoints](/api/RESTAPI) for how to interact with Cromwell in `server` mode. +`server` mode accepts no arguments and runs Cromwell as a web server that accepts REST requests. The default mode for most applications of Cromwell, suitable for production use. Cromwell self-documents its API with Swagger, viewable at `http://localhost:8000` or equivalent depending on your setup. ## `run` @@ -65,7 +65,7 @@ If you use sub-workflows within your primary workflow then you must include a ZI See the documentation on [Imports](Imports) for more information. * **`--metadata-output`** -You can specify a filename where Cromwell will write workflow metadata JSON such as start/end timestamps, status, inputs and outputs. By default Cromwell does not write workflow metadata. The metadata format in the `--metadata-output` file is the same as described for the [REST API](api/RESTAPI#get-workflow-and-call-level-metadata-for-a-specified-workflow). +You can specify a filename where Cromwell will write workflow metadata JSON such as start/end timestamps, status, inputs and outputs. By default Cromwell does not write workflow metadata. The metadata format in the `--metadata-output` file is the same as described for the `/metadata` endpoint. * **`--version`** The `--version` option prints the version of Cromwell and exits. diff --git a/docs/Imports.md b/docs/Imports.md index f929ada5883..f58ac0fe3c6 100644 --- a/docs/Imports.md +++ b/docs/Imports.md @@ -66,5 +66,4 @@ In [Run](Modes#run) mode, a sample command to run _workflow.wdl_ would be: $ java -jar cromwell.jar run workflow.wdl --imports imports.zip ``` -In [Server](Modes#server) mode, pass in a ZIP file using the parameter `workflowDependencies` via the [Submit](api/RESTAPI#submit-a-workflow-for-execution) endpoint. - +In [Server](Modes#server) mode, pass in a ZIP file using the parameter `workflowDependencies` via the Submit endpoint. diff --git a/docs/Modes.md b/docs/Modes.md index 5758d262309..fe77cf2bed0 100644 --- a/docs/Modes.md +++ b/docs/Modes.md @@ -2,7 +2,7 @@ The default mode for most applications of Cromwell, suitable for production use. Server mode starts Cromwell as a web server that exposes REST endpoints. All features and APIs are available. -By default the server will be accessible at `http://localhost:8000`. See the [Server Section](Configuring#server) of the configuration for more information on how to configure it. A description of the endpoints can be found in the [API Section](api/RESTAPI). +By default the server will be accessible at `http://localhost:8000`. See the [Server Section](Configuring#server) of the configuration for more information on how to configure it. Follow the [Server Tutorial](tutorials/ServerMode) to get your Cromwell server up and running in a few steps. diff --git a/docs/api/ErrorHandling.md b/docs/api/ErrorHandling.md deleted file mode 100644 index bd90b0471a9..00000000000 --- a/docs/api/ErrorHandling.md +++ /dev/null @@ -1,55 +0,0 @@ -Requests that Cromwell can't process return a failure in the form of a JSON response respecting the following JSON schema: - -``` -{ - "$schema": "http://json-schema.org/draft-04/schema#", - "description": "Error response schema", - "type": "object", - "properties": { - "status": { - "enum": [ "fail", "error"] - }, - "message": { - "type": "string" - }, - "errors": { - "type": "array", - "minItems": 1, - "items": { "type": "string" }, - "uniqueItems": true - } - }, - "required": ["status", "message"] -} -``` - -The `status` field can be `"fail"` or `"error"`. - -`"fail"` means that the request was invalid and/or data validation failed. `"fail"` status is most likely returned with a 4xx HTTP Status code. - -*For example,* - -``` -{ - "status": "fail", - "message": "Workflow input processing failed.", - "errors": [ - "Required workflow input 'helloworld.input' not specified." - ] -} -``` - -`"error"` means that an error occurred while processing the request. `"error"` status is most likely returned with a 5xx HTTP Status code. - -*For example,* - -``` -{ - "status": "error", - "message": "Connection to the database failed." -} -``` - -The `message` field contains a short description of the error. - -The `errors` field is optional and may contain additional information about why the request failed. \ No newline at end of file diff --git a/docs/api/RESTAPI.md b/docs/api/RESTAPI.md deleted file mode 100644 index 64ffe19a3fd..00000000000 --- a/docs/api/RESTAPI.md +++ /dev/null @@ -1,1394 +0,0 @@ - -# Cromwell Server REST API - - - -**Overview** -Describes the REST API provided by a Cromwell server - - -**Version information** -*Version* : 30 - - -**License information** -*License* : BSD -*License URL* : https://github.com/broadinstitute/cromwell/blob/develop/LICENSE.txt -*Terms of service* : null - - -**Produces** - -* `application/json` - - - - - - -## Run workflow -``` -POST /api/ga4gh/wes/v1/runs -``` - - -#### Description -This endpoint creates a new workflow run and returns a `RunId` to monitor its progress. - -The `workflow_attachment` array may be used to upload files that are required to execute the workflow, including the primary workflow, tools imported by the workflow, other files referenced by the workflow, or files which are part of the input. The implementation should stage these files to a temporary directory and execute the workflow from there. These parts must have a Content-Disposition header with a "filename" provided for each part. Filenames may include subdirectories, but must not include references to parent directories with '..' -- implementations should guard against maliciously constructed filenames. - -The `workflow_url` is either an absolute URL to a workflow file that is accessible by the WES endpoint, or a relative URL corresponding to one of the files attached using `workflow_attachment`. - -The `workflow_params` JSON object specifies input parameters, such as input files. The exact format of the JSON object depends on the conventions of the workflow language being used. Input files should either be absolute URLs, or relative URLs corresponding to files uploaded using `workflow_attachment`. The WES endpoint must understand and be able to access URLs supplied in the input. This is implementation specific. - -The `workflow_type` is the type of workflow language and must be "WDL" currently (or another alternative supported by this WES instance). - -The `workflow_type_version` is the version of the workflow language submitted and must be one supported by this WES instance. - -See the `RunRequest` documentation for details about other fields. - - -#### Parameters - -|Type|Name|Schema| -|---|---|---| -|**FormData**|**tags**
*optional*|string| -|**FormData**|**workflow_attachment**
*optional*|< file (binary) > array| -|**FormData**|**workflow_engine_parameters**
*optional*|string| -|**FormData**|**workflow_params**
*optional*|string| -|**FormData**|**workflow_type**
*optional*|string| -|**FormData**|**workflow_type_version**
*optional*|string| -|**FormData**|**workflow_url**
*optional*|string| - - -#### Responses - -|HTTP Code|Description|Schema| -|---|---|---| -|**201**|Successful Request|[RunId](#runid)| -|**400**|Malformed Workflow ID|No Content| -|**401**|Invalid submission request|No Content| -|**403**|Workflow in terminal status|No Content| -|**500**|Internal Error|No Content| - - -#### Consumes - -* `multipart/form-data` - - -#### Tags - -* GA4GH Workflow Execution Service (WES) - Alpha preview - - - -## List runs -``` -GET /api/ga4gh/wes/v1/runs -``` - - -#### Description -Runs are listed from newest to oldest. When paging through the list, the client should not make assumptions about live updates, but should assume the contents of the list reflect the workflow list at the moment that the first page is requested. To monitor a specific workflow run, use GetRunStatus or GetRunLog. - - -#### Parameters - -|Type|Name|Description|Schema| -|---|---|---|---| -|**Query**|**page_size**
*optional*|OPTIONAL The preferred number of workflow runs to return in a page. If not provided, the implementation should use a default page size. The implementation must not return more items than `page_size`, but it may return fewer. Clients should not assume that if fewer than `page_size` items are returned that all items have been returned. The availability of additional pages is indicated by the value of `next_page_token` in the response.|integer (int64)| -|**Query**|**page_token**
*optional*|OPTIONAL Token to use to indicate where to start getting results. If unspecified, return the first page of results.|string| - - -#### Responses - -|HTTP Code|Description|Schema| -|---|---|---| -|**200**|Successful Request|[RunListResponse](#runlistresponse)| -|**400**|Malformed Workflow ID|No Content| -|**401**|Invalid submission request|No Content| -|**403**|Workflow in terminal status|No Content| -|**500**|Internal Error|No Content| - - -#### Tags - -* GA4GH Workflow Execution Service (WES) - Alpha preview - - - -## Get run log -``` -GET /api/ga4gh/wes/v1/runs/{run_id} -``` - - -#### Description -This endpoint provides detailed information about a given workflow run. The returned result has information about the outputs produced by this workflow (if available), a log object which allows the stderr and stdout to be retrieved, a log array so stderr/stdout for individual tasks can be retrieved, and the overall state of the workflow run (e.g. RUNNING, see the State section). - - -#### Parameters - -|Type|Name|Schema| -|---|---|---| -|**Path**|**run_id**
*required*|string| - - -#### Responses - -|HTTP Code|Description|Schema| -|---|---|---| -|**200**|Successful Request|[RunLog](#runlog)| -|**401**|Invalid submission request|No Content| -|**403**|Workflow in terminal status|No Content| -|**404**|Workflow ID Not Found|No Content| -|**500**|Internal Error|No Content| - - -#### Tags - -* GA4GH Workflow Execution Service (WES) - Alpha preview - - - -## Cancel run -``` -POST /api/ga4gh/wes/v1/runs/{run_id}/cancel -``` - - -#### Description -Cancel a running workflow. - - -#### Parameters - -|Type|Name|Description|Schema| -|---|---|---|---| -|**Path**|**run_id**
*required*|Run ID|string| - - -#### Responses - -|HTTP Code|Description|Schema| -|---|---|---| -|**200**|Successful Request|[RunId](#runid)| -|**401**|Invalid submission request|No Content| -|**403**|Workflow in terminal status|No Content| -|**404**|Workflow ID Not Found|No Content| -|**500**|Internal Error|No Content| - - -#### Tags - -* GA4GH Workflow Execution Service (WES) - Alpha preview - - - -## Get run status -``` -GET /api/ga4gh/wes/v1/runs/{run_id}/status -``` - - -#### Description -This provides an abbreviated (and likely fast depending on implementation) status of the running workflow, returning a simple result with the overall state of the workflow run (e.g. RUNNING, see the State section). - - -#### Parameters - -|Type|Name|Description|Schema| -|---|---|---|---| -|**Path**|**run_id**
*required*|Run ID|string| - - -#### Responses - -|HTTP Code|Description|Schema| -|---|---|---| -|**200**|Successful Request|[RunStatus](#runstatus)| -|**401**|Invalid submission request|No Content| -|**403**|Workflow in terminal status|No Content| -|**404**|Workflow ID Not Found|No Content| -|**500**|Internal Error|No Content| - - -#### Tags - -* GA4GH Workflow Execution Service (WES) - Alpha preview - - - -## Get service info -``` -GET /api/ga4gh/wes/v1/service-info -``` - - -#### Description -May include information related (but not limited to) the workflow descriptor formats, versions supported, the WES API versions supported, and information about general service availability. - - -#### Responses - -|HTTP Code|Description|Schema| -|---|---|---| -|**200**|Successful Request|[ServiceInfo](#serviceinfo)| -|**500**|Internal Error|No Content| - - -#### Tags - -* GA4GH Workflow Execution Service (WES) - Alpha preview - - - -## Machine-readable description of a workflow, including inputs and outputs -``` -POST /api/womtool/{version}/describe -``` - - -#### Parameters - -|Type|Name|Description|Schema|Default| -|---|---|---|---|---| -|**Path**|**version**
*required*|Cromwell API Version|string|`"v1"`| -|**FormData**|**workflowInputs**
*optional*|JSON or YAML file containing the inputs as an object.|file|| -|**FormData**|**workflowSource**
*optional*|The workflow source file to submit for execution. Either workflow source or workflow url is required.|file|| -|**FormData**|**workflowType**
*optional*|The workflow language for the file you submitted. Cromwell currently supports WDL.|enum (WDL)|| -|**FormData**|**workflowTypeVersion**
*optional*|The specification version for the workflow language being used. For WDL, Cromwell currently supports draft-2 and 1.0.|enum (draft-2, 1.0)|| -|**FormData**|**workflowUrl**
*optional*|URL which points to the workflow. Either workflow source or workflow url is required.|string|| - - -#### Responses - -|HTTP Code|Description|Schema| -|---|---|---| -|**200**|Workflow description.|[WorkflowDescription](#workflowdescription)| - - -#### Consumes - -* `multipart/form-data` - - -#### Tags - -* Womtool - - - -## Submit a workflow for execution -``` -POST /api/workflows/{version} -``` - - -#### Description -Submits a workflow to Cromwell. Note that this endpoint can accept an unlimited number of input files via workflowInputs_N but swagger needs them to be explicitly defined so we have provided 5 as an example. - - -#### Parameters - -|Type|Name|Description|Schema|Default| -|---|---|---|---|---| -|**Path**|**version**
*required*|Cromwell API Version|string|`"v1"`| -|**FormData**|**labels**
*optional*|JSON object of labels to apply to this workflow.|file|| -|**FormData**|**requestedWorkflowId**
*optional*|An ID to ascribe to this workflow. Must be a JSON string in UUID-format. If not supplied a random ID will be generated for the workflow.|string|| -|**FormData**|**workflowDependencies**
*optional*|ZIP file containing workflow source files that are used to resolve local imports. This zip bundle will be unpacked in a sandbox accessible to this workflow.|file|| -|**FormData**|**workflowInputs**
*optional*|JSON or YAML file containing the inputs as an object. For WDL workflows a skeleton file can be generated from WOMtool using the "inputs" subcommand. When multiple files are specified, in case of key conflicts between multiple input JSON files, higher values of x in workflowInputs_x override lower values. For example, an input specified in workflowInputs_3 will override an input with the same name in workflowInputs or workflowInputs_2. Similarly, an input key specified in workflowInputs_5 will override an identical input key in any other input file.|file|| -|**FormData**|**workflowInputs_2**
*optional*|A second JSON or YAML file containing inputs.|file|| -|**FormData**|**workflowInputs_3**
*optional*|A third JSON or YAML file containing inputs.|file|| -|**FormData**|**workflowInputs_4**
*optional*|A fourth JSON or YAML file containing inputs.|file|| -|**FormData**|**workflowInputs_5**
*optional*|A fifth JSON or YAML file containing inputs.|file|| -|**FormData**|**workflowOnHold**
*optional*|Put workflow on hold upon submission. By default, it is taken as false.|boolean|| -|**FormData**|**workflowOptions**
*optional*|JSON file containing configuration options for the execution of this workflow.|file|| -|**FormData**|**workflowSource**
*optional*|The workflow source file to submit for execution. Either workflow source or workflow url is required.|file|| -|**FormData**|**workflowType**
*optional*|The workflow language for the file you submitted. Cromwell currently supports WDL.|enum (WDL)|| -|**FormData**|**workflowTypeVersion**
*optional*|The specification version for the workflow language being used. For WDL, Cromwell currently supports draft-2 and 1.0.|enum (draft-2, 1.0)|| -|**FormData**|**workflowUrl**
*optional*|URL which points to the workflow. Either workflow source or workflow url is required.|string|| - - -#### Responses - -|HTTP Code|Description|Schema| -|---|---|---| -|**201**|Successful Request|[WorkflowIdAndStatus](#workflowidandstatus)| -|**400**|Invalid submission request|No Content| -|**500**|Internal Error|No Content| - - -#### Consumes - -* `multipart/form-data` - - -#### Tags - -* Workflows - - - -## List the supported backends -``` -GET /api/workflows/{version}/backends -``` - - -#### Description -Returns the backends supported by this Cromwell server, as well as the default backend. - - -#### Parameters - -|Type|Name|Description|Schema|Default| -|---|---|---|---|---| -|**Path**|**version**
*required*|Cromwell API Version|string|`"v1"`| - - -#### Responses - -|HTTP Code|Description|Schema| -|---|---|---| -|**200**|Successful Request|[BackendResponse](#backendresponse)| - - -#### Tags - -* Workflows - - - -## Submit a batch of workflows for execution -``` -POST /api/workflows/{version}/batch -``` - - -#### Description -In instances where you want to run the same workflow multiple times with varying inputs you may submit a workflow batch. This endpoint is fundamentally the same as the standard submission endpoint with the exception that the inputs JSON will be an array of objects instead of a single object. - - -#### Parameters - -|Type|Name|Description|Schema|Default| -|---|---|---|---|---| -|**Path**|**version**
*required*|Cromwell API Version|string|`"v1"`| -|**FormData**|**labels**
*optional*|JSON object of labels to apply to this workflow.|file|| -|**FormData**|**requestedWorkflowId**
*optional*|A set of IDs to ascribe to these workflows. Must be a JSON list of strings in UUID-format. Must have the same number of entries and be in the same order as the workflow inputs list. If not supplied, random ID will be generated for the workflows.|string|| -|**FormData**|**workflowDependencies**
*optional*|ZIP file containing workflow source files that are used to resolve local imports. This zip bundle will be unpacked in a sandbox accessible to these workflows.|file|| -|**FormData**|**workflowInputs**
*required*|JSON file containing the inputs as an array of objects. Every element of the array will correspond to a single workflow. For WDL workflows a skeleton file can be generated from WOMtool using the "inputs" subcommand. When multiple files are specified, in case of key conflicts between multiple input JSON files, higher values of x in workflowInputs_x override lower values. For example, an input specified in workflowInputs_3 will override an input with the same name in workflowInputs or workflowInputs_2. Similarly, an input key specified in workflowInputs_5 will override an identical input key in any other input file.|file|| -|**FormData**|**workflowOnHold**
*optional*|Put workflow on hold upon submission. By default, it is taken as false.|boolean|| -|**FormData**|**workflowOptions**
*optional*|JSON file containing configuration options for the execution of this workflow.|file|| -|**FormData**|**workflowSource**
*optional*|The workflow source file to submit for execution. Either workflow source or workflow url is required.|file|| -|**FormData**|**workflowType**
*optional*|The workflow language for the file you submitted. Cromwell currently supports WDL.|enum (WDL)|| -|**FormData**|**workflowTypeVersion**
*optional*|The specification version for the workflow language being used. For WDL, Cromwell currently supports draft-2 and 1.0.|enum (draft-2, 1.0)|| -|**FormData**|**workflowUrl**
*optional*|URL which points to the workflow. Either workflow source or workflow url is required.|string|| - - -#### Responses - -|HTTP Code|Description|Schema| -|---|---|---| -|**200**|Successful Request|< [WorkflowIdAndStatus](#workflowidandstatus) > array| -|**400**|Malformed Workflow ID|No Content| -|**500**|Internal Error|No Content| - - -#### Consumes - -* `multipart/form-data` - - -#### Tags - -* Workflows - - - -## Explain hashing differences for 2 calls -``` -GET /api/workflows/{version}/callcaching/diff -``` - - -#### Description -This endpoint returns the hash differences between 2 completed (successfully or not) calls. - - -#### Parameters - -|Type|Name|Description|Schema|Default| -|---|---|---|---|---| -|**Path**|**version**
*required*|Cromwell API Version|string|`"v1"`| -|**Query**|**callA**
*required*|Fully qualified name, including workflow name, of the first call.|string|| -|**Query**|**callB**
*required*|Fully qualified name, including workflow name, of the second call|string|| -|**Query**|**indexA**
*optional*|Shard index for the first call for cases where the requested call was part of a scatter.|integer|| -|**Query**|**indexB**
*optional*|Shard index for the second call for cases where the requested call was part of a scatter.|integer|| -|**Query**|**workflowA**
*required*|Workflow Id of the first workflow|string|| -|**Query**|**workflowB**
*required*|Workflow Id of the second workflow|string|| - - -#### Responses - -|HTTP Code|Description|Schema| -|---|---|---| -|**200**|Successful Request|[WorkflowIdAndStatus](#workflowidandstatus)| -|**400**|Malformed Workflow ID|No Content| -|**404**|No matching cache entry. Cromwell versions prior to 28 will not have recorded information necessary for this endpoint and thus will also appear to not exist.|No Content| -|**500**|Internal Error|No Content| - - -#### Tags - -* Workflows - - - -## Get workflows matching some criteria -``` -POST /api/workflows/{version}/query -``` - - -#### Description -Query workflows by start dates, end dates, names, ids, labels, or statuses. - - -#### Parameters - -|Type|Name|Description|Schema|Default| -|---|---|---|---|---| -|**Path**|**version**
*required*|Cromwell API Version|string|`"v1"`| -|**Body**|**parameters**
*required*|Same query parameters as GET /query endpoint, submitted as a json list. Example: [{"status":"Success"},{"status":"Failed"}]|< [WorkflowQueryParameter](#workflowqueryparameter) > array|| - - -#### Responses - -|HTTP Code|Description|Schema| -|---|---|---| -|**200**|Successful Request|[WorkflowQueryResponse](#workflowqueryresponse)| -|**400**|Malformed Workflow ID|No Content| -|**500**|Internal Error|No Content| - - -#### Tags - -* Workflows - - - -## Get workflows matching some criteria -``` -GET /api/workflows/{version}/query -``` - - -#### Description -Query for workflows which match various criteria. When a combination of criteria are applied the endpoint will return - - -#### Parameters - -|Type|Name|Description|Schema|Default| -|---|---|---|---|---| -|**Path**|**version**
*required*|Cromwell API Version|string|`"v1"`| -|**Query**|**additionalQueryResultFields**
*optional*|Currently only 'labels' is a valid value here. Use it to include a list of labels with each result.|< string > array(multi)|| -|**Query**|**end**
*optional*|Returns only workflows with an equal or earlier end datetime. Can be specified at most once. If both start and end date are specified, start date must be before or equal to end date.|string (date-time)|| -|**Query**|**excludeLabelAnd**
*optional*|Excludes workflows with the specified label. If specified multiple times, excludes workflows with all of the specified label keys. Specify the label key and label value pair as separated with "label-key:label-value"|< string > array(multi)|| -|**Query**|**excludeLabelOr**
*optional*|Excludes workflows with the specified label. If specified multiple times, excludes workflows with any of the specified label keys. Specify the label key and label value pair as separated with "label-key:label-value"|< string > array(multi)|| -|**Query**|**id**
*optional*|Returns only workflows with the specified workflow id. If specified multiple times, returns workflows with any of the specified workflow ids.|< string > array(multi)|| -|**Query**|**includeSubworkflows**
*optional*|Include subworkflows in results. By default, it is taken as true.|boolean (boolean)|| -|**Query**|**label**
*optional*|Returns workflows with the specified label keys. If specified multiple times, returns workflows with all of the specified label keys. Specify the label key and label value pair as separated with "label-key:label-value"|< string > array(multi)|| -|**Query**|**labelor**
*optional*|Returns workflows with the specified label keys. If specified multiple times, returns workflows with any of the specified label keys. Specify the label key and label value pair as separated with "label-key:label-value"|< string > array(multi)|| -|**Query**|**name**
*optional*|Returns only workflows with the specified name. If specified multiple times, returns workflows with any of the specified names.|< string > array(multi)|| -|**Query**|**start**
*optional*|Returns only workflows with an equal or later start datetime. Can be specified at most once. If both start and end date are specified, start date must be before or equal to end date.|string (date-time)|| -|**Query**|**status**
*optional*|Returns only workflows with the specified status. If specified multiple times, returns workflows in any of the specified statuses.|< string > array(multi)|| -|**Query**|**submission**
*optional*|Returns only workflows with an equal or later submission time. Can be specified at most once. If both submission time and start date are specified, submission time should be before or equal to start date.|string (date-time)|| - - -#### Responses - -|HTTP Code|Description|Schema| -|---|---|---| -|**200**|Successful Request|[WorkflowQueryResponse](#workflowqueryresponse)| -|**403**|Workflow in terminal status|No Content| -|**404**|Workflow ID Not Found|No Content| -|**500**|Internal Error|No Content| - - -#### Tags - -* Workflows - - - -## Abort a running workflow -``` -POST /api/workflows/{version}/{id}/abort -``` - - -#### Description -Request Cromwell to abort a running workflow. For instance this might be necessary in cases where you have submitted a workflow with incorrect inputs or no longer need the results. Cromwell will schedule a halt of all currently running jobs from this workflow. - - -#### Parameters - -|Type|Name|Description|Schema|Default| -|---|---|---|---|---| -|**Path**|**id**
*required*|A workflow ID|string|| -|**Path**|**version**
*required*|Cromwell API Version|string|`"v1"`| - - -#### Responses - -|HTTP Code|Description|Schema| -|---|---|---| -|**200**|Successful Request|[WorkflowIdAndStatus](#workflowidandstatus)| -|**400**|Malformed Workflow ID|No Content| -|**403**|Workflow in terminal status|No Content| -|**404**|Workflow ID Not Found|No Content| -|**500**|Internal Error|No Content| - - -#### Tags - -* Workflows - - - -## Retrieves the current labels for a workflow -``` -GET /api/workflows/{version}/{id}/labels -``` - - -#### Parameters - -|Type|Name|Description|Schema|Default| -|---|---|---|---|---| -|**Path**|**id**
*required*|A workflow ID|string|| -|**Path**|**version**
*required*|Cromwell API Version|string|`"v1"`| - - -#### Responses - -|HTTP Code|Description|Schema| -|---|---|---| -|**200**|Successful Request|[LabelsResponse](#labelsresponse)| -|**400**|Malformed Workflow ID|No Content| -|**404**|Workflow ID Not Found|No Content| -|**500**|Internal Error|No Content| - - -#### Tags - -* Workflows - - - -## Update labels for a workflow -``` -PATCH /api/workflows/{version}/{id}/labels -``` - - -#### Description -Update multiple labels for an existing workflow. When supplying a label with a key unique to the workflow submission, a new label key/value entry is appended to that workflow's metadata. When supplying a label with a key that is already associated to the workflow submission, the original label value is updated with the new value for that workflow's metadata. - - -#### Parameters - -|Type|Name|Description|Schema|Default| -|---|---|---|---|---| -|**Path**|**id**
*required*|Workflow ID|string|| -|**Path**|**version**
*required*|Cromwell API Version|string|`"v1"`| -|**Body**|**labels**
*required*|Custom labels submitted as JSON. Example: {"key-1":"value-1","key-2":"value-2"}|object|| - - -#### Responses - -|HTTP Code|Description|Schema| -|---|---|---| -|**200**|Successful Request|[LabelsResponse](#labelsresponse)| -|**400**|Malformed Workflow ID|No Content| -|**403**|Workflow in terminal status|No Content| -|**404**|Workflow ID Not Found|No Content| -|**500**|Internal Error|No Content| - - -#### Tags - -* Workflows - - - -## Get the logs for a workflow -``` -GET /api/workflows/{version}/{id}/logs -``` - - -#### Description -Returns paths to the standard out and standard error files that were generated during the execution of all calls in a workflow. A call has one or more standard out and standard error logs, depending on if the call was scattered or not. In the latter case, one log is provided for each instance of the call that has been run. - - -#### Parameters - -|Type|Name|Description|Schema|Default| -|---|---|---|---|---| -|**Path**|**id**
*required*|A workflow ID|string|| -|**Path**|**version**
*required*|Cromwell API Version|string|`"v1"`| - - -#### Responses - -|HTTP Code|Description|Schema| -|---|---|---| -|**200**|Successful Request|[WorkflowIdAndStatus](#workflowidandstatus)| -|**400**|Malformed Workflow ID|No Content| -|**404**|Workflow ID Not Found|No Content| -|**500**|Internal Error|No Content| - - -#### Tags - -* Workflows - - - -## Get workflow and call-level metadata for a specified workflow -``` -GET /api/workflows/{version}/{id}/metadata -``` - - -#### Parameters - -|Type|Name|Description|Schema|Default| -|---|---|---|---|---| -|**Path**|**id**
*required*|A workflow ID|string|| -|**Path**|**version**
*required*|Cromwell API Version|string|`"v1"`| -|**Query**|**excludeKey**
*optional*|When specified, filters metadata to not return any field with a name which begins with this value. This key is used relative to the root of the response *and* relative to each call's metadata fields. Use 'calls' to filter out all call level metadata.|< string > array(multi)|| -|**Query**|**expandSubWorkflows**
*optional*|When true, metadata for sub workflows will be fetched and inserted automatically in the metadata response.|boolean|| -|**Query**|**includeKey**
*optional*|When specified, filters metadata to only return fields with names which begins with this value. This key is used relative to the root of the response *and* relative to each call's metadata fields.|< string > array(multi)|| - - -#### Responses - -|HTTP Code|Description|Schema| -|---|---|---| -|**200**|Successful Request|[WorkflowMetadataResponse](#workflowmetadataresponse)| -|**400**|Malformed Workflow ID|No Content| -|**404**|Workflow ID Not Found|No Content| -|**500**|Internal Error|No Content| - - -#### Tags - -* Workflows - - - -## Get call-level metadata of failed tasks for a specified root workflow -``` -GET /api/workflows/{version}/{id}/metadata/failed-jobs -``` - - -#### Parameters - -|Type|Name|Description|Schema|Default| -|---|---|---|---|---| -|**Path**|**id**
*required*|A workflow ID|string|| -|**Path**|**version**
*required*|Cromwell API Version|string|`"v1"`| - - -#### Responses - -|HTTP Code|Description|Schema| -|---|---|---| -|**200**|Successful request|[WorkflowMetadataResponse](#workflowmetadataresponse)| -|**400**|Malformed Workflow ID|No Content| -|**404**|Workflow ID Not Found|No Content| -|**500**|Internal Error|No Content| - - -#### Tags - -* Workflows - - - -## Get the outputs for a workflow -``` -GET /api/workflows/{version}/{id}/outputs -``` - - -#### Description -Retrieve the outputs for the specified workflow. Cromwell will return any outputs which currently exist even if a workflow has not successfully completed. - - -#### Parameters - -|Type|Name|Description|Schema|Default| -|---|---|---|---|---| -|**Path**|**id**
*required*|A workflow ID|string|| -|**Path**|**version**
*required*|Cromwell API Version|string|`"v1"`| - - -#### Responses - -|HTTP Code|Description|Schema| -|---|---|---| -|**200**|Successful Request|[WorkflowIdAndStatus](#workflowidandstatus)| -|**400**|Malformed Workflow ID|No Content| -|**404**|Workflow ID Not Found|No Content| -|**500**|Internal Error|No Content| - - -#### Tags - -* Workflows - - - -## Switch a workflow from 'On Hold' to 'Submitted' status -``` -POST /api/workflows/{version}/{id}/releaseHold -``` - - -#### Description -Request Cromwell to release the hold on a workflow. It will switch the status of a workflow from 'On Hold' to 'Submitted' so it can be picked for running. For instance this might be necessary in cases where you have submitted a workflow with workflowOnHold = true. - - -#### Parameters - -|Type|Name|Description|Schema|Default| -|---|---|---|---|---| -|**Path**|**id**
*required*|A workflow ID|string|| -|**Path**|**version**
*required*|Cromwell API Version|string|`"v1"`| - - -#### Responses - -|HTTP Code|Description|Schema| -|---|---|---| -|**200**|Successful Request|[WorkflowIdAndStatus](#workflowidandstatus)| -|**400**|Malformed Workflow ID|No Content| -|**403**|Workflow in terminal status|No Content| -|**404**|Workflow ID Not Found|No Content| -|**500**|Internal Error|No Content| - - -#### Tags - -* Workflows - - - -## Retrieves the current state for a workflow -``` -GET /api/workflows/{version}/{id}/status -``` - - -#### Parameters - -|Type|Name|Description|Schema|Default| -|---|---|---|---|---| -|**Path**|**id**
*required*|A workflow ID|string|| -|**Path**|**version**
*required*|Cromwell API Version|string|`"v1"`| - - -#### Responses - -|HTTP Code|Description|Schema| -|---|---|---| -|**200**|Successful Request|[WorkflowIdAndStatus](#workflowidandstatus)| -|**400**|Malformed Workflow ID|No Content| -|**404**|Workflow ID Not Found|No Content| -|**500**|Internal Error|No Content| - - -#### Tags - -* Workflows - - - -## Get a visual diagram of a running workflow -``` -GET /api/workflows/{version}/{id}/timing -``` - - -#### Description -Returns a javascript file which will render a Gantt chart for the requested workflow. The bars in the chart represent start and end times for individual task invocations. This javascript is intended to be embedded into another web page. - - -#### Parameters - -|Type|Name|Description|Schema|Default| -|---|---|---|---|---| -|**Path**|**id**
*required*|A workflow ID|string|| -|**Path**|**version**
*required*|Cromwell API Version|string|`"v1"`| - - -#### Responses - -|HTTP Code|Description|Schema| -|---|---|---| -|**200**|Successful Request|[WorkflowIdAndStatus](#workflowidandstatus)| -|**400**|Malformed Workflow ID|No Content| -|**404**|Workflow ID Not Found|No Content| -|**500**|Internal Error|No Content| - - -#### Tags - -* Workflows - - - -## Return the current health status of any monitored subsystems -``` -GET /engine/{version}/status -``` - - -#### Parameters - -|Type|Name|Description|Schema|Default| -|---|---|---|---|---| -|**Path**|**version**
*required*|Cromwell API Version|string|`"v1"`| - - -#### Responses - -|HTTP Code|Description|Schema| -|---|---|---| -|**200**|All subsystems report an "ok" status|[StatusResponse](#statusresponse)| -|**500**|At least one subsystem does not have an "ok" status|[StatusResponse](#statusresponse)| - - -#### Tags - -* Engine - - - -## Return the version of this Cromwell server -``` -GET /engine/{version}/version -``` - - -#### Parameters - -|Type|Name|Description|Schema|Default| -|---|---|---|---|---| -|**Path**|**version**
*required*|Cromwell API Version|string|`"v1"`| - - -#### Responses - -|HTTP Code|Description|Schema| -|---|---|---| -|**200**|Successful Request|[VersionResponse](#versionresponse)| - - -#### Tags - -* Engine - - - -## Definitions - - -### BackendResponse - -|Name|Description|Schema| -|---|---|---| -|**defaultBackend**
*required*|The default backend of this server|string| -|**supportedBackends**
*required*|The backends supported by this server|< string > array| - - - -### CallMetadata -Call level metadata - - -|Name|Description|Schema| -|---|---|---| -|**backend**
*optional*|The type of backend on which the call executed (e.g. JES, SGE, Local)|string| -|**backendLogs**
*optional*|Paths to backend specific logs for this call|object| -|**backendStatus**
*optional*|Status in backend-specific terms. Currently this will only be defined for the JES backend.|string| -|**end**
*optional*|End datetime of the call execution in ISO8601 format with milliseconds|string (date-time)| -|**executionStatus**
*optional*|Status in Cromwell execution terms.|string| -|**failures**
*optional*||< [FailureMessage](#failuremessage) > array| -|**inputs**
*optional*|Mapping of input fully qualified names to stringified values|object| -|**jobId**
*optional*|Backend-specific job ID|string| -|**returnCode**
*optional*|Call execution return code|integer| -|**start**
*optional*|Start datetime of the call execution in ISO8601 format with milliseconds|string (date-time)| -|**stderr**
*optional*|Path to the standard error file for this call|string| -|**stdout**
*optional*|Path to the standard output file for this call|string| - - - -### DefaultWorkflowEngineParameter -A message that allows one to describe default parameters for a workflow engine. - - -|Name|Description|Schema| -|---|---|---| -|**default_value**
*optional*|The stringified version of the default parameter. e.g. "2.45".|string| -|**name**
*optional*|The name of the parameter|string| -|**type**
*optional*|Describes the type of the parameter, e.g. float.|string| - - - -### DescriptorType -One from a list of descriptor type strings (e.g. WDL). Note that these files can also include associated Docker/container files and test parameters that further describe a version of a tool - -*Type* : enum (WDL) - - - -### DescriptorTypeAndVersion -A workflow descriptor file type and version. - - -|Name|Description|Schema| -|---|---|---| -|**descriptorType**
*required*||[DescriptorType](#descriptortype)| -|**descriptorTypeVersion**
*required*|**Example** : `"1.0"`|string| - - - -### FailureMessage -Failure messages - - -|Name|Description|Schema| -|---|---|---| -|**causedBy**
*optional*||< [FailureMessage](#failuremessage) > array| -|**message**
*optional*|The failure message|string| - - - -### LabelsResponse - -|Name|Description|Schema| -|---|---|---| -|**id**
*required*|The identifier of the workflow
**Example** : `"label-key-1"`|string| -|**labels**
*required*|The labels which have been updated
**Example** : `"label-value-1"`|string| - - - -### Log -Log and other info - - -|Name|Description|Schema| -|---|---|---| -|**cmd**
*optional*|The command line that was executed|< string > array| -|**end_time**
*optional*|When the command stopped executing (completed, failed, or cancelled), in ISO 8601 format "%Y-%m-%dT%H:%M:%SZ"|string| -|**exit_code**
*optional*|Exit code of the program|integer (int32)| -|**name**
*optional*|The task or workflow name|string| -|**start_time**
*optional*|When the command started executing, in ISO 8601 format "%Y-%m-%dT%H:%M:%SZ"|string| -|**stderr**
*optional*|A URL to retrieve standard error logs of the workflow run or task. This URL may change between status requests, or may not be available until the task or workflow has finished execution. Should be available using the same credentials used to access the WES endpoint.|string| -|**stdout**
*optional*|A URL to retrieve standard output logs of the workflow run or task. This URL may change between status requests, or may not be available until the task or workflow has finished execution. Should be available using the same credentials used to access the WES endpoint.|string| - - - -### MapValueType -A type representing a map from one type to another. - - -|Name|Schema| -|---|---| -|**keyType**
*required*|[ValueType](#valuetype)| -|**valueType**
*required*|[ValueType](#valuetype)| - - - -### RunId - -|Name|Description|Schema| -|---|---|---| -|**run_id**
*optional*|workflow run ID|string| - - - -### RunListResponse -The service will return a RunListResponse when receiving a successful RunListRequest. - - -|Name|Description|Schema| -|---|---|---| -|**next_page_token**
*optional*|A token which may be supplied as `page_token` in workflow run list request to get the next page of results. An empty string indicates there are no more items to return.|string| -|**runs**
*optional*|A list of workflow runs that the service has executed or is executing. The list is filtered to only include runs that the caller has permission to see.|< [RunStatus](#runstatus) > array| - - - -### RunLog - -|Name|Description|Schema| -|---|---|---| -|**outputs**
*optional*|The outputs from the workflow run.|object| -|**request**
*optional*||[RunRequest](#runrequest)| -|**run_id**
*optional*|workflow run ID|string| -|**run_log**
*optional*||[Log](#log)| -|**state**
*optional*||[State](#state)| -|**task_logs**
*optional*|The logs, and other key info like timing and exit code, for each step in the workflow run.|< [Log](#log) > array| - - - -### RunRequest -To execute a workflow, send a run request including all the details needed to begin downloading and executing a given workflow. - - -|Name|Description|Schema| -|---|---|---| -|**tags**
*optional*||< string, string > map| -|**workflow_engine_parameters**
*optional*||< string, string > map| -|**workflow_params**
*optional*|REQUIRED The workflow run parameterizations (JSON encoded), including input and output file locations|object| -|**workflow_type**
*optional*|REQUIRED The workflow descriptor type, must be "WDL" currently (or another alternative supported by this WES instance)|string| -|**workflow_type_version**
*optional*|REQUIRED The workflow descriptor type version, must be one supported by this WES instance|string| -|**workflow_url**
*optional*|REQUIRED The workflow WDL document. When `workflow_attachments` is used to attach files, the `workflow_url` may be a relative path to one of the attachments.|string| - - - -### RunStatus -Small description of a workflow run, returned by server during listing - - -|Name|Schema| -|---|---| -|**run_id**
*required*|string| -|**state**
*optional*|[State](#state)| - - - -### Service -GA4GH service - - -|Name|Description|Schema| -|---|---|---| -|**contactUrl**
*optional*|URL of the contact for the provider of this service, e.g. a link to a contact form (RFC 3986 format), or an email (RFC 2368 format).
**Example** : `"mailto:support@example.com"`|string (uri)| -|**createdAt**
*optional*|Timestamp describing when the service was first deployed and available (RFC 3339 format)
**Example** : `"2019-06-04T12:58:19Z"`|string (date-time)| -|**description**
*optional*|Description of the service. Should be human readable and provide information about the service.
**Example** : `"This service provides..."`|string| -|**documentationUrl**
*optional*|URL of the documentation of this service (RFC 3986 format). This should help someone learn how to use your service, including any specifics required to access data, e.g. authentication.
**Example** : `"https://docs.myservice.example.com"`|string (uri)| -|**environment**
*optional*|Environment the service is running in. Use this to distinguish between production, development and testing/staging deployments. Suggested values are prod, test, dev, staging. However this is advised and not enforced.
**Example** : `"test"`|string| -|**id**
*required*|Unique ID of this service. Reverse domain name notation is recommended, though not required. The identifier should attempt to be globally unique so it can be used in downstream aggregator services e.g. Service Registry.
**Example** : `"org.ga4gh.myservice"`|string| -|**name**
*required*|Name of this service. Should be human readable.
**Example** : `"My project"`|string| -|**organization**
*required*|Organization providing the service|[organization](#service-organization)| -|**type**
*required*||[ServiceType](#servicetype)| -|**updatedAt**
*optional*|Timestamp describing when the service was last updated (RFC 3339 format)
**Example** : `"2019-06-04T12:58:19Z"`|string (date-time)| -|**version**
*required*|Version of the service being described. Semantic versioning is recommended, but other identifiers, such as dates or commit hashes, are also allowed. The version should be changed whenever the service is updated.
**Example** : `"1.0.0"`|string| - - -**organization** - -|Name|Description|Schema| -|---|---|---| -|**name**
*required*|Name of the organization responsible for the service
**Example** : `"My organization"`|string| -|**url**
*required*|URL of the website of the organization (RFC 3986 format)
**Example** : `"https://example.com"`|string (uri)| - - - -### ServiceInfo -*Polymorphism* : Composition - - -|Name|Description|Schema| -|---|---|---| -|**auth_instructions_url**
*required*|A web page URL with human-readable instructions on how to get an authorization token for use with a specific WES endpoint.|string| -|**contactUrl**
*optional*|URL of the contact for the provider of this service, e.g. a link to a contact form (RFC 3986 format), or an email (RFC 2368 format).
**Example** : `"mailto:support@example.com"`|string (uri)| -|**createdAt**
*optional*|Timestamp describing when the service was first deployed and available (RFC 3339 format)
**Example** : `"2019-06-04T12:58:19Z"`|string (date-time)| -|**default_workflow_engine_parameters**
*required*|Each workflow engine can present additional parameters that can be sent to the workflow engine. This message will list the default values, and their types for each workflow engine.|< [DefaultWorkflowEngineParameter](#defaultworkflowengineparameter) > array| -|**description**
*optional*|Description of the service. Should be human readable and provide information about the service.
**Example** : `"This service provides..."`|string| -|**documentationUrl**
*optional*|URL of the documentation of this service (RFC 3986 format). This should help someone learn how to use your service, including any specifics required to access data, e.g. authentication.
**Example** : `"https://docs.myservice.example.com"`|string (uri)| -|**environment**
*optional*|Environment the service is running in. Use this to distinguish between production, development and testing/staging deployments. Suggested values are prod, test, dev, staging. However this is advised and not enforced.
**Example** : `"test"`|string| -|**id**
*required*|Unique ID of this service. Reverse domain name notation is recommended, though not required. The identifier should attempt to be globally unique so it can be used in downstream aggregator services e.g. Service Registry.
**Example** : `"org.ga4gh.myservice"`|string| -|**name**
*required*|Name of this service. Should be human readable.
**Example** : `"My project"`|string| -|**organization**
*required*|Organization providing the service|[organization](#serviceinfo-organization)| -|**supported_filesystem_protocols**
*required*|The filesystem protocols supported by this service, currently these may include common protocols using the terms 'http', 'https', 'sftp', 's3', 'gs', 'file', or 'synapse', but others are possible and the terms beyond these core protocols are currently not fixed. This section reports those protocols (either common or not) supported by this WES service.|< string > array| -|**supported_wes_versions**
*required*|The version(s) of the WES schema supported by this service|< string > array| -|**system_state_counts**
*required*||< string, integer (int64) > map| -|**tags**
*required*||< string, string > map| -|**type**
*required*||[ServiceType](#servicetype)| -|**updatedAt**
*optional*|Timestamp describing when the service was last updated (RFC 3339 format)
**Example** : `"2019-06-04T12:58:19Z"`|string (date-time)| -|**version**
*required*|Version of the service being described. Semantic versioning is recommended, but other identifiers, such as dates or commit hashes, are also allowed. The version should be changed whenever the service is updated.
**Example** : `"1.0.0"`|string| -|**workflow_engine_versions**
*required*||< string, string > map| -|**workflow_type_versions**
*required*||< string, [WorkflowTypeVersion](#workflowtypeversion) > map| - - -**organization** - -|Name|Description|Schema| -|---|---|---| -|**name**
*required*|Name of the organization responsible for the service
**Example** : `"My organization"`|string| -|**url**
*required*|URL of the website of the organization (RFC 3986 format)
**Example** : `"https://example.com"`|string (uri)| - - - -### ServiceType -Type of a GA4GH service - - -|Name|Description|Schema| -|---|---|---| -|**artifact**
*required*|Name of the API or GA4GH specification implemented. Official GA4GH types should be assigned as part of standards approval process. Custom artifacts are supported.
**Example** : `"beacon"`|string| -|**group**
*required*|Namespace in reverse domain name format. Use `org.ga4gh` for implementations compliant with official GA4GH specifications. For services with custom APIs not standardized by GA4GH, or implementations diverging from official GA4GH specifications, use a different namespace (e.g. your organization's reverse domain name).
**Example** : `"org.ga4gh"`|string| -|**version**
*required*|Version of the API or specification. GA4GH specifications use semantic versioning.
**Example** : `"1.0.0"`|string| - - - -### State -- UNKNOWN: The state of the task is unknown. This provides a safe default for messages where this field is missing, for example, so that a missing field does not accidentally imply that the state is QUEUED. - - - - QUEUED: The task is queued. - - -- INITIALIZING: The task has been assigned to a worker and is currently preparing to run. For example, the worker may be turning on, downloading input files, etc. - -- RUNNING: The task is running. Input files are downloaded and the first Executor has been started. - -- PAUSED: The task is paused. An implementation may have the ability to pause a task, but this is not required. - - - - COMPLETE: The task has completed running. Executors have exited without error -and output files have been successfully uploaded. - - - - EXECUTOR_ERROR: The task encountered an error in one of the Executor processes. Generally, -this means that an Executor exited with a non-zero exit code. - - - - SYSTEM_ERROR: The task was stopped due to a system error, but not from an Executor, -for example an upload failed due to network issues, the worker's ran out of disk space, etc. - - - - CANCELED: The task was canceled by the user. - - - - CANCELING: The task was canceled by the user, and is in the process of stopping. - -*Type* : enum (UNKNOWN, QUEUED, INITIALIZING, RUNNING, PAUSED, COMPLETE, EXECUTOR_ERROR, SYSTEM_ERROR, CANCELED, CANCELING) - - - -### StatusResponse -Returns the status of monitored subsystems. - - -|Name|Schema| -|---|---| -|**serviceName**
*optional*|[serviceName](#statusresponse-servicename)| - - -**serviceName** - -|Name|Schema| -|---|---| -|**messages**
*optional*|< string > array| -|**ok**
*optional*|boolean| - - - -### ToolInputParameter -An input parameter for a tool or workflow. - - -|Name|Description|Schema| -|---|---|---| -|**default**
*required*|The in-language expression used to evaluate a default value for this parameter, if none is supplied.|string| -|**name**
*required*|The name of this input value (formatted as expected by the tool)|string| -|**optional**
*required*|Whether the tool allows this value to not be specified|boolean| -|**typeDisplayName**
*required*|An easy-to-read display name for the type of the input|string| -|**valueType**
*required*||[ValueType](#valuetype)| - - - -### ToolOutputParameter -An output parameter for a tool or workflow. - - -|Name|Description|Schema| -|---|---|---| -|**name**
*required*|The name of this input value (formatted as expected by the tool)|string| -|**typeDisplayName**
*required*|An easy-to-read display name for the type of the output|string| -|**valueType**
*required*||[ValueType](#valuetype)| - - - -### ValueType -The type expected for a given value. - - -|Name|Description|Schema| -|---|---|---| -|**arrayType**
*optional*||[ValueType](#valuetype)| -|**mapType**
*optional*||[MapValueType](#mapvaluetype)| -|**nonEmpty**
*optional*|Boolean representation of the contents in an array.|boolean| -|**objectFieldTypes**
*optional*||< [objectFieldTypes](#valuetype-objectfieldtypes) > array| -|**optionalType**
*optional*||[ValueType](#valuetype)| -|**pairType**
*optional*||[pairType](#valuetype-pairtype)| -|**tupleTypes**
*optional*||< [ValueType](#valuetype) > array| -|**typeName**
*optional*|The type of this value|enum (String, File, Directory, Float, Int, Boolean, Optional, Array, Tuple, Map, Object, Pair)| - - -**objectFieldTypes** - -|Name|Schema| -|---|---| -|**fieldName**
*optional*|string| -|**fieldType**
*optional*|[ValueType](#valuetype)| - - -**pairType** - -|Name|Schema| -|---|---| -|**leftType**
*optional*|[ValueType](#valuetype)| -|**rightType**
*optional*|[ValueType](#valuetype)| - - - -### VersionResponse -Returns the version of Cromwell - - -|Name|Description|Schema| -|---|---|---| -|**cromwell**
*optional*|The version of the Cromwell Engine
**Example** : `"30"`|string| - - - -### WorkflowDescription - -|Name|Description|Schema| -|---|---|---| -|**errors**
*required*|The set of validation failure messages
**Example** : `[ "The 'errors' field will be filled if 'valid' is false", "We might also provide warnings to a 'valid' workflow here", "Otherwise, 'errors' will be the empty array" ]`|< string > array| -|**images**
*optional*||< string > array| -|**importedDescriptorTypes**
*optional*||< [DescriptorTypeAndVersion](#descriptortypeandversion) > array| -|**inputs**
*required*|A list of inputs for this tool
**Example** : `[ {
"name" : "my_wf.string_input",
"valueType" : {
"typeName" : "String"
},
"optional" : false,
"default" : null,
"typeDisplayName" : "String"
}, {
"name" : "my_wf.array_input",
"valueType" : {
"typeName" : "Array",
"arrayType" : {
"typeName" : "String"
}
},
"optional" : false,
"default" : null,
"typeDisplayName" : "Array[String]"
}, {
"name" : "my_wf.optional_input",
"valueType" : {
"typeName" : "Optional",
"optionalType" : {
"typeName" : "String"
}
},
"optional" : true,
"default" : "hello",
"typeDisplayName" : "String?"
}, {
"name" : "my_wf.map_input",
"valueType" : {
"typeName" : "Map",
"mapType" : {
"keyType" : {
"typeName" : "String"
},
"valueType" : {
"typeName" : "Int"
}
}
},
"optional" : false,
"default" : null,
"typeDisplayName" : "Map[String, Int]"
}, {
"name" : "my_wf.object_input",
"valueType" : {
"typeName" : "Object",
"objectFieldTypes" : [ {
"fieldName" : "int_field",
"fieldType" : {
"typeName" : "Int"
}
}, {
"fieldName" : "int_array_field",
"fieldType" : {
"typeName" : "Array",
"arrayType" : {
"typeName" : "Int"
}
}
} ]
},
"optional" : false,
"default" : null,
"typeDisplayName" : "Object"
}, {
"name" : "my_wf.int_string_pair_input",
"valueType" : {
"typeName" : "Pair",
"pairTypes" : [ {
"leftType" : [ {
"typeName" : "Int"
} ]
}, {
"rightType" : [ {
"typeName" : "String"
} ]
} ]
},
"optional" : false,
"default" : null,
"typeDisplayName" : "Pair[Int, String]"
} ]`|< [ToolInputParameter](#toolinputparameter) > array| -|**isRunnableWorkflow**
*required*|Indicates whether this file can be run on its own (e.g. a WDL workflow)|boolean| -|**meta**
*optional*||object| -|**name**
*required*|For a source file with one workflow and zero or more tasks, the name of the workflow. For a single task, the name of the task. For a source file with multiple tasks but no workflows, the empty string.|string| -|**outputs**
*required*|A list of outputs for this tool
**Example** : `[ {
"name" : "my_wf.string_output",
"valueType" : {
"typeName" : "String"
},
"typeDisplayName" : "String"
}, {
"name" : "my_wf.array_output",
"valueType" : {
"typeName" : "Array",
"arrayType" : {
"typeName" : "String"
}
},
"typeDisplayName" : "Array[String]"
}, {
"name" : "my_wf.map_output",
"valueType" : {
"typeName" : "Map",
"mapType" : {
"keyType" : {
"typeName" : "String"
},
"valueType" : {
"typeName" : "Int"
}
}
},
"typeDisplayName" : "Map[String, Int]"
}, {
"name" : "my_wf.object_output",
"valueType" : {
"typeName" : "Object",
"objectFieldTypes" : [ {
"fieldName" : "int_field",
"fieldType" : {
"typeName" : "Int"
}
}, {
"fieldName" : "int_array_field",
"fieldType" : {
"typeName" : "Array",
"arrayType" : {
"typeName" : "Int"
}
}
} ]
},
"typeDisplayName" : "Object"
}, {
"name" : "my_wf.int_string_pair_output",
"valueType" : {
"typeName" : "Pair",
"tupleTypes" : [ {
"typeName" : "Int"
}, {
"typeName" : "String"
} ]
},
"typeDisplayName" : "Pair[Int, String]"
} ]`|< [ToolOutputParameter](#tooloutputparameter) > array| -|**parameterMeta**
*optional*||object| -|**submittedDescriptorType**
*required*||[DescriptorTypeAndVersion](#descriptortypeandversion)| -|**valid**
*required*|Indicates that the workflow is valid and that the inputs, if provided, are compatible with the workflow.|boolean| -|**validWorkflow**
*required*|Indicates whether the workflow file is valid by itself. If inputs are provided, they are not considered when calculating this field; if inputs are not provided, the value is identical to `valid`.|boolean| - - - -### WorkflowIdAndStatus - -|Name|Description|Schema| -|---|---|---| -|**id**
*required*|The identifier of the workflow
**Example** : `"00001111-2222-3333-aaaa-bbbbccccdddd"`|string| -|**status**
*required*|The status of the workflow
**Example** : `"Submitted"`|string| - - - -### WorkflowMetadataResponse -Workflow and call level metadata - - -|Name|Description|Schema| -|---|---|---| -|**calls**
*optional*||[CallMetadata](#callmetadata)| -|**end**
*optional*|End datetime of the workflow in ISO8601 format with milliseconds|string (date-time)| -|**failures**
*optional*||< [FailureMessage](#failuremessage) > array| -|**id**
*optional*|The identifier of the workflow|string| -|**inputs**
*optional*|Map of input keys to input values|object| -|**outputs**
*optional*|Map of output keys to output values|object| -|**start**
*optional*|Start datetime of the workflow in ISO8601 format with milliseconds|string (date-time)| -|**status**
*optional*|The status of the workflow|string| -|**submission**
*optional*|Submission datetime of the workflow in ISO8601 format with milliseconds|string (date-time)| - - - -### WorkflowQueryParameter -Workflow query parameters - - -|Name|Description|Schema| -|---|---|---| -|**end**
*optional*|Returns only workflows with an equal or earlier end datetime. Can be specified at most once. If both start and end date are specified, start date must be before or equal to end date.|string (date-time)| -|**excludeLabelAnd**
*optional*|Excludes workflows with the specified label. If specified multiple times, excludes workflows with all of the specified label keys. Specify the label key and label value pair as separated with "label-key:label-value"
**Pattern** : `"^([a-z][-a-z0-9]*[a-z0-9])?[:]([a-z][-a-z0-9]*[a-z0-9])?$"`|string (array)| -|**excludeLabelOr**
*optional*|Excludes workflows with the specified label. If specified multiple times, excludes workflows with any of the specified label keys. Specify the label key and label value pair as separated with "label-key:label-value"
**Pattern** : `"^([a-z][-a-z0-9]*[a-z0-9])?[:]([a-z][-a-z0-9]*[a-z0-9])?$"`|string (array)| -|**id**
*optional*|Returns only workflows with the specified workflow id. If specified multiple times, returns workflows with any of the specified workflow ids.
**Pattern** : `"^[0-9A-Fa-f]{8}-[0-9A-Fa-f]{4}-[0-9A-Fa-f]{4}-[0-9A-Fa-f]{4}-[0-9A-Fa-f]{12}$"`|string| -|**includeSubworkflows**
*optional*|Include subworkflows in results. By default, it is taken as true.|string (boolean)| -|**name**
*optional*|Returns only workflows with the specified name. If specified multiple times, returns workflows with any of the specified names.
**Pattern** : `"^[a-zA-Z][a-zA-Z0-9_]*$"`|string| -|**page**
*optional*|When pageSize is set, which page of results to return. If not set, the first page of 'pageSize' results will be returned.|integer| -|**pageSize**
*optional*|The number of results to return at a time|integer| -|**start**
*optional*|Returns only workflows with an equal or later start datetime. Can be specified at most once. If both start and end date are specified, start date must be before or equal to end date.|string (date-time)| -|**status**
*optional*|Returns only workflows with the specified status. If specified multiple times, returns workflows in any of the specified statuses.|enum (Submitted, Running, Aborting, Failed, Succeeded, Aborted)| -|**submission**
*optional*|Returns only workflows with an equal or later submission time. Can be specified at most once. If both submission time and start date are specified, submission time should be before or equal to start date.|string (date-time)| - - - -### WorkflowQueryResponse -Response to a workflow query - - -|Name|Schema| -|---|---| -|**results**
*required*|< [WorkflowQueryResult](#workflowqueryresult) > array| -|**totalResultsCount**
*required*|integer| - - - -### WorkflowQueryResult -Result for an individual workflow returned by a workflow query - - -|Name|Description|Schema| -|---|---|---| -|**end**
*optional*|Workflow end datetime|string (date-time)| -|**id**
*required*|Workflow ID|string| -|**metadataArchiveStatus**
*optional*|Status in metadata archives|string| -|**name**
*optional*|Workflow name|string| -|**start**
*optional*|Workflow start datetime|string (date-time)| -|**status**
*required*|Workflow status|string| -|**submission**
*optional*|Workflow submission datetime|string (date-time)| - - - -### WorkflowSubmitResponse - -|Name|Description|Schema| -|---|---|---| -|**id**
*required*|The identifier of the workflow
**Example** : `"00001111-2222-3333-aaaa-bbbbccccdddd"`|string| -|**status**
*required*|The status of the workflow
**Example** : `"Submitted"`|string| - - - -### WorkflowTypeVersion -Available workflow types supported by a given instance of the service. - - -|Name|Description|Schema| -|---|---|---| -|**workflow_type_version**
*optional*|an array of one or more acceptable types for the `workflow_type`|< string > array| - diff --git a/docs/cromwell_features/Labels.md b/docs/cromwell_features/Labels.md index b81608ff25d..bf5178ef143 100644 --- a/docs/cromwell_features/Labels.md +++ b/docs/cromwell_features/Labels.md @@ -14,12 +14,12 @@ In order to assign labels to a workflow, the first step is to create a JSON file When choosing key-value pairs, it's important to make sure you're adhering to Cromwell supported label syntax below. There are two ways to add labels to a workflow: -1. Upon workflow submission set the `labels` parameter of the [Submit endpoint](api/RESTAPI#submit-a-workflow-for-execution), or +1. Upon workflow submission set the `labels` parameter of the Submit endpoint, or 2. Setting the `-l` argument when running in [Command Line](/CommandLine) mode. -Labels can be added to existing workflows by using the [Labels patch endpoint](api/RESTAPI#update-labels-for-a-workflow). +Labels can be added to existing workflows by using the `/labels` patch endpoint. -After adding labels to your workflows, you can take advantage of features like [Query](api/RESTAPI#get-workflows-matching-some-criteria) to filter tagged workflows. The Google backend supports labelling cloud resources and you can learn more about that [here](backends/Google#google-labels). +After adding labels to your workflows, you can take advantage of the `/query` endpoint to filter tagged workflows. The Google backend supports labelling cloud resources and you can learn more about that [here](backends/Google#google-labels). #### Label Format diff --git a/docs/developers/Contribute.md b/docs/developers/Contribute.md index 9f40fb982ee..3d03afa72ff 100644 --- a/docs/developers/Contribute.md +++ b/docs/developers/Contribute.md @@ -72,15 +72,6 @@ _produces this block_ * _Example:_ `![](https://www.broadinstitute.org/sites/all/themes/custom/at_broad/logo.png)` _produces this image_ ![](https://www.broadinstitute.org/sites/all/themes/custom/at_broad/logo.png) -### REST API & Menu - -**REST API:** - -1. Edit the [`cromwell.yaml`](https://github.com/broadinstitute/cromwell/blob/develop/engine/src/main/resources/swagger/cromwell.yaml) to make any changes to the REST API content. -2. Regenerate the REST API markdown file by running `sbt generateRestApiDocs` from the main Cromwell directory. -3. Commit both the changes to the `cromwell.yaml` and the (re)generated [RESTAPI.md](https://github.com/broadinstitute/cromwell/blob/develop/docs/api/RESTAPI.md). -4. Once your branch is merged to the [`develop` branch](https://github.com/broadinstitute/cromwell/tree/develop), you will see your changes on the [REST API page](api/RESTAPI/). - **Left-side menu:** To add or remove items from the menu, edit [mkdocs.yml](https://github.com/broadinstitute/cromwell/blob/develop/mkdocs.yml) in Cromwell. @@ -97,9 +88,3 @@ If not, [kindly ask the team to merge it](https://github.com/broadinstitute/crom * **Did you add the file(s) to the YAML file?** If not, [add it here](https://github.com/broadinstitute/cromwell/blob/develop/mkdocs.yml). - -**_How do I add to the REST API documentation?_** - -* **Don't forget to regenerate** -After you edit the [`cromwell.yaml`](https://github.com/broadinstitute/cromwell/blob/develop/engine/src/main/resources/swagger/cromwell.yaml), run `sbt generateRestApiDocs` and commit all changes. -_Hint:_ Once you have regenerated the docs correctly, the hidden timestamp at the top of the [`RESTAPI.md` file](https://raw.githubusercontent.com/broadinstitute/cromwell/develop/docs/api/RESTAPI.md) will show the current time. diff --git a/docs/execution/ExecutionTwists.md b/docs/execution/ExecutionTwists.md index 69603d9a2f1..7126d69b17a 100644 --- a/docs/execution/ExecutionTwists.md +++ b/docs/execution/ExecutionTwists.md @@ -60,7 +60,7 @@ If the failure mode is **`ContinueWhilePossible`**, then A **will be** retried ( In both [Run](../Modes/#run) and [Server](../Modes/#server) mode, you can abort a running workflow. This section explains what that entails. -When aborting a workflow, either through the [abort endpoint](../api/RESTAPI#abort-a-running-workflow) or by terminating the [Cromwell run process](../Modes) (if [configured](../Configuring#abort) to do so), Cromwell does the following: +When aborting a workflow, either through the `/abort` endpoint or by terminating the [Cromwell run process](../Modes) (if [configured](../Configuring#abort) to do so), Cromwell does the following: 1. Changes the status of the workflow to `Aborting`, 2. Does not start any new jobs, diff --git a/docs/tutorials/ServerMode.md b/docs/tutorials/ServerMode.md index 4a17684d3b0..42b0e8137c3 100644 --- a/docs/tutorials/ServerMode.md +++ b/docs/tutorials/ServerMode.md @@ -60,10 +60,9 @@ Paste the following into a file called `inputs.json`: # What happened? -* [Did it work? Check the status endpoint](../api/RESTAPI#api-workflows-version-id-status-get) -* Holy logs! [How can I just see my outputs?](../api/RESTAPI#api-workflows-version-id-outputs-get) -* [Check out metadata related to your workflow.](../api/RESTAPI#api-workflows-version-id-metadata-get) -* [All kinds of other interesting info.](../api/RESTAPI) +* Did it work? Check the `/status` endpoint. +* How can I just see my outputs? `/outputs` endpoint. +* Check out metadata related to your workflow with `/metadata`. ### Next Steps diff --git a/docs/wf_options/Overview.md b/docs/wf_options/Overview.md index 7f42e0e9944..084fbbdf943 100644 --- a/docs/wf_options/Overview.md +++ b/docs/wf_options/Overview.md @@ -2,7 +2,7 @@ Workflow options can affect the execution of a single workflow without having to change configuration options or restart Cromwell. -You provide workflow options to Cromwell in a JSON format. This can be supplied at workflow-submit time either via the [CLI](../CommandLine.md) or the [REST endpoint](../api/RESTAPI.md): +You provide workflow options to Cromwell in a JSON format. This can be supplied at workflow-submit time either via the [CLI](../CommandLine.md) or the Submit endpoint: ```json { diff --git a/engine/src/main/resources/swagger/cromwell.yaml b/engine/src/main/resources/swagger/cromwell.yaml index dca96b20ceb..59a202df393 100644 --- a/engine/src/main/resources/swagger/cromwell.yaml +++ b/engine/src/main/resources/swagger/cromwell.yaml @@ -1,7 +1,3 @@ -# -# After any edits to this file run `sbt generateRestApiDocs` and commit the generated changes to `docs/api/RESTAPI.md`. -# - swagger: '2.0' # This line is included when the YAML is served by Cromwell if it sees an environment variable called "SWAGGER_BASE_PATH" diff --git a/mkdocs.yml b/mkdocs.yml index 87d48d3171e..7a6065910b3 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -68,9 +68,6 @@ nav: - Google Cloud Storage: filesystems/GoogleCloudStorage.md - HTTP: filesystems/HTTP.md - Data Repository Service: filesystems/DataRepositoryService.md -- REST API: - - Overview: api/RESTAPI.md - - Errors: api/ErrorHandling.md - For Developers: - How to Build: developers/Building.md - Architecture: developers/Arch.md diff --git a/project/GenerateRestApiDocs.scala b/project/GenerateRestApiDocs.scala deleted file mode 100644 index 3e0297a09f5..00000000000 --- a/project/GenerateRestApiDocs.scala +++ /dev/null @@ -1,167 +0,0 @@ -import io.github.swagger2markup.builder.Swagger2MarkupConfigBuilder -import io.github.swagger2markup.{MarkupLanguage, Swagger2MarkupConverter} -import org.apache.commons.lang3.{ClassUtils, StringUtils} -import sbt.Keys._ -import sbt._ - -import java.time.OffsetDateTime -import java.time.format.DateTimeFormatter - -/** - * Provides a task to generate the REST API markdown from the Swagger YAML. - * - * Uses imports provided by swagger2markup.sbt. - * - * Based partially on https://github.com/21re/sbt-swagger-plugin - */ -object GenerateRestApiDocs { - private lazy val generateRestApiDocs = taskKey[Unit]("Generates the docs/api/RESTAPI.md") - private lazy val checkRestApiDocs = - taskKey[Unit]("Compares the existing docs/api/RESTAPI.md against the generated version") - - /** Returns a timestamped preamble for the modified markdown output. */ - private def preamble(): String = { - val now = OffsetDateTime.now() - s"""| - |""".stripMargin - } - - // Path to the input swagger yaml - private val SwaggerYamlFile = new File("engine/src/main/resources/swagger/cromwell.yaml") - - // Path to the generated markdown - private val RestApiMarkdownFile = new File("docs/api/RESTAPI.md") - - // A regex to locate the collection of swagger paths. - private val PathsRegex = "(?s)(.*## Paths)(.*)(## Definitions.*)".r - // A regex to locate an individual swagger path within the above collection. Each is turned into "## " - private val PathRegex = "(?s)### (.*)".r - - private val GenericReplacements = List( - // Change a few section headers into bolded text. - "## Overview" -> "**Overview** ", - "### Version information" -> "**Version information** ", - "### License information" -> "**License information** ", - "### Produces" -> "**Produces** ", - // Since individual paths are moved from "###" to "##" we don't need "Paths" anymore. - "" -> "", - "## Paths" -> "" - ) - - /** - * Move contents of the "## Paths" section up a level. - * - * Each "### example path" will be come "## example path". - * - * Should be run before `replaceGenerics` as it looks for the string "## Paths". - * - * @param content The original contents of the RESTAPI.md. - * @return The contents with updated paths. - */ - private def replacePaths(content: String): String = - content match { - case PathsRegex(start, paths, end) => - val replacedPaths = paths.linesWithSeparators map { - case PathRegex(pathDescription) => s"## $pathDescription" - case other => other - } - replacedPaths.mkString(start, "", end) - case _ => - throw new IllegalArgumentException( - "Content did not match expected regex. " + - "Did the swagger2markdown format change significantly? " + - "If so, a new regex may be required." - ) - } - - /** - * Replaces generic strings in the generated RESTAPI.md. - * - * @param content The contents of the RESTAPI.md. - * @return The contents with generic replacements. - */ - private def replaceGenerics(content: String): String = - GenericReplacements.foldRight(content)(replaceGeneric) - - /** - * Replaces a single generic string in the generated RESTAPI.md. - * - * @param tokens The original and replacement strings. - * @param content The contents of the RESTAPI.md. - * @return The contents with generic replacements. - */ - private def replaceGeneric(tokens: (String, String), content: String): String = { - val (original, replacement) = tokens - content.replace(original, replacement) - } - - /** - * Apache commons tries to dynamically load classes from the current thread classloader. Unfortunately the - * classloader in the current thread from SBT does NOT have all of the libraries loaded. - * - * So-- hack in ClassUtils' classloader instead, that has access to all the required libraries including - * org.apache.commons.configuration2.PropertiesConfiguration - * - * Otherwise, a ClassNotFoundException will occur. - */ - private def withPatchedClassLoader[A](block: => A): A = { - val classUtilsClassLoader = classOf[ClassUtils].getClassLoader - val currentThread = Thread.currentThread - val originalThreadClassLoader = currentThread.getContextClassLoader - try { - currentThread.setContextClassLoader(classUtilsClassLoader) - block - } finally - currentThread.setContextClassLoader(originalThreadClassLoader) - } - - private def getModifiedMarkdown: String = - withPatchedClassLoader { - val config = new Swagger2MarkupConfigBuilder() - .withMarkupLanguage(MarkupLanguage.MARKDOWN) - .build() - val converter = Swagger2MarkupConverter - .from(SwaggerYamlFile.toPath) - .withConfig(config) - .build() - val contents = converter.toString - replaceGenerics(replacePaths(contents)) - } - - /** - * Generates the markdown from the swagger YAML, with some Cromwell customizations. - */ - private def writeModifiedMarkdown(): Unit = { - val replacedContents = preamble() + getModifiedMarkdown - IO.write(RestApiMarkdownFile, replacedContents) - } - - /** - * Compares the generated markdown against the existing markdown. - */ - private def checkModifiedMarkdown(log: Logger): Unit = { - val markdownContents = StringUtils.substringAfter(IO.read(RestApiMarkdownFile), "-->" + System.lineSeparator) - val generatedContents = getModifiedMarkdown - if (markdownContents != generatedContents) { - val message = "The file docs/api/RESTAPI.md is not up to date. Please run: sbt generateRestApiDocs" - // Throwing the exception below should log the message... but it doesn't on sbt 1.2.1, so do it here. - log.error(message) - throw new IllegalStateException(message) - } else { - log.info("checkRestApiDocs: The file docs/api/RESTAPI.md is up to date.") - } - } - - // Returns a settings including the `generateRestApiDocs` task. - val generateRestApiDocsSettings: Seq[Setting[_]] = List( - generateRestApiDocs := writeModifiedMarkdown(), - checkRestApiDocs := checkModifiedMarkdown(streams.value.log) - ) -} diff --git a/project/Publishing.scala b/project/Publishing.scala index c506532ffbb..dec61ab128b 100644 --- a/project/Publishing.scala +++ b/project/Publishing.scala @@ -153,7 +153,7 @@ object Publishing { import java.time.{ZoneId, ZonedDateTime} import java.time.format.DateTimeFormatter - val buildTime = ZonedDateTime.now(ZoneId.systemDefault()).format(DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss")) + val buildTime = ZonedDateTime.now(ZoneId.systemDefault()).format(DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss z")) // It is optimal to use a single `Run` instruction to minimize the number of layers in the image. // diff --git a/project/Settings.scala b/project/Settings.scala index 628a77db4c6..5251ac7283a 100644 --- a/project/Settings.scala +++ b/project/Settings.scala @@ -1,6 +1,5 @@ import ContinuousIntegration._ import Dependencies._ -import GenerateRestApiDocs._ import Merging._ import Publishing._ import Testing._ @@ -220,7 +219,6 @@ object Settings { _ .disablePlugins(AssemblyPlugin) .settings(publish := {}) - .settings(generateRestApiDocsSettings) .settings(ciSettings) .settings(rootPublishingSettings) ) diff --git a/project/swagger2markup.sbt b/project/swagger2markup.sbt deleted file mode 100644 index 1fa882328f1..00000000000 --- a/project/swagger2markup.sbt +++ /dev/null @@ -1,40 +0,0 @@ -/* -Provides imports used by GenerateRestApiDocs. - -Borrowed from https://github.com/21re/sbt-swagger-plugin -Uses latest (at time of coding) s2m 1.3.1 - */ -val apacheHttpclientV = "4.3.6" -val asmV = "5.2" -val commonsCodecV = "1.10" -val commonsLang3V = "3.5" -val commonsLoggingV = "1.2" -val guavaV = "18.0" -val jacksonV = "2.8.4" -val javaslangV = "2.0.5" -val plexusUtilsV = "3.0.22" -val slf4jV = "1.7.21" -val swagger2markupV = "1.3.4" - -resolvers ++= List( - "JCenter" at "https://jcenter.bintray.com" -) - -libraryDependencies ++= List( - "io.github.swagger2markup" % "swagger2markup" % swagger2markupV -) - -dependencyOverrides ++= List( - "com.fasterxml.jackson.core" % "jackson-annotations" % jacksonV, - "com.fasterxml.jackson.core" % "jackson-databind" % jacksonV, - "com.google.guava" % "guava" % guavaV, - "commons-codec" % "commons-codec" % commonsCodecV, - "commons-logging" % "commons-logging" % commonsLoggingV, - "io.javaslang" % "javaslang" % javaslangV, - "org.apache.commons" % "commons-lang3" % commonsLang3V, - "org.apache.httpcomponents" % "httpclient" % apacheHttpclientV, - "org.codehaus.plexus" % "plexus-utils" % plexusUtilsV, - "org.ow2.asm" % "asm" % asmV, - "org.ow2.asm" % "asm-tree" % asmV, - "org.slf4j" % "slf4j-api" % slf4jV -) diff --git a/src/ci/bin/testCheckPublish.sh b/src/ci/bin/testCheckPublish.sh index 60989795ed0..46ca9033d9a 100755 --- a/src/ci/bin/testCheckPublish.sh +++ b/src/ci/bin/testCheckPublish.sh @@ -10,6 +10,6 @@ cromwell::build::setup_common_environment cromwell::build::pip_install mkdocs mkdocs build -s -sbt -Dsbt.supershell=false --warn checkRestApiDocs +package assembly dockerPushCheck +doc +sbt -Dsbt.supershell=false --warn +package assembly dockerPushCheck +doc git secrets --scan-history