Operandi 372 - Extend the workflow job submission endpoint with preserve_file_grps field

[MehmedGIT]

Add a new request field preserve_file_grps to allow specifying just the file groups to be preserved after the workflow job execution. This field is mutually exclusive with remove_file_grps, which specifies the file groups to be removed. Specifying file groups to remove was a tedious process since the user had to check which file groups will be produced by the workflow job and specify these in the request.

The Operandi Server can also extract Nextflow workflow metadata to identify which file groups are to be created with the execution of the workflow job. This allows appending future file groups to the list of file groups to be removed.

Consider the following example:
The user uploads a workspace that contains 5 file groups: DEFAULT, MIN, MAX, PRESENTATION, and THUMBS identified with 06b6ba63-e898-4373-8d14-c7d73cb33d06. Then the user starts a workflow job with the default_workflow_with_MS and the following json parameters:

POST SERVER_ADDRESS/workflow/default_workflow_with_MS

{
  "workflow_args": {
    "workspace_id": "06b6ba63-e898-4373-8d14-c7d73cb33d06",
    "input_file_grp": "DEFAULT",
    "preserve_file_grps": "DEFAULT,OCR-D-OCR",
    "mets_name": "mets.xml"
  }
}

The workflow job will produce 8 file groups (based on the used workflow):

"OCR-D-BIN"
"OCR-D-CROP"
"OCR-D-BIN2"
"OCR-D-BIN-DENOISE"
"OCR-D-BIN-DENOISE-DESKEW"
"OCR-D-SEG"
"OCR-D-SEG-LINE-RESEG-DEWARP"
"OCR-D-OCR"

The

"preserve_file_grps": "DEFAULT,OCR-D-OCR"

will lead to the deletion of all file groups except DEFAULT and OCR-D-OCR.