You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In https://sagebionetworks.jira.com/browse/MC2DPQC-51 we found that files were being staged during the samplesheet_split step. As this occurs in the headjob rather than a process it proved impossible to slow/queue/limit this step and avoid hitting rate-limits on the GTex server.
Thanks to Brad for proposing a solution.
My feeling would be to add a new optional process called something like STAGE_FILES that occurs after SPLIT and before HISTOQC.
Passing through this process should be optional based on a param (say params.stage_in_process) which would be a bool with default false
The text was updated successfully, but these errors were encountered:
In https://sagebionetworks.jira.com/browse/MC2DPQC-51 we found that files were being staged during the samplesheet_split step. As this occurs in the headjob rather than a process it proved impossible to slow/queue/limit this step and avoid hitting rate-limits on the GTex server.
Thanks to Brad for proposing a solution.
My feeling would be to add a new optional process called something like STAGE_FILES that occurs after SPLIT and before HISTOQC.
Passing through this process should be optional based on a param (say params.stage_in_process) which would be a bool with default false
The text was updated successfully, but these errors were encountered: