Replies: 5 comments 7 replies
-
|
Oh there's a way to have multiple workflows in the tutorial? Thank god, I had concerns. As a learner and trainer, I would find it useful to see the timestamps (for how long a tool took) by the tools in the tutorials themselves, to indicate how long I should expect it to take. |
Beta Was this translation helpful? Give feedback.
-
|
Would you be interested in historic run data then? in addition to just a "latest run"? |
Beta Was this translation helpful? Give feedback.
-
|
and yes, e.g. 16s short has a lot of workflows, hence the display/location of such data being an important question. |
Beta Was this translation helpful? Give feedback.
-
|
Yes historic data would be ace! |
Beta Was this translation helpful? Give feedback.
-
|
As a server admin, I would find it useful to see everything listed in Q1 if possible, priority would be link to test history, metrics and file I think. Thank you for doing this! |
Beta Was this translation helpful? Give feedback.
-
|
this is very helpful, thank you 💛 |
Beta Was this translation helpful? Give feedback.
-
|
As a tutorial author/teacher, it seems to me of interest to have all points listed in Q1, with a particular emphasis for 1/ time used by each tool execution and 2/ access to an existing history created by the workflow (at least to have possibility to reuse existing good results if there is issues regarding tool execution or tool schedulong at the moment of a trinaing). Maybe display it on the tuto page on a workflow oriented part with maybe a possibility to scroll down elements can be top! AMAZING work !!!! Thank you !!!! |
Beta Was this translation helpful? Give feedback.
-
|
Thanks @yvanlebras great to know! I suspected the tool run times would be useful, thank you for confirming it and providing some suggestions :) |
Beta Was this translation helpful? Give feedback.
-
|
As a tutorial author and LifeMonitor developer (with @simleo and @kikkomep), I would like to see these test outcomes on a LifeMonitor dashboard. One of the LifeMonitor service’s goals is to collect in a single place test outcomes from multiple workflows of interest, potentially running on multiple infrastructures (we call them “test instances"). Once a workflow test is registered, interested people can subscribe to see it on their own dashboards and/or receive notifications about its status. Test results can also be retrieved via REST API for easy integration. There’s also a GitHub app that can help keep LifeMonitor (and WorkflowHub) entries up-to-date. This is already in place for the IWC workflows (we can thank @simleo for that). Naturally, if there is interest in registering these workflows in LifeMonitor, we’d be happy to discuss the details to come up with a strategy. |
Beta Was this translation helpful? Give feedback.
-
|
Hey @ilveroluca that's great to hear about. I hadn't had time to fully go through the lifemonitor tutorial but I clearly need to now! We've been looking for a way to keep the GTN workflows updated on Workflow Hub for some time now (#3049) but hadn't found anything acceptable, since all of the solutions required us to manually create all 200+ entries in WFH. We won't be able to change the repository layout significantly so, looking at the documentation, we'd need to write some automation around generating the WTROCs ourselves, and then sending them to LifeMonitor? Is that correct? Additionally how we'll launch tests is maybe a bit complicated since we're trying to do some special things like, launching the workflows in one github action, and collecting the results in a separate one, as we don't want to burn github action minutes while some of these workflows take multiple hours to run against EU/Org. And that's our third oddity, we're testing workflows against multiple backend servers. I'm not sure if that sort of workflow is supported? We should definitely discuss the best solution here since in general I like the suggestion of also pushing the results to LM, though we'll probably continue to develop some form of internal dashboard given different audience needs |
Beta Was this translation helpful? Give feedback.
-
|
Hi @hexylena! We reasoned a bit about the information you provided about the GTN setup. I think we can more or less split the discussion over two main points: test monitoring and automated creation/maintenance of workflow entries on WorkflowHub and LifeMonitor. First point: test monitoringFrom how we interpreted your comments, it seems that the GTN workflow testing setup is not implemented yet, and it we hypothesize that the plan is to have an action that launches multiple tests on one or more (maybe public) Galaxy services and have a separate single action to wait and collect all test results. This last point doesn't work with LifeMonitor's model (but please correct us if we didn't understand your plan). LM doesn't know about Planemo or any other test engines. What it currently does is look at the success or failure of a CI workflow. It thus follows that there must be one action per workflow. Maybe you can find a way to have your single waiting action dispatch other workflow-specific actions to collect and propagate test results (e.g., GH Actions allows one workflow to call other workflows, but the limitations seem to get in the way of using this approach for 200+ workflows). On the other hand, monitoring multiple testing services for the same workflow is not a problem: LifeMonitor supports the concept of having multiple test instances for the same workflow. These test instances can be on different servers and they can even be different type of CI services (e.g., the same tests running on both Jenkins and GH Actions). Second point: automated creation/maintenance of workflow entries on WorkflowHub and LifeMonitorThis functionality is supported through the LifeMonitor GitHub app. It is indeed a more complex feature to support in the GTN case, since at the moment LM makes the assumption of one workflow per repository (and RO-Crates in a way also make this assumption, as the metadata file is supposed to be at the root of the RO-Crate archive and WROC represents a single workflow). To enable automatic registration we have to be able to automatically create a WTROC for the workflow. For this purpose, we have developed the repo2crate package, which LifeMonitor uses. Currently, LM executes repo2crate at the base of the repository and uses the resulting WTRO-Crate to register or update the workflow's entries on LifeMonitor and/or WorkflowHub. Note that you may also find repo2crate and ro-crate-py useful for any RO-Crate-related custom automation. We reasoned that LifeMonitor should in theory be able to support a multi-workflow repository such as GTN's by associating a "prefix" path to the workflow to select a subdirectory of the repository to be considered as its "root path". This would require some development work on our part. Also, the end user would have to identify each workflow and the associated prefix path and the associated test instance through the lifemonitor.yml configuration file (through the creation could be partly automated). At this stage we're not sure when such a feature could be integrated, though interest from the GTN would be good motivation. A final thing I should mention is that both LifeMonitor and WorkflowHub provide REST APIs (LifeMonitor API explorer, Workflowhub API, WorkflowHub RO-Crate submission API). You may also consider targeting these directly in your automation workflow. |
Beta Was this translation helpful? Give feedback.
-
|
I'd like a summary not just by tutorial but by theme (i.e. single cell) so communities know who to chase / check in on |
Beta Was this translation helpful? Give feedback.
-
The GTN is finally implementing automated workflow testing for the workflows across the GTN. We have started the implementation, but we have some questions for you, the community! Namely, how should we present these results to you?
Q1: What information is useful?
Q2: Where would you like to see it?
We would appreciate you making individual reply posts below, something like:
So we can group use cases :) Please feel free to also add your response to an existing reply, if you feel like it fits in the same category as theirs.
Beta Was this translation helpful? Give feedback.
All reactions