"Username / “MyTardis” / Experiment / Dataset" folder configuration not working properly

[RichardPBerry]

Attempting to use this configuration setting in MyData seems to require that the "MyTardis" folder under the username folder is the only subfolder at that level. Adding another subfolder throws the error:
"A folder name of "[FolderName]" was found where a "MyTardis" folder was expected.

I'm assuming this is not the expected behaviour. Found this is on Mac OS, and replicated on the Windows version of the client.

[wettenhj]

Ther Username / "MyTardis" / Experiment / Dataset folder structure was introduced to support ingestion from an old legacy data ingestion workflow. We don't use it any more and I think it has been removed from the latest v0.7.0-beta. Do you need to use this folder structure?

[RichardPBerry]

Hi James, no we were intending to use this but have recently changed to encourage uploads to user groups rather than individuals, so will probably use on of the group structures (e.g. User Group / Instrument / Full Name / Dataset).

Just thought I would report it as I came across it.

[wettenhj]

OK, thanks for letting us know!

[nrmay]

We would like this feature re-instated, and also that it should support more that one folder under the username level. At one facility, MyData will be installed on machines with authenticated user folders and additional back-up methods. Therefore, we need to support a method whereby the users can QA the datafiles before dropping them into a sub-folder for ingested into a MyTardis.

e.g.

• username
  • scratch (backed up elsewhere)
    • other info not to be ingested
  • MyTardis (ingested into MyTardis)
    • Exp_1
      • Dataset_1
        • Datafile_1
    • etc

[wettenhj]

Hi @nrmay and @RichardPBerry,

The latest update is that support for this folder structure has not in fact been removed from MyData. I had considered removing it, but I was mistaken when I said it had already been removed.

In this commit: ec2ddd2 I removed the code which caused settings validation to fail when the "Validate folder structure" checkbox was ticked and there was a folder other than "MyTardis" at the same level as the "MyTardis" folder.

This change will be available in the next v0.7.0-beta build - v0.7.0-beta5.

[RichardPBerry]

Thanks @wettenhj that is good to know. In the future I can also a use for this construct in with a group structure rather than username - i.e. groupname / "My Tardis" / Experiment / Dataset. We have similar case for one instrument where we will aim to use a network share as scratch space for multiple user groups, with a designated MyTardis ingest folder where a user can copy data after processing and QA.

[wettenhj]

This fix is included in the latest v0.7.0 beta build: https://github.com/mytardis/mydata/releases/tag/v0.7.0-beta5