Sandbox URLs

[andre-merzky]

We have currently two proposals on how to refer to various sandbox locations. Those are:

VERSION A:

client   sandbox: client:///data.dat
session  sandbox: session:///data.dat
resource sandbox: resource:///data.dat
pilot    sandbox: pilot:///data.dat
task     sandbox: task:///data.dat

Note that pilot, session and task sandboxes always are interpreted in a specific context can can mean different locations. For example, if task.000000 is scheduled on pilot.0000, then pilot:///data.dat refers to a file in that pilot sandbox, while the same URL for task000001 which is scheduled on pilot.0001 refers to a different location, possibly on a different resource.

VERSION B:

client   sandbox: sandbox://client/data.dat
session  sandbox: sandbox://rp.session.local.001/data.dat
resource sandbox: sandbox://local.localhost/data.dat
pilot    sandbox: sandbox://pilot.0000/data.dat
task     sandbox: sandbox://task.000001/data.dat

Note that with this schema, a task.000000 can, in its staging directives, refer to another task's (task.000001) sandbox which is not possible in version a. But also, one needs to know the uid's for all referred entity sandboxes. For a task description with staging directives, one would need to know the pilot ID on which this task will be running in order to stage to the pilot sandbox this way, which limits the set of use cases.

What do people think about those approaches? Do you see a way to combine the advantages for both into a single schema? Should we support both?

[mtitov]

Have a proposal for a mix of A&B

<type>://[<id_or_label>:]<path>

Thus, by default we may not to use <id_or_label> and it will be assumed a local space/sandbox, e.g., for task we can use pilot://input.dat and it would assume a sandbox of the pilot to which task is related, and task://output.dat is a task's sandbox, but task://task.000001:output_dir/output.dat is a sandbox of a different task from the same session.

I think it is better to have a protocol, which is entity type specific, since at least client and resource with remote submission will be handled differently, but for session/pilot/task we can use sandbox://<uid>/<path> additionally. Important to keep in mind that pilot and task uids are unique within session only. Thus if we want data from previously run session, then data from that session should be kept within session sandbox and not a pilot sandbox of that session -> session://rp.session.0000:output.dat

[andre-merzky]

I like the distinction between task:///output.dat and task://task.000001/output.dat (slightly changed), one referencing the 'current' task, one referencing any task identified by UID. Also, god point about the reference to other sessions - we could for example use task://task.000001@rp.session.0001/output.dat for a somewhat intuitive way to reference IDs from other session?

FWIW, if the above gives us a consistent set of URLs, I would not mind dropping the sandbox:// approach - having two schemas will likely be confusing...