Allow for externally-supplied storage_options to S3 loader and further down to Reductionist

[valeriupredoi]

Description

This is a locally-constructed, and off the back of #180 version of #178 - merely translating @davidhassell 's work here, but with a twist 😁

What we did here:

• Active now ingests two new kwargs storage_options=None (type: dict) and active_storage_url=None (type: str) by means of which a user can specify S3 credentials straight into Active (or pass them from a higher level eg cf-python). These can be varied, and tested for - e.g. a collection of items like the credentials needed to get into a standard S3 bucket, or an anon=True bucket. See tests/test_compression.py and tests/test_compression_remote_reductionist.py - there we use storage_options and active_storage_url kwargs to perfectly reproduce connecting to a localhost/Minio data object from a local Reductionist, and also to connect to a valid yet anon=True bucket via a remote Reductionist (this latter stops at Access Denied since Reductionist is not yet able to support anon=True buckets)
• these storage options have now been integrated in the open mechanisms in Active and netcdf_to_zarr

Closes #177 and the valiant #178

Before you get started

• ☝ Create an issue to discuss what you are going to do

Checklist

• This pull request has a descriptive title and labels
• This pull request has a minimal description (most was discussed in the issue, but a two-liner description is still desirable)
• Unit tests have been added (if codecov test fails)
• Any changed dependencies have been added or removed correctly (if need be)
• All tests pass

[davidhassell]

Hi V - looks fine to me, apart from the whole pre-configured options thing. I'm in the office tomorrow if you want to chat about it.

[codecov]

Codecov Report

Attention: 2 lines in your changes are missing coverage. Please review.

Comparison is base (bb769e5) 87.11% compared to head (8812593) 88.11%.

Files	Patch %	Lines
activestorage/active.py	92.85%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #182      +/-   ##
==========================================
+ Coverage   87.11%   88.11%   +1.00%     
==========================================
  Files           8        8              
  Lines         613      648      +35     
==========================================
+ Hits          534      571      +37     
+ Misses         79       77       -2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[valeriupredoi]

OK we are in very good shape - this thing does a dance and half, pretty much toasts the bread and spreads the butter too, don't mind the S3 tests failing, those are supposed to fail given Reductionist doesn't support anon=True buckets, but the cf-python integration and all else is there. Needs a bit more unit testing, but let's first talk about it tomorrow 🍺

[davidhassell]

Hi V, all after our chat :) Those two modified tests put in as suggestions. Good to go when after that.

[davidhassell]

Done

[valeriupredoi]

Done

mega, many thanks @davidhassell 🍻