Download test repos via coursier #4066
Conversation
So that the repo archives are cached, and the tasks that use them work offline
| val cache = FileCache() | ||
| // Uncomment when rebootstrapping Mill, to get better logging | ||
| // .withLogger(mill.util.CoursierSupport.millLogger()) | ||
| val file = cache.file(Artifact(s"https://github.com/$label/archive/$commit.zip")) |
There was a problem hiding this comment.
Noob question: I just can create a file cache and download a arbitrary file?
Where is this then cached and what is its cache key? I assume the key is the URL?
Because this seems very useful in general. I have a 'manual' cache so far.
There was a problem hiding this comment.
This is using the cache of coursier, which downloads dependencies in Mill (JVM ones at least). The exact cache location depends on the OS.
That doesn't support query strings, and if the artifact might change, one should call withChanging(true) on the Artifact created in the code above (then if the last check is older than a TTL, coursier will check if the file changed).
|
I think this should be part of a broader discussion in #3930. Should Coursier be the recommended way to download-and-cache stable URLs? All stable URLs or only some stable URLs? What are the considerations and alternatives here? Obviously this PR works for this specific scenario and does what it says on the box, but we really should have a more rigorous analysis of what the options are, and if this is the recommended option then it should not be used just here but instead also makes its way into the user-facing documentation |
|
@lihaoyi Sure
I didn't think much about alternatives, but I don't think there are that many out there 🤔 As discussed in #3930, other ecosystems tend to have caches quite tied to their ecosystem. The cache of coursier is much more generic OTOH. AFAIK, only the Nix cache offers similar features. |
|
Letting you decide if you'd rather merge this or not, while things are being discussed in #3930 |
|
as I mentioned in #3930, let's hold off on this for the moment and come back after general support in Mill is in place |
So that the repo archives are cached, and the tasks that use them work offline.
That's useful to build the website while being offline, for example.