Maintenance of manifests by vendor

[radeksimko]

Hi folks,
I work for HashiCorp and I firstly want to thank the community (most recently @r15ch13 ) for the amazing work on maintaining manifests for our software, namely terraform, vault, packer, consul, nomad and vagrant.

Thank you!

---

I’m in the process of researching our options with regards to automating the process of publishing new versions of our software to different platforms including Windows/Scoop. I am aware that a 3rd party bucket is an option for us and I think it’s viable, but I’d like to ask some questions before we embark on that journey.

Are there any best practices for software vendors (like HashiCorp) whose software is already being distributed through the main bucket (as linked above)? I do not want to cause too much disruption to users who are already consuming our software through the main bucket, but if we do begin to publish updates through our own bucket we’d likely want to treat it as the preferred way.

• Would you (Scoop maintainers) want/expect vendor’s bucket to be the exclusive place from where to install/update their software from, or would you prefer the bucket to be additional to the existing manifests in the main bucket?
• If you prefer the manifests to remain in the main bucket - is there any fully automateable process for publishing updates that we could integrate into our release pipeline, or is the main bucket designed for human-based updates? i.e. would automation be fragile as humans edit these files via PRs?
• If you prefer for the manifests to move to our bucket - is there any migration mechanism or strategy available?
• If you prefer our bucket to be additional - is there anything to do or avoid doing with regards to the UX?
  • I noticed that installed apps share a single global namespace, which means that explicit "qualified" name with bucket prefix (e.g. hashicorp/terraform) is only needed for the initial installation. Are there any other side effects to be aware of?
  • I noticed that conflicts are dealt with on the basis of manifest names - i.e. it's impossible to install two apps of the same name. I suspect this is intended and we can use this as a way to indicate that two apps (in main bucket and custom bucket) are the same?

---

Generally custom bucket would be the preferred option from my personal perspective as we can distribute updates in a timely fashion (by plugging scripts into our release pipeline), which is important especially for security patches.

That said I don’t want the decision to come across as disrespectful towards Scoop maintainers or users - hence I’m raising the question here before taking actions.

[rashil2000]

I'd like to firstly apologise; the Scoop project had been defunct for a long time and hence the delay in response.

• Would you (Scoop maintainers) want/expect vendor’s bucket to be the exclusive place from where to install/update their software from, or would you prefer the bucket to be additional to the existing manifests in the main bucket?

If the manifests already exist in Main bucket, we would prefer that they be used. I'd advocate for the 'one source of truth' policy here, as it helps users use the apps the way they are intended as opposed to having multiple options with subtly different manifests.

• If you prefer the manifests to remain in the main bucket - is there any fully automateable process for publishing updates that we could integrate into our release pipeline, or is the main bucket designed for human-based updates? i.e. would automation be fragile as humans edit these files via PRs?

The update process is fully automated; Scoop manifests contain a URL to fetch information about releases from - ideally, that URL should be well-integrated with the release pipelines of software vendors so that Scoop can update the manifests accordingly. Sometimes PRs are made manually to tweak the installation procedure followed in the manifest, or to update the aforementioned URL/regexes to fetch updates from.

• If you prefer for the manifests to move to our bucket - is there any migration mechanism or strategy available?

As of now, no such migration process is available.

• If you prefer our bucket to be additional - is there anything to do or avoid doing with regards to the UX?

  • I noticed that installed apps share a single global namespace, which means that explicit "qualified" name with bucket prefix (e.g. hashicorp/terraform) is only needed for the initial installation. Are there any other side effects to be aware of?
  • I noticed that conflicts are dealt with on the basis of manifest names - i.e. it's impossible to install two apps of the same name. I suspect this is intended and we can use this as a way to indicate that two apps (in main bucket and custom bucket) are the same?

They aren't necessarily the same, when you do scoop list, the output clearly mentions the bucket the app manifest is from; this is to indicate to the users that there can be subtle differences as I mentioned above, either in installation procedure, data persistence, full or slim builds etc.

Generally custom bucket would be the preferred option from my personal perspective as we can distribute updates in a timely fashion (by plugging scripts into our release pipeline), which is important especially for security patches.

The rationale is very understandable; however, as I said, the place where Scoop gets its autoupdate information from, if it is updated in a timely manner, the manifests will also get updated. That said, we wouldn't really want to enforce the Main bucket - if you feel you want to control the installation/update process in an opinionated manner, then by all means you can direct users to add your custom hashicorp bucket.

[rasa]

@radeksimko My apologies, but I never saw your post, or I would have responded sooner. I use your tools all the time, and have contributed to packer.

Maybe the best of both worlds would be that hashicorp maintains a scoop bucket, and Excavator pulls updates from there to push into the Main bucket. This is a great way to distribute the maintenance workload, while providing users an up-to-date centralized repository.