Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speed up installation of a large number of packages #155

Open
tomasz-sobczyk-wttech opened this issue Jul 26, 2023 · 5 comments · May be fixed by #162
Open

Speed up installation of a large number of packages #155

tomasz-sobczyk-wttech opened this issue Jul 26, 2023 · 5 comments · May be fixed by #162

Comments

@tomasz-sobczyk-wttech
Copy link

Overview
Today every package installation has it's own health checks - there is a param called done_threshold that defines how many times we check if system is stable before announcing package is installed.

Reducing the value of this parameter to sth like 2 is a good idea to speed things up. But!
There are cases where you might want to double-triple-quadruple check the stability - example is when your package contaings Sling Maping changes. Such a change can cause an asynchronously invoked (event driven) restart of most-if-not-all AEM bundles.

How to eat the cake and have the cake? A.k.a. potential solution:
Introduce (apart from existing global configuration for done_threshold) a new configuration where I will be able to:

  1. Define package name (regular expression)
  2. Define done_threshold for that particular package
    This should be an array type of data (multiple configurations allowed)
    Code should verify the package being installed against the list provided, first matched configuration should be picked up.
    If not matching configuration is found - it should pick up the value from global config.

This configuration can be still overwritten from the command line (CLI direct params should always take precedense)

Regexp?
Why regexp for package name: Package names have versions in them often etc - let's keep it open ended.

@krystian-panek-vmltech
Copy link
Contributor

thanks for reporting, will be implemented within this month ;)

@krystian-panek-vmltech
Copy link
Contributor

when deploying a package that is changing some configuration of ResourceResolverFactory then AEM may restart many bundles and become unstable for a longer time. but... this usually is a one-time situation for the first package deployment. maybe config activation could be smart that activates depending on the first or next package deployment?

@krystian-panek-vmltech
Copy link
Contributor

thinking about sth like that:

image

then such foo pkg settings will be automatically in-use basing on path pattern and the fact if package was deployed before or not, AEM_INSTANCE_CHECK_PACKAGE=foo sh aemw pkg deploy --url ... will activate the settings explicitly

@tomasz-sobczyk-wttech , @maciej-majchrzak-wttech WDYT?

or maybe just there should be 2 different done_thresholds

done_threshold: 
   first: 5
   next: 2

@absa-maciej-majchrzak
Copy link

This first config is very confusing for me. Why it's even needed? Ideally, I'd like to specify three types of checks:

  1. generic check - if AEM is stable, could be used regardless of package installation in any cases by calling aemw instance await
  2. default package installation check - to check if instance is stable after installation. It should be possible to completely ignore stability check after installation, so that we don't need to wait until all events stopped processing. This would be useful in a scenario of installing more packages at once. In this case, one could install packages one after another and in the end of the process call the aemw instance await. Additionally, in case of problematic packages they could use the third types of check, which is...
  3. specific package installation check - something like you propose with the pattern. done_threshold configuration is fine here.

@krystian-panek-vmltech
Copy link
Contributor

regarding second point - will be addressed by #159
generic check will have a form of current configuration/options that AEMC gives currently;
third point is a thing to address here / missing feature that we are planning to implement. I guess within next weeks I will find some time to have it done.

@krystian-panek-vmltech krystian-panek-vmltech linked a pull request Aug 12, 2023 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants