Build-eclipse-launcher not run since Feb 2023?

[jonahgraham]

It looks like there is a build step that hasn't been run since Feb 2023:

https://ci.eclipse.org/releng/job/Build-eclipse-launcher/ was last run then, which means that changes since then haven't been added to the binaries.

This means the fix in eclipse-equinox/equinox#117 never got rolled out AFAICT.

@sravanlakkimsetti the last build (before fix to eclipse-equinox/equinox#117) was manually run by you - do you know who (or what automatic process) should be handling this?

[sravanlakkimsetti]

@jonahgraham This is a manual process. Need to manually run those jobs.

Here are the instructions on how to build https://wiki.eclipse.org/Building_Equinox_Launcher. This actually builds and commits to launcher binaries repo. Successfully running this will rebuild the launchers.

Since the launcher was built quite sometime ago. You may want rebuild all platforms. Check all 4 options

[jonahgraham]

Thanks @sravanlakkimsetti.

As it turns out this has come up before in eclipse-equinox/equinox#16 (comment)

At a minimum I think this needs to be added to a release checklist to ensure that the binaries are up to date, ideally there should be an automatic check that at least errors if the launcher sources are newer than the binaries.

For now, @MohananRahul can this step be added to the releng document and once we are done with M1 lets see if we (collectively) can get this updated. There is some work planned on the launcher for Eclipse IDE WG Funded Development Effort #24 and I want to help make sure that the processes are streamlined around that launcher.

For reference @Torbjorn-Svensson added a GitHub action that auto-builds the launcher for PRs here eclipse-equinox/equinox#247

[MohananRahul]

For now, @MohananRahul can this step be added to the releng document and once we are done with M1 lets see if we (collectively) can get this updated. There is some work planned on the launcher for Eclipse IDE WG Funded Development Effort #24 and I want to help make sure that the processes are streamlined around that launcher.

@jonahgraham @sravanlakkimsetti Does this job need to be done at every milestone?

[Torbjorn-Svensson]

@jonahgraham @sravanlakkimsetti Does this job need to be done at every milestone?

I'd say that it should be part of every milestone yes.
If it's not done for every milestone, then any changes in the binaries will not be testable before the final release as it's hard to build the binaries for all the platforms manually. I also presume that any potential issue should be handled as early as possible in the increment and not deferred to an extra service release to correct some issue that could have easily been seen in a milestone build.

[sravanlakkimsetti]

Launcher is quite stable and not many changes come in that way.
@MohananRahul Build it once per release. May be you can have a check at every milestone by verifying the timestamps of https://github.com/eclipse-equinox/equinox/tree/master/features/org.eclipse.equinox.executable.feature/library and latest Recompiled binaries in https://github.com/eclipse-equinox/equinox.binaries/

[iloveeclipse]

May be you can have a check at every milestone by verifying the timestamps of https://github.com/eclipse-equinox/equinox/tree/master/features/org.eclipse.equinox.executable.feature/library

Could it be done via unit test case that would hard code last timestamp/commit and which would fail as soon as new version will be committed?

[sravanlakkimsetti]

unit test way is one of the options. I would prefer an automation similar to swt native builds.

Launcher gets fixes once a year on an average. Need to weigh in on the effort and reward in this case.

[jonahgraham]

@laeubi I think helping here falls under the IDE WG funded dev effort for release engineering support. Can you consider it, or let me know what additional info you need?

[laeubi]

@jonahgraham I'm not sure from the discussion what ultimately should be the outcome here.

Can't we just trigger the Job each time https://github.com/eclipse-equinox/equinox/tree/master/features/org.eclipse.equinox.executable.feature/library changes? Or is there any drawback / manual steps required?

[jonahgraham]

@laeubi I don't have a strong opinion. I think the minimum is a job somewhere that fails if the executables are out of date, with the maximum/ideal being that they are automatically rebuilt & committed as a side effect of the source changing.

Or is there any drawback / manual steps required?

I don't have any info beyond what is in https://wiki.eclipse.org/Building_Equinox_Launcher (as linked by @sravanlakkimsetti in #1407 (comment)) - I hope that if you take on this task you can identify if there are any currently undocumented or assumed steps that need resolving.

[MohananRahul]

Job completed https://ci.eclipse.org/releng/job/Build-eclipse-launcher/89/

[laeubi]

@sravanlakkimsetti @jonahgraham I took a look at the buildjob and it seems quite complex, is not yet part of the repository and only special people are allowed to modify it.

But maybe the following can help you to make the job running automatic instead of manually:

1. The job should be triggered on every change in equinox
2. Then as a first step the following should be executed by the job:

git clone https://github.com/eclipse-equinox/equinox.git
git clone https://github.com/eclipse-equinox/equinox.binaries
export TS_GENERAL="$(git -C equinox log -1 --format=%ct features/org.eclipse.equinox.executable.feature/library)"
export TS_COCOA="$(git -C equinox log -1 --format=%ct features/org.eclipse.equinox.executable.feature/library/cocoa)"
export TS_GTK="$(git -C equinox log -1 --format=%ct features/org.eclipse.equinox.executable.feature/library/gtk)"
export TS_WIN32="$(git -C equinox log -1 --format=%ct features/org.eclipse.equinox.executable.feature/library/win32)"
export TS_LAUNCHER="$(git -C equinox.binaries log -1 --format=%ct)"

3. then (only pseudo code because my bash foo is kind of basic):

• if TS_GENERAL <= TS_LAUNCHER nothing has to be done and the job should simply terminate
• otherwise the increment variable of the Job should be set to true
• if TS_COCOA > TS_LAUNCHER variable buildCocoa should be set to true
• if TS_GTK > TS_LAUNCHER variable buildGtk should be set to true
• if TS_WIN32 > TS_LAUNCHER variable buildWin32 should be set to true
• if *all of TS_COCOA, TS_GTK, TS_WIN32 are <= TS_LAUNCHER, buildCocoa, buildGtk and buildWin32 should be set to true.

This then have the desired effect that whenever there is a source change the affected binaries are rebuild and pushed, for all other changes nothing happens.

[sravanlakkimsetti]

@jonahgraham I'm not sure from the discussion what ultimately should be the outcome here.

Can't we just trigger the Job each time https://github.com/eclipse-equinox/equinox/tree/master/features/org.eclipse.equinox.executable.feature/library changes? Or is there any drawback / manual steps required?

I used to trigger the job whenever there is change in the library. The issue is not closed till a new launcher build is committed. This is how I used to keep track and make sure launcher gets built every time there is a change.

@sravanlakkimsetti @jonahgraham I took a look at the buildjob and it seems quite complex, is not yet part of the repository and only special people are allowed to modify it.

It looks complicated. but actually quite simple. Described the steps here eclipse-equinox/equinox#318 (comment).

I would rather go with increment and build all launchers whenever there is a change in library. That way this will be quite simple solution. Please go with that solution.

TS_GENERAL folder contains common code between all os platforms. if there is any change in that code we need to build all. lets avoid this kind of clomplications

[laeubi]

I would rather go with increment and build all launchers whenever there is a change in library.

You mean trigger it manually? Because thats what obviously last time did not happens and seem to caused some confusion, therefore the idea to automate this "is there a change" step.

[sravanlakkimsetti]

No. I mean build automatically with increment, and build all launchers gtk, cocoa and win32. Don't worry about if individual os versions are not updated.

[laeubi]

Then one only needs to use

export TS_GENERAL="$(git -C equinox log -1 --format=%ct features/org.eclipse.equinox.executable.feature/library)"
export TS_LAUNCHER="$(git -C equinox.binaries log -1 --format=%ct)"

and somehow adjust the buildskript to do nothing unless TS_GENERAL > TS_LAUNCHER ...

[sravanlakkimsetti]

Then one only needs to use

export TS_GENERAL="$(git -C equinox log -1 --format=%ct features/org.eclipse.equinox.executable.feature/library)"
export TS_LAUNCHER="$(git -C equinox.binaries log -1 --format=%ct)"

and somehow adjust the buildskript to do nothing unless TS_GENERAL > TS_LAUNCHER ...

yes thats correct.

[HannesWell]

unit test way is one of the options. I would prefer an automation similar to swt native builds.

Agree. In SWT each commit that changed the natives gets a special and unique git tag applied and the Jenkins workflow checks if there was a change in the directories that contain the natives code since the last tag and then builds the natives again.

We could even consider to also use git-lfs to store the natives logically in the same repo.
For SWT I started to work on this but got distracted from that. I'll try to prioritize my work there and once it is completed it could be used as template for the Equinox launcher binaries.

• Merge swt.binaries repo in the swt one eclipse.platform.swt.binaries#2

[akurtakov]

This one is fixed, right?

[akurtakov]

No interest makes me believe it has been fixed.

[HannesWell]

unit test way is one of the options. I would prefer an automation similar to swt native builds.

That automation has just been implemented via eclipse-equinox/equinox#603.
The previous launcher building job is already de-activated and I added a corresponding description:
https://ci.eclipse.org/releng/view/Launcher/job/Build-eclipse-launcher/

Once the new automated pipeline has proofed to work well I'll delete all manual and now obsolete launcher jobs.