Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update gitlab4j@5.1.0 and use Long for Gitlab entities' IDs #414

Open
wants to merge 9 commits into
base: master
Choose a base branch
from

Conversation

akphi
Copy link
Contributor

@akphi akphi commented Apr 18, 2022

I'm starting to see failing calls as pipeline job ID exceeds INTEGER.MAX_VALUE, so we have to use Long instead of Integer for IDs, this change scope includes project, version, workflow, workflow job

This will require us to update to the latest version of gitlab4j-api 5.0.1 as it has converted all of its POJO to use long for ids

Issue

I'm seeing this error while using Legend at https://legend.finos.org/sdlc (pointing at https://gitlab.com):

https://legend.finos.org/sdlc/api/projects/UAT-27078619/workspaces/Tarta/workflows/570831036/jobs

{
"code": 500,
"message": "Error getting jobs for workflow 570831036 in workspace Tarta in project UAT-27078619: Error reading entity from input stream.",
"timestamp": "2022-07-04T20:12:20.841734Z"
}

You can see the job ID is 2628618079 from https://gitlab.com/finosfoundation/legend/showcase/demo-project/-/jobs/2628618079


UPDATED: This seems to be more complicated than I thought.

First, gitlab4j-api >=14.16.0 relies on org.glassfish.jersey.jackson.internal.jackson.jaxrs.json.JacksonJaxbJsonProvider which is only available in jersey > 2.26. Whereas, we're using com.fasterxml.jackson.jaxrs.json.JacksonJaxbJsonProvider (part of our jersey version 2.25.1). This means in legend-sdlc-server, I have to include org.glassfish.jersey.media:jersey-media-json-jackson:2.35 as a runtime dependency. In theory, we could have just upgraded all of our Jersey dependencies to >2.26, however, this will clash with dropwizard@1.3 (see this (issue)[https://github.com/dropwizard/dropwizard/issues/2148]) and thus, requires us to upgrade to Dropwizard 2 first (see @ivan-kyosev-gs PR).

However, this poses another problem, due to the auto-discoverable feature mechanism of jersey, our Jersey client/server accidentally registers the newer Jackson provider. This is problematic, especially in test, because we only register/bind the mixin classes (InMemoryWorkspace -> Workspace, etc.) to the older provider. As such, we need to somehow exclude the newer provider leaving only the older one to be picked up by Jackson at runtime. Note that we only need to do this for LegendSDLC server, Gitlab client used by gitlab4j-api will still function normally.

To do this, we can explicitly set LegendSDLC Jersey client/server flag CommonProperties.FEATURE_AUTO_DISCOVERY_DISABLE = true (See this). This sounds risky on paper, because it will also exclude any other things that are meant to be registered with AutoDiscoverable interface.

I have taken the time to find the list of AutoDiscoverable by debugging the ServiceLoader for Jersey at runtime and seems like only org.glassfish.jersey.jackson.internal.JacksonAutoDiscoverable (coming from org.glassfish.jersey.media:jersey-media-json-jackson:2.35) are included. As such, I consider this is safe enough for us to use as a workaround till #371 happen and we could go the normal route of upgrading Jersey.

@akphi akphi force-pushed the update-gitlab4j branch 3 times, most recently from a44bf9f to d69c7b8 Compare April 19, 2022 02:21
@akphi
Copy link
Contributor Author

akphi commented Apr 22, 2022

@kevin-m-knight-gs Seems like we're blocked on this one since gitlab4j@5.0.1 uses JacksonFeature which comes with newer version of jersey (in their repo, they do 2.35 for jersey), which is not compatible with dropwizard@1 - see https://www.dropwizard.io/en/release-2.0.x/manual/upgrade-notes/upgrade-notes-2_0_x.html#jersey

See their code here:

https://github.com/gitlab4j/gitlab4j-api/blame/09ed795b9354bd4736fee15f72bf355927faeef4/src/main/java/org/gitlab4j/api/GitLabApiClient.java

They make the call clientConfig.register(JacksonFeature.class);

@github-actions
Copy link

github-actions bot commented Jul 3, 2022

Test Results

  89 files    89 suites   4m 15s ⏱️
446 tests 445 ✔️ 1 💤 0

Results for commit db690dd.

♻️ This comment has been updated with latest results.

@akphi akphi force-pushed the update-gitlab4j branch 2 times, most recently from 5d608e8 to 950da5a Compare July 4, 2022 18:22
@akphi akphi marked this pull request as ready for review July 4, 2022 19:51
@akphi
Copy link
Contributor Author

akphi commented Jul 4, 2022

@kevin-m-knight-gs Seems like we're blocked on this one since gitlab4j@5.0.1 uses JacksonFeature which comes with newer version of jersey (in their repo, they do 2.35 for jersey), which is not compatible with dropwizard@1 - see https://www.dropwizard.io/en/release-2.0.x/manual/upgrade-notes/upgrade-notes-2_0_x.html#jersey

See their code here:

https://github.com/gitlab4j/gitlab4j-api/blame/09ed795b9354bd4736fee15f72bf355927faeef4/src/main/java/org/gitlab4j/api/GitLabApiClient.java

They make the call clientConfig.register(JacksonFeature.class);

@kevin-m-knight-gs I think I found a way to make this work. Please checkout the explanation stated in the PR description for my approach.

I don't think a lot people have complained about this, but I start seeing more and more on my pipelines on gitlab.com using https://legend.finos.org/sdlc - e.g.

https://legend.finos.org/sdlc/api/projects/UAT-27078619/workspaces/Tarta/workflows/570831036/jobs

{
"code": 500,
"message": "Error getting jobs for workflow 570831036 in workspace Tarta in project UAT-27078619: Error reading entity from input stream.",
"timestamp": "2022-07-04T20:12:20.841734Z"
}

You can see the job ID is 2628618079 from https://gitlab.com/finosfoundation/legend/showcase/demo-project/-/jobs/2628618079

@akphi akphi force-pushed the update-gitlab4j branch 2 times, most recently from 2d3e993 to 63ea663 Compare July 19, 2022 21:27
@kevin-m-knight-gs kevin-m-knight-gs marked this pull request as draft September 16, 2022 13:11
@akphi akphi changed the title update Gitlab4J@5.0.1 and use Long for Gitlab entities' IDs update Gitlab4J@5.1.0 and use Long for Gitlab entities' IDs Apr 6, 2023
@akphi akphi force-pushed the update-gitlab4j branch 2 times, most recently from 5e86942 to 563a8d2 Compare April 6, 2023 18:59
@akphi akphi marked this pull request as ready for review April 7, 2023 14:46
@akphi akphi requested a review from a team as a code owner April 7, 2023 14:46
Copy link
Contributor

@kevin-m-knight-gs kevin-m-knight-gs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few minor changes

@akphi akphi changed the title update Gitlab4J@5.1.0 and use Long for Gitlab entities' IDs update gitlab4j@5.1.0 and use Long for Gitlab entities' IDs Aug 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants