Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: update c++ base images #6073

Merged
merged 6 commits into from
Sep 18, 2024

Conversation

jcferretti
Copy link
Member

No description provided.

@jcferretti jcferretti self-assigned this Sep 17, 2024
@jcferretti jcferretti added this to the 0.37.0 milestone Sep 17, 2024
kosak
kosak previously approved these changes Sep 17, 2024
@kosak kosak self-requested a review September 17, 2024 22:48
kosak
kosak previously approved these changes Sep 17, 2024
kosak
kosak previously approved these changes Sep 17, 2024
@jcferretti jcferretti enabled auto-merge (squash) September 18, 2024 01:36
Comment on lines +41 to +42
// Need to disable until the proto base image can catch up to protobuf 28.2
tasks.getByName('compareProtobuf').enabled = false
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm confused why this is necessary; do we not use the same code to generate the c++ protobuf that we use to compare it?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The proto targets generate code for all languages with the protoc image. The C++ build has never used that output, there is a separate script

https://github.com/deephaven/deephaven-core/blob/main/proto/proto-backplane-grpc/src/main/proto/build-cpp-protos.sh

that is used to generate code under cpp-client/deephaven/proto. This code in the cpo-client/build.gradle was added to force both to be consistent. But given the current issues in protobuf versions across languages and the state of chaos the protubuf team created with their breaking changes, this is not advisable anymore. It actually never was advisable; there is no reason to force different languages to use the same proto version. We arguably should not be generating C++ code if it is not used, but forcing something generated and not used to be consistent with something that is used and works and pass tests is not the solution to that IMHO

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Went back and forth with Cristian a bit on this, and my understanding is that we previously had two ways to generate cpp code from protos, and they happened to have the same output - proto/proto-backplane-grpc's gradle/docker wiring, and the build-cpp-protos.sh script.

The fact that they generated the same output was wonderful, but ultimately irrelevant - the build-cpp-protos.sh script is going to use the protoc in the cpp docker image, and so is "more" correct.

My proposal is that this merge as-is, with a new issue to be filed to follow-up here to restore this task. Both the updateProto and compareProto tasks should instead read from build-cpp-protos.sh output in its own docker task, and the rest of the cpp codegen in proto/proto-backplane-grpc should be removed.

We can revert this later and return to proto/proto-backplane-grpc generating cpp bindings, but if we do, we should at the same time delete build-cpp-protos.sh so there is only one way to generate this code. I'm not sure this is a step worth taking, especially given the current protobuf versioning ... scheme.

Comment on lines +41 to +42
// Need to disable until the proto base image can catch up to protobuf 28.2
tasks.getByName('compareProtobuf').enabled = false
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Went back and forth with Cristian a bit on this, and my understanding is that we previously had two ways to generate cpp code from protos, and they happened to have the same output - proto/proto-backplane-grpc's gradle/docker wiring, and the build-cpp-protos.sh script.

The fact that they generated the same output was wonderful, but ultimately irrelevant - the build-cpp-protos.sh script is going to use the protoc in the cpp docker image, and so is "more" correct.

My proposal is that this merge as-is, with a new issue to be filed to follow-up here to restore this task. Both the updateProto and compareProto tasks should instead read from build-cpp-protos.sh output in its own docker task, and the rest of the cpp codegen in proto/proto-backplane-grpc should be removed.

We can revert this later and return to proto/proto-backplane-grpc generating cpp bindings, but if we do, we should at the same time delete build-cpp-protos.sh so there is only one way to generate this code. I'm not sure this is a step worth taking, especially given the current protobuf versioning ... scheme.

@jcferretti jcferretti merged commit 5a6bdc1 into deephaven:main Sep 18, 2024
16 checks passed
@jcferretti jcferretti deleted the cfs-update-cpp-again-again branch September 18, 2024 20:02
@github-actions github-actions bot locked and limited conversation to collaborators Sep 18, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants