Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix(docker): Use ubuntu and eclipse-temurin base image #14

Merged
merged 1 commit into from
Aug 21, 2024
Merged

fix(docker): Use ubuntu and eclipse-temurin base image #14

merged 1 commit into from
Aug 21, 2024

Conversation

CodeLieutenant
Copy link
Collaborator

Problem described in #13, missing libsnappy and linker loader errors requires us to change from alpine base to ubuntu base (default base image for eclipse-temurin).

Some environmental variables for Linker loader have been added to point to the libraries installed by the package manager.

@CodeLieutenant
fix(docker): Use ubuntu and eclipse-temurin base image
fea4373 
Problem described in scylladb#13, missing libsnappy and linker loader
errors requires us to change from alpine base to ubuntu base
(default base image for `eclipse-temurin`).

Some environmental variables for Linker loader have been
added to point to the libraries installed by the package
manager.

Signed-off-by: Dusan Malusev <dusan.malusev@scylladb.com>
@CodeLieutenant CodeLieutenant self-assigned this Aug 19, 2024
@CodeLieutenant CodeLieutenant linked an issue Aug 19, 2024 that may be closed by this pull request
Linker error not able to find Snappy compression library #13
Closed
mykaul
mykaul reviewed Aug 20, 2024
View reviewed changes
Dockerfile Show resolved Hide resolved
@roydahan
Copy link
Contributor

I can take it for a ride if needed, I'm in a middle of sessions building my own dockers and running some tests.

@CodeLieutenant
Copy link
Collaborator Author

I can take it for a ride if needed, I'm in a middle of sessions building my own dockers and running some tests.

If you have time, go for it.

I will not merge this as there is some issue with building the image inside github actions, looks like credentials are not in order as seen in #15 PR

@CodeLieutenant CodeLieutenant merged commit dcf5ba1 into scylladb:master Aug 21, 2024
2 checks passed
@CodeLieutenant CodeLieutenant deleted the fix/libsnappy-missing branch August 21, 2024 17:57
@dimakr
Copy link
Contributor

dimakr commented Aug 26, 2024

@CodeLieutenant @fruch If I understand it correctly this fix has not landed yet (probably because of #15). Now almost all rolling upgrade tests for the latest 2024.1.8 RC are failing with the:

  < t:2024-08-24 13:20:42,464 f:base.py         l:146  c:RemoteLibSSH2CmdRunner p:ERROR > Error executing command: "sudo  docker exec e450cf4c5db93281c796427c59a663bcb48843bc9d04a8c9fdb5ac9f7a218d30 /bin/sh -c 'echo TAG: loader_idx:1-cpu_idx:0-keyspace_idx:1; STRESS_TEST_MARKER=TH7AXMN52O2XWGTSVEW2; cassandra-stress read no-warmup cl=QUORUM duration=10m -schema keyspace=keyspace1 '"'"'replication(strategy=NetworkTopologyStrategy,replication_factor=3) compression=SnappyCompressor'"'"' -mode cql3 native compression=snappy  user=cassandra password=cassandra -rate threads=1000 -pop seq=1..10100200 -log interval=5 -node 10.4.2.25,10.4.1.228,10.4.1.48,10.4.3.83 -errors skip-unsupported-columns'"; Exit status: 1
< t:2024-08-24 13:20:42,464 f:base.py         l:148  c:RemoteLibSSH2CmdRunner p:DEBUG > STDOUT:   0 []
< t:2024-08-24 13:20:42,464 f:base.py         l:148  c:RemoteLibSSH2CmdRunner p:DEBUG > Total GC count            : 0
< t:2024-08-24 13:20:42,464 f:base.py         l:148  c:RemoteLibSSH2CmdRunner p:DEBUG > Total GC memory           : 0.000 KiB
< t:2024-08-24 13:20:42,464 f:base.py         l:148  c:RemoteLibSSH2CmdRunner p:DEBUG > Total GC time             :    0.0 seconds
< t:2024-08-24 13:20:42,464 f:base.py         l:148  c:RemoteLibSSH2CmdRunner p:DEBUG > Avg GC time               :    NaN ms
< t:2024-08-24 13:20:42,464 f:base.py         l:148  c:RemoteLibSSH2CmdRunner p:DEBUG > StdDev GC time            :    0.0 ms
< t:2024-08-24 13:20:42,464 f:base.py         l:148  c:RemoteLibSSH2CmdRunner p:DEBUG > Total operation time      : 00:10:00
< t:2024-08-24 13:20:42,464 f:base.py         l:148  c:RemoteLibSSH2CmdRunner p:DEBUG > 
< t:2024-08-24 13:20:42,464 f:base.py         l:148  c:RemoteLibSSH2CmdRunner p:DEBUG > FAILURE
< t:2024-08-24 13:20:42,464 f:base.py         l:150  c:RemoteLibSSH2CmdRunner p:DEBUG > STDERR: Failed to connect over JMX; not collecting these stats
< t:2024-08-24 13:20:42,464 f:base.py         l:150  c:RemoteLibSSH2CmdRunner p:DEBUG > java.lang.IllegalStateException: The requested compression is not available (some compression require a JAR to be found in the classpath)

@CodeLieutenant
Copy link
Collaborator Author

@CodeLieutenant @fruch If I understand it correctly this fix has not landed yet (probably because of #15). Now almost all rolling upgrade tests for the latest 2024.1.8 RC are failing with the:

  < t:2024-08-24 13:20:42,464 f:base.py         l:146  c:RemoteLibSSH2CmdRunner p:ERROR > Error executing command: "sudo  docker exec e450cf4c5db93281c796427c59a663bcb48843bc9d04a8c9fdb5ac9f7a218d30 /bin/sh -c 'echo TAG: loader_idx:1-cpu_idx:0-keyspace_idx:1; STRESS_TEST_MARKER=TH7AXMN52O2XWGTSVEW2; cassandra-stress read no-warmup cl=QUORUM duration=10m -schema keyspace=keyspace1 '"'"'replication(strategy=NetworkTopologyStrategy,replication_factor=3) compression=SnappyCompressor'"'"' -mode cql3 native compression=snappy  user=cassandra password=cassandra -rate threads=1000 -pop seq=1..10100200 -log interval=5 -node 10.4.2.25,10.4.1.228,10.4.1.48,10.4.3.83 -errors skip-unsupported-columns'"; Exit status: 1
< t:2024-08-24 13:20:42,464 f:base.py         l:148  c:RemoteLibSSH2CmdRunner p:DEBUG > STDOUT:   0 []
< t:2024-08-24 13:20:42,464 f:base.py         l:148  c:RemoteLibSSH2CmdRunner p:DEBUG > Total GC count            : 0
< t:2024-08-24 13:20:42,464 f:base.py         l:148  c:RemoteLibSSH2CmdRunner p:DEBUG > Total GC memory           : 0.000 KiB
< t:2024-08-24 13:20:42,464 f:base.py         l:148  c:RemoteLibSSH2CmdRunner p:DEBUG > Total GC time             :    0.0 seconds
< t:2024-08-24 13:20:42,464 f:base.py         l:148  c:RemoteLibSSH2CmdRunner p:DEBUG > Avg GC time               :    NaN ms
< t:2024-08-24 13:20:42,464 f:base.py         l:148  c:RemoteLibSSH2CmdRunner p:DEBUG > StdDev GC time            :    0.0 ms
< t:2024-08-24 13:20:42,464 f:base.py         l:148  c:RemoteLibSSH2CmdRunner p:DEBUG > Total operation time      : 00:10:00
< t:2024-08-24 13:20:42,464 f:base.py         l:148  c:RemoteLibSSH2CmdRunner p:DEBUG > 
< t:2024-08-24 13:20:42,464 f:base.py         l:148  c:RemoteLibSSH2CmdRunner p:DEBUG > FAILURE
< t:2024-08-24 13:20:42,464 f:base.py         l:150  c:RemoteLibSSH2CmdRunner p:DEBUG > STDERR: Failed to connect over JMX; not collecting these stats
< t:2024-08-24 13:20:42,464 f:base.py         l:150  c:RemoteLibSSH2CmdRunner p:DEBUG > java.lang.IllegalStateException: The requested compression is not available (some compression require a JAR to be found in the classpath)

This fix has landed in v3.13.0 just SCT is not updated with that tag.
#15 does not have anything with this, just an experiment for us to be able to use labels to build images

@fruch
Copy link

fruch commented Aug 26, 2024

@CodeLieutenant @fruch If I understand it correctly this fix has not landed yet (probably because of #15). Now almost all rolling upgrade tests for the latest 2024.1.8 RC are failing with the:

  < t:2024-08-24 13:20:42,464 f:base.py         l:146  c:RemoteLibSSH2CmdRunner p:ERROR > Error executing command: "sudo  docker exec e450cf4c5db93281c796427c59a663bcb48843bc9d04a8c9fdb5ac9f7a218d30 /bin/sh -c 'echo TAG: loader_idx:1-cpu_idx:0-keyspace_idx:1; STRESS_TEST_MARKER=TH7AXMN52O2XWGTSVEW2; cassandra-stress read no-warmup cl=QUORUM duration=10m -schema keyspace=keyspace1 '"'"'replication(strategy=NetworkTopologyStrategy,replication_factor=3) compression=SnappyCompressor'"'"' -mode cql3 native compression=snappy  user=cassandra password=cassandra -rate threads=1000 -pop seq=1..10100200 -log interval=5 -node 10.4.2.25,10.4.1.228,10.4.1.48,10.4.3.83 -errors skip-unsupported-columns'"; Exit status: 1
< t:2024-08-24 13:20:42,464 f:base.py         l:148  c:RemoteLibSSH2CmdRunner p:DEBUG > STDOUT:   0 []
< t:2024-08-24 13:20:42,464 f:base.py         l:148  c:RemoteLibSSH2CmdRunner p:DEBUG > Total GC count            : 0
< t:2024-08-24 13:20:42,464 f:base.py         l:148  c:RemoteLibSSH2CmdRunner p:DEBUG > Total GC memory           : 0.000 KiB
< t:2024-08-24 13:20:42,464 f:base.py         l:148  c:RemoteLibSSH2CmdRunner p:DEBUG > Total GC time             :    0.0 seconds
< t:2024-08-24 13:20:42,464 f:base.py         l:148  c:RemoteLibSSH2CmdRunner p:DEBUG > Avg GC time               :    NaN ms
< t:2024-08-24 13:20:42,464 f:base.py         l:148  c:RemoteLibSSH2CmdRunner p:DEBUG > StdDev GC time            :    0.0 ms
< t:2024-08-24 13:20:42,464 f:base.py         l:148  c:RemoteLibSSH2CmdRunner p:DEBUG > Total operation time      : 00:10:00
< t:2024-08-24 13:20:42,464 f:base.py         l:148  c:RemoteLibSSH2CmdRunner p:DEBUG > 
< t:2024-08-24 13:20:42,464 f:base.py         l:148  c:RemoteLibSSH2CmdRunner p:DEBUG > FAILURE
< t:2024-08-24 13:20:42,464 f:base.py         l:150  c:RemoteLibSSH2CmdRunner p:DEBUG > STDERR: Failed to connect over JMX; not collecting these stats
< t:2024-08-24 13:20:42,464 f:base.py         l:150  c:RemoteLibSSH2CmdRunner p:DEBUG > java.lang.IllegalStateException: The requested compression is not available (some compression require a JAR to be found in the classpath)

This fix has landed in v3.13.0 just SCT is not updated with that tag.
#15 does not have anything with this, just an experiment for us to be able to use labels to build images

@CodeLieutenant what is stopping us from updating SCT with the new image ?

@CodeLieutenant
Copy link
Collaborator Author

@CodeLieutenant @fruch If I understand it correctly this fix has not landed yet (probably because of #15). Now almost all rolling upgrade tests for the latest 2024.1.8 RC are failing with the:

  < t:2024-08-24 13:20:42,464 f:base.py         l:146  c:RemoteLibSSH2CmdRunner p:ERROR > Error executing command: "sudo  docker exec e450cf4c5db93281c796427c59a663bcb48843bc9d04a8c9fdb5ac9f7a218d30 /bin/sh -c 'echo TAG: loader_idx:1-cpu_idx:0-keyspace_idx:1; STRESS_TEST_MARKER=TH7AXMN52O2XWGTSVEW2; cassandra-stress read no-warmup cl=QUORUM duration=10m -schema keyspace=keyspace1 '"'"'replication(strategy=NetworkTopologyStrategy,replication_factor=3) compression=SnappyCompressor'"'"' -mode cql3 native compression=snappy  user=cassandra password=cassandra -rate threads=1000 -pop seq=1..10100200 -log interval=5 -node 10.4.2.25,10.4.1.228,10.4.1.48,10.4.3.83 -errors skip-unsupported-columns'"; Exit status: 1
< t:2024-08-24 13:20:42,464 f:base.py         l:148  c:RemoteLibSSH2CmdRunner p:DEBUG > STDOUT:   0 []
< t:2024-08-24 13:20:42,464 f:base.py         l:148  c:RemoteLibSSH2CmdRunner p:DEBUG > Total GC count            : 0
< t:2024-08-24 13:20:42,464 f:base.py         l:148  c:RemoteLibSSH2CmdRunner p:DEBUG > Total GC memory           : 0.000 KiB
< t:2024-08-24 13:20:42,464 f:base.py         l:148  c:RemoteLibSSH2CmdRunner p:DEBUG > Total GC time             :    0.0 seconds
< t:2024-08-24 13:20:42,464 f:base.py         l:148  c:RemoteLibSSH2CmdRunner p:DEBUG > Avg GC time               :    NaN ms
< t:2024-08-24 13:20:42,464 f:base.py         l:148  c:RemoteLibSSH2CmdRunner p:DEBUG > StdDev GC time            :    0.0 ms
< t:2024-08-24 13:20:42,464 f:base.py         l:148  c:RemoteLibSSH2CmdRunner p:DEBUG > Total operation time      : 00:10:00
< t:2024-08-24 13:20:42,464 f:base.py         l:148  c:RemoteLibSSH2CmdRunner p:DEBUG > 
< t:2024-08-24 13:20:42,464 f:base.py         l:148  c:RemoteLibSSH2CmdRunner p:DEBUG > FAILURE
< t:2024-08-24 13:20:42,464 f:base.py         l:150  c:RemoteLibSSH2CmdRunner p:DEBUG > STDERR: Failed to connect over JMX; not collecting these stats
< t:2024-08-24 13:20:42,464 f:base.py         l:150  c:RemoteLibSSH2CmdRunner p:DEBUG > java.lang.IllegalStateException: The requested compression is not available (some compression require a JAR to be found in the classpath)

This fix has landed in v3.13.0 just SCT is not updated with that tag.
#15 does not have anything with this, just an experiment for us to be able to use labels to build images

@CodeLieutenant what is stopping us from updating SCT with the new image ?

most likely nothing, it's just not well tested

@fruch
Copy link

fruch commented Aug 26, 2024

@CodeLieutenant @fruch If I understand it correctly this fix has not landed yet (probably because of #15). Now almost all rolling upgrade tests for the latest 2024.1.8 RC are failing with the:

  < t:2024-08-24 13:20:42,464 f:base.py         l:146  c:RemoteLibSSH2CmdRunner p:ERROR > Error executing command: "sudo  docker exec e450cf4c5db93281c796427c59a663bcb48843bc9d04a8c9fdb5ac9f7a218d30 /bin/sh -c 'echo TAG: loader_idx:1-cpu_idx:0-keyspace_idx:1; STRESS_TEST_MARKER=TH7AXMN52O2XWGTSVEW2; cassandra-stress read no-warmup cl=QUORUM duration=10m -schema keyspace=keyspace1 '"'"'replication(strategy=NetworkTopologyStrategy,replication_factor=3) compression=SnappyCompressor'"'"' -mode cql3 native compression=snappy  user=cassandra password=cassandra -rate threads=1000 -pop seq=1..10100200 -log interval=5 -node 10.4.2.25,10.4.1.228,10.4.1.48,10.4.3.83 -errors skip-unsupported-columns'"; Exit status: 1
< t:2024-08-24 13:20:42,464 f:base.py         l:148  c:RemoteLibSSH2CmdRunner p:DEBUG > STDOUT:   0 []
< t:2024-08-24 13:20:42,464 f:base.py         l:148  c:RemoteLibSSH2CmdRunner p:DEBUG > Total GC count            : 0
< t:2024-08-24 13:20:42,464 f:base.py         l:148  c:RemoteLibSSH2CmdRunner p:DEBUG > Total GC memory           : 0.000 KiB
< t:2024-08-24 13:20:42,464 f:base.py         l:148  c:RemoteLibSSH2CmdRunner p:DEBUG > Total GC time             :    0.0 seconds
< t:2024-08-24 13:20:42,464 f:base.py         l:148  c:RemoteLibSSH2CmdRunner p:DEBUG > Avg GC time               :    NaN ms
< t:2024-08-24 13:20:42,464 f:base.py         l:148  c:RemoteLibSSH2CmdRunner p:DEBUG > StdDev GC time            :    0.0 ms
< t:2024-08-24 13:20:42,464 f:base.py         l:148  c:RemoteLibSSH2CmdRunner p:DEBUG > Total operation time      : 00:10:00
< t:2024-08-24 13:20:42,464 f:base.py         l:148  c:RemoteLibSSH2CmdRunner p:DEBUG > 
< t:2024-08-24 13:20:42,464 f:base.py         l:148  c:RemoteLibSSH2CmdRunner p:DEBUG > FAILURE
< t:2024-08-24 13:20:42,464 f:base.py         l:150  c:RemoteLibSSH2CmdRunner p:DEBUG > STDERR: Failed to connect over JMX; not collecting these stats
< t:2024-08-24 13:20:42,464 f:base.py         l:150  c:RemoteLibSSH2CmdRunner p:DEBUG > java.lang.IllegalStateException: The requested compression is not available (some compression require a JAR to be found in the classpath)

This fix has landed in v3.13.0 just SCT is not updated with that tag.
#15 does not have anything with this, just an experiment for us to be able to use labels to build images

@CodeLieutenant what is stopping us from updating SCT with the new image ?

most likely nothing, it's just not well tested

open a PR in SCT, and we'll see how to test it

@dimakr
Copy link
Contributor

dimakr commented Aug 28, 2024

open a PR in SCT, and we'll see how to test it

@CodeLieutenant can we plan adding this thing to SCT? It's still a blocker for testing rolling upgrades in 2014.1.8

@CodeLieutenant
Copy link
Collaborator Author

open a PR in SCT, and we'll see how to test it

@CodeLieutenant can we plan adding this thing to SCT? It's still a blocker for testing rolling upgrades in 2014.1.8

Already Created PR 8477

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mykaul mykaul mykaul left review comments

@fruch fruch Awaiting requested review from fruch

@roydahan roydahan Awaiting requested review from roydahan

Assignees

@CodeLieutenant CodeLieutenant

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Linker error not able to find Snappy compression library
5 participants
@CodeLieutenant @roydahan @dimakr @fruch @mykaul