Docker container killed after uploading PDF file on Apple Silicon (macOS Sonoma 14.5)

[Daniel-EST]

Description:

After updating to macOS Sonoma 14.5, I've noticed that a Docker container from image lfoppiano/grobid:0.8.0 is being killed shortly after sending a PDF file to Grobid. Before macOS Sonoma 14.5 version it used to work as intended. This is happening on both M3 and M1 processors.

The container starts normally when run with the command:

docker run -d --ulimit core=0 -p 8070:8070 --name grobid lfoppiano/grobid:0.8.0

However, it is terminated immediately afterwards uploading a PDF file with the following error message:

2024-05-23 11:38:51 [thread 44 also had an error]
2024-05-23 11:38:51 #
2024-05-23 11:38:51 # A fatal error has been detected by the Java Runtime Environment:
2024-05-23 11:38:51 #
2024-05-23 11:38:51 #  SIGSEGV (0xb) at pc=0x00007ffffee7ac84, pid=1, tid=39
2024-05-23 11:38:51 #
2024-05-23 11:38:51 # JRE version: OpenJDK Runtime Environment (17.0.2+8) (build 17.0.2+8-86)
2024-05-23 11:38:51 # Java VM: OpenJDK 64-Bit Server VM (17.0.2+8-86, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, g1 gc, linux-amd64)
2024-05-23 11:38:51 # Problematic frame:
2024-05-23 11:38:51 # V  [libjvm.so+0xb37c84]  markWord::displaced_mark_helper() const+0x14
2024-05-23 11:38:51 #
2024-05-23 11:38:51 # No core dump will be written. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
2024-05-23 11:38:51 #
2024-05-23 11:38:51 # An error report file with more information is saved as:
2024-05-23 11:38:51 # /opt/grobid/hs_err_pid1.log
2024-05-23 11:38:51 #
2024-05-23 11:38:51 # If you would like to submit a bug report, please visit:
2024-05-23 11:38:51 #   https://bugreport.java.com/bugreport/crash.jsp
2024-05-23 11:38:51 #
2024-05-23 11:38:51 
2024-05-23 11:38:51 [error occurred during error reporting (), id 0xb, SIGSEGV (0xb) at pc=0x00007fffff5d7602]

Environment:

• Operating System: macOS Sonoma 14.5
• Docker version: 26.1.1
• Apple Silicon: Yes

[lfoppiano]

I @Daniel-EST thank you for your interest in Grobid. Now that you mention it, I've scheduled the update on Sonoma 14.5 tonight on my M2 💦

Could you try the image lfoppiano/grobid:0.8.0-arm here?

For information, you can follow up the background information about this image on #1014

[m-blasiak]

@lfoppiano I got the exact same issue after updating to Sonoma 14.5. I tried using lfoppiano/grobid:0.8.0-arm, as well as latest-crf unfortunately the issue persists.

[Daniel-EST]

@lfoppiano, just tested the lfoppiano/grobid:0.8.0-arm version and the same problem persists. However, while going through the docker images tags lfoppiano/grobid:latest-full worked as intended. Where do I get more information about the differences between both versions?

[lfoppiano]

OK, so the latest-* images are generally for development.

In this case you're lucky because it's the same as lfoppiano/grobid/0.8.0-full-slim here

This image was built from the branch 0.8.0-fixes which is the version 0.8.0 with post-release fixes.

could you try to run again the lfoppiano/grobid:0.8.0 or lfoppiano/grobid:0.8.0-arm and specify the --init param, and let me know if there is any change?

[Daniel-EST]

could you try to run again the lfoppiano/grobid:0.8.0 or lfoppiano/grobid:0.8.0-arm and specify the --init param, and let me know if there is any change?

Running
docker run --init --ulimit core=0 -p 8070:8070 --name grobid lfoppiano/grobid:0.8.0-arm
The error persisted, with the same trace as described earlier. However, running:

docker run --init --ulimit core=0 -p 8070:8070 --name grobid lfoppiano/grobid:0.8.0-full-slim
I was able to upload an PDF file.

[lfoppiano]

OK, I'm glad you have a working image, but I don't know why it works... Maybe something has changed in the way java is handled 🤔

About the lfoppiano/grobid:0.8.0-full-slim, does it work well with more than one PDF? Could you run it without issues on let's say 1000 PDF documents?

[Daniel-EST]

I'm going to create a script and try to upload multiple PDF documents. However, I've noticed that the container is now more resource-intensive, consuming significantly more RAM and CPU. Some huge documents that I was able to parse in the past are now killing the container due to lack of resources.

[lfoppiano]

Could you share the log and remind me which image are you using?
If you are using the lfoppiano/grobid:0.8.0-full-slim make sure that you are using the Deep learning models without GPU.

[lfoppiano]

@Daniel-EST did you manage to fix the issue? I've built a crf-only image that should support ARM, but I haven't tested. If you have time, it's on #1165 .

[Daniel-EST]

@lfoppiano Just tested the docker image lfoppiano/grobid:latest-crf-multi-arch from #1165.

Everything worked well, I uploaded about 10 documents and all were correctly parsed.

docker run -d --ulimit core=0 --platform linux/amd64 -p 8070:8070 --name grobid lfoppiano/grobid:latest-crf-multi-arch

It is important to add the --platform linux/amd64 to force the usage of amd64 architecture. Otherwise you might get the following error:
rosetta error: failed to open elf at /lib64/ld-linux-x86-64.so.2

Tested on macOS Sequoia 15.0, on a M3 processor. Important to notice that the macOS version was updated since I opened the issue.

[lfoppiano]

Thanks for testing @Daniel-EST .

I did some more tests, I think you should add also --init or the child process won't be cleaned up. I tried without and the JVM was crashing.
After adding --init it seems more stable, but I would need to test it a bit more.

I've added some documentation on the related branch, here.

[lfoppiano]

One solution would be to change the fork mechanism based on the architecture, I've been trying to change the final Grobid ENV variable see here without success. Any help is welcome 😄