[bitnami/ghost] startup issues when upgrading past v5.68.0

[97WaterPolo]

Name and Version

bitnami/ghost:5.68.0

What architecture are you using?

amd64

What steps will reproduce the bug?

1. Using Unraid (or docker command line), pull and run the containers v5.68.0

2. Application will start and run,
   

3. Using Unraid (or docker command line), upgrade and pull the next version v5.69.0

4. Application has the same startup message but the container has exited
   

What is the expected behavior?

I expected the upgrade to complete without any issues, I was on a 5.3.X and jumped to latest and it failed to start. I slowly tried differing versions until I came from the upgrade to 5.68.0 > 5.69.0 which is when it started to fail to start. I don't see any logs in the docker container logs and the container isn't ever on long enough for me to check the logs while it is running.

What do you see instead?

The container fails to start once upgrading to 5.69.0

Additional information

Docker Client:
 Version:           20.10.21
 API version:       1.41
 Go version:        go1.18.7
 Git commit:        baeda1f
 Built:             Tue Oct 25 17:56:30 2022
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.21
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.18.7
  Git commit:       3056208
  Built:            Tue Oct 25 18:02:03 2022
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          v1.6.9
  GitCommit:        1c90a442489720eec95342e1789ee8a5e1b9536f
 runc:
  Version:          1.1.4
  GitCommit:        v1.1.4-0-g5fd4c4d1
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

[javsalgar]

Hi,

Could you try with BITNAMI_DEBUG=true as environment variable?

[97WaterPolo]

Looks like the only difference I see in the console output is the Executing SQL Command. There isn't any other debug showing up

[javsalgar]

Another option to further debug the issue could be running a sleep infinity as entrypoint in the ghost container, then enter the container and run this command:

/opt/bitnami/scripts/ghost/entrypoint.sh /opt/bitnami/scripts/ghost/run.sh

If that fails, you can also run the command ghost run with NODE_ENV=development

https://forum.ghost.org/t/how-to-enable-debugging-output-in-ghost/8479/5

[97WaterPolo]

This is after running the command on a sleep infinity version of it!

Since I am trying to debug it, I then run ghost run only for it to exit the running process as soon as I enter the command, just like when I run the script.

On the other hand if I built it from the 5.68.0 version that does work, when I execute your entrypoint.sh and run.sh it doesn't return, which makes me thing the process is running. Not sure why it is breaking out of the process once I upgrade past that version.

[97WaterPolo]

I did some more digging, I run ghost with the -D! parameter and I got an error, looks like in this version it fails to open SQLITE database? Which is weird as everything I have is configured to use mysql. Any idea why it would be trying to initialize SqlLite?

When I try to chase the error "/opt/bitnami/ghost/versions/5.94.0/node_modules/knex-migrator/lib/database.js" I threw in some console.log statements, and it looks like with this version jump is causing it to default to sqlite3 instead of mysql. (see below)

I ran ghost config database.client and it outputted the target DB was MySQL (see very bottom)

I also included my environment variables in-case I am missing one? But I didn't see any on https://github.com/bitnami/containers/tree/main/bitnami/ghost#configuration that I missed.

[javsalgar]

Hi,

Thank you so much for all the digging. This behavior of defaulting to sqlite when the config returns that the database should be mysql seems more like an upstream issue rather than an issue with the Bitnami packaging. I think it is worth sharing with the upstream developers.

https://github.com/TryGhost/Ghost

[97WaterPolo]

Unfortunately they are rejecting it as a bug and stating that it is an environment issue. Going through more of the Ghost documentation it looks like the ghost configuration is loaded from a config.json, not environment variables that I have provided for this container. Any chance you know where those config files are stored within the container and I can manually validate / alter it to see if I can get it working?

Do you think it has anything to do with the env vars? It looks like the old template I used for UnraidOS has it mapped to MARIADB_HOST and MARIADB_PORT_NUMBER rather than GHOST_DATABASE_HOST and GHOST_DATABASE_PORT_NUMBER? If the env vars changed then my legacy container 5.68.X might use those two env vars but when upgrading past 5.69.X it references the new ones prefixed with GHOST_*. Do you think this could be a possibility that if it doesn't find the database host it fallbacks to Sqlite3?

[javsalgar]

Hi!

The configuration file should be in /opt/bitnami/ghost/config.production.json

[97WaterPolo]

v5.94.0

v5.68.0

Looks like the /opt/bitnami/ghost/versions/5.XX.0/node_modules/knex-migrator/lib/database.js is getting the incorrect client, any support from Ghost is claiming it is a docker environment issue not an issue with the application.

I've confirmed that the generated config.production.json file is the exact same between v5.68.0 and 5.98.0 located at /opt/bitnami/ghost/config.production.json. Did maybe the location where the config.production.json file is loaded change?

Any command running ghost config database.client or ghost config. database.connection.password returns the correct information even though console.log in the database.js says otherwise.

[javsalgar]

In case it helps, these are the commands we use to build ghost:

ghost install 5.98.0 --dir /opt/bitnami/ghost --no-prompt --no-setup --no-enable --no-stack --dbhost skip.check.db
yarn add ghost-cli --production
yarn install

[97WaterPolo]

I was running a previous version of mariadb that doesn't seem to support the ALTER TABLE command
/usr/bin/mysql Ver 15.1 Distrib 10.4.21-MariaDB, for debian-linux-gnu (x86_64) using readline 5.2

I ended up upgrading to
/usr/bin/mysql Ver 15.1 Distrib 10.11.8-MariaDB, for Linux (x86_64) using readline 5.1

which is now causing this error, the syntax support should be there, not sure why it is still failing the database migration when I run ghost log

Any suggestions on what I should do?

[javsalgar]

Are you able to run that alter table command manually? Just to ensure if the syntax is supported

[97WaterPolo]

I tried updating to 5.69.0 and it still failed with the same error.

I created an empty table with a column matching the statement, I tried executing it and it failed

I then did a SELECT VERSION() and tried it again and without a restart or any other changes it worked.

I restarted with version 5.69.0 and it looked like it migrated correctly, from this point on I was able to go to the latest and got everything going. Not sure why I had issues after I upgraded the database, but it seems to be all resolved now. Thanks for all the help troubleshooting!

[javsalgar]

Hi!

Thanks for letting us know!

[github-actions]

This Issue has been automatically marked as "stale" because it has not had recent activity (for 15 days). It will be closed if no further activity occurs. Thanks for the feedback.

[github-actions]

Due to the lack of activity in the last 5 days since it was marked as "stale", we proceed to close this Issue. Do not hesitate to reopen it later if necessary.