-
-
Notifications
You must be signed in to change notification settings - Fork 4.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Bug]: occ db:convert-type
broken since PrimaryReadReplicaConnection
support added
#45257
Comments
Logging the Parameter FromDB
ToDB
It is notable that in the second case (which should be the postgres-database configured via the CLI arguments), the host, dbname and port in the top-level array are set correctly from the CLI, Both the |
Possibly related: #45097 I don't have time to look tonight, but your analysis seems sound (and, given the added evidence from the log output, even more so). Cc: @juliushaertl |
I can verify but can't contribute logs. I used the --clear-schema option in my attempt and my original MariaDB DB got deleted. Good thing I have backups. |
I modified https://github.com/nextcloud/server/blob/master/core/Command/Db/ConvertType.php#L247
and seems all converted Also have to remove mysql.utf8mb4 parameter from config.php to perform conversion to remove client_encoding from params of DbFactory |
Got hit by this a week ago. Is that particular Nextcloud instance completely screwed? |
Yes, unfortunately. 😢 The end result is the parameters get swapped so it likely dropped the source database. Fortunately, recovery should be a simple matter of restoring from backup (just the db portion): https://docs.nextcloud.com/server/latest/admin_manual/maintenance/restore.html#restore-database (or whatever other mechanism you utilize in your environment for database backup/recovery) |
@Delagen Mind pushing that as PR? I know there may be additional changes needed, but it'll start things in motion. |
Needed until #45257 is addressed to prevent data loss Signed-off-by: Josh <josh.t.richards@gmail.com>
@joshtrichards I don't mind that my changes solve the problem correctly. It only makes possible for me successful conversion from MySQL to Postgres |
I have just experienced this problem on a new installation. |
Needed until #45257 is addressed to prevent data loss Signed-off-by: Josh <josh.t.richards@gmail.com>
Would not be it the simplest solution, if the ConnectionFactory::getConnection() method would first check if the |
Needed until #45257 is addressed to prevent data loss Signed-off-by: Josh <josh.t.richards@gmail.com>
Hi, is this #45013 fixing the problem? |
@Delogon I'm getting an error message:
I'm not sure if relevant, but this table is empty for me.
removing the group folder plugin solves the issue... however, now the group folders are gone, but this I can survive. |
@Delogon Since your fix seems to work, do you want to make a pull request? If not, I might be able to do it. |
So I am also trying to move from mariadb to postgres and on nextcloud v30. I did the fix that @Delogon posted and that got me further. However i end up with this during the conversion.
Any ideas? |
Found this related bug and it seems to be the same. Looks like NC has a bit more work to do on the postgres side |
@hagak what is your encoding in mysql? |
UTF8 the issue from what i gather in that other related bug that i apparently forgot to put the link to. Shows that it is due to postgres schema using a TEXT column instead of a Bytea . I believe there is a PR to resolve that as well Particular data in the oc_properties table with a valuetype=3. I only had one entry that was causing this so I just manually deleted that row and the convert finished. Add the link to the issue number #37754 |
I created the pull request #48666 but tbh I don't know if I did that correct, since creation nothing has happened. @joshtrichards or @juliusknorr maybe you could tell me what to do, to get it merged. |
I want to migrate from a standalone nginx/php-fpm/mariadb Nextcloud installation to AIO "The official Nextcloud installation method" but I'm blocked by this issue since several weeks. Is the patch provided by @Delogon here #45257 (comment) or in the PR #48666 safe to use? |
I cannot answer that but even if it is you should make a database backup anyway |
Of course. I'm just worried to use it, don't detect any problem, and start using my new postgres DB without detecting a possible data loss (or other problem) :-/ |
I just applied the PR [#48666] manually on nextcloud 29.0.9-fpm-alpine and successfully converted from mysql to postgres. So far everyhting looks good. |
I'll be, probably, trying to migrate this evening. I'll report how it goes. |
That bug is already known. A potential solution would be to disable the group folder plugin, but I do not know how suitable that is for you.
|
That crossed my mind but Nextcloud kinda doesn't want to do it. I don't use the plugin but it was disabled but installed. While writing this I managed to removed it with EDIT: God Talk is broken now... |
Ok. The database migration itself went smoothly after disabling group folders. Importing data into AIO is the part that was full of pain. It had issues with the database dump, then the data files, then 3 apps that I had installed, ... But it seems to be done now. MAKE BACKUPS and keep them! Even after you migrate and everything seems well you might find something broken. I, for example, had all my calendar entries missing and I had to redump the PostgreSQL database, change the data directory within it and the owner of the database and then re-import it again to get it back. |
I applied the path and run the
I'm not sure if it comes from an app and which (no 'theming' related one as I can see). I dumped the mysql db and tried to find any duplicate entry with no success edit: it seems to come from a default app |
disabling user theming seems sufficient |
I've applied the patch too and tried to run the migration, got stuck with the same issue of @daks. After fixing it I was able to perform the migration, however I've got these two tables not being converted:
For |
This is weird as I have 2FA enabled and I set up. Migration worked without issues in regards to 2FA as it's still working. |
I think that probably it does print that message because the table is empty and there is nothing to migrate:
|
I just discovered that the db:convert-type has been broken for a while, as I was needing it! Regards |
2024-12-08 THAT WAS NOT A GOOD IDEA... EVERYTHING WORKED FINE IN POSTGRESQL UNTIL I NEEDED TO UPGRADE MY INSTANCE... I REVERTED TO MARIADB
|
I corrected my last comment... It broke! Please, Nextcloud team, fix the DB migration tool! My MariaDB instance is not as stable as I would like it to be, an now, I can't migrate my database to postgreSQL. I lack the technical knowlege to do it myself, and the proposed manual fixes listed earlier do not work for me (it fails to create the db shema) |
I hate to bump, but it's been 7 months and it's almost Christmas. A fix as a Christmas gift (even a late one) would be lovely. |
PR 48666 to re-enable DB conversion is still active. |
Bug description
The OCC-Command
db:convert-type
seems to ignore the DB properties (type, user, host, database, port, password) provided through the CLI and instead operates on the database configured inconfig.php
I believe this may be related to the commit 79c4986 as this reworked the parameter handling in
ConnectionFactory.php
.If I read this correctly, the default parameters are now always generated based on the system config instead of the passed arguments.
Running
db:convert-type --clean-schema
drops the tables in the "old" database.Steps to reproduce
php occ db:convert-type --password=<PASSWORD> --port=5432 pgsql nextcloud_app postgres nextcloud
(note: the hostpostgres
is a Kubernetes service name)oc_accounts
, I assume because it tries to migrate the content of the old database into itself.Expected behavior
db:convert-type
Installation method
Community Docker image
Nextcloud Server version
29
Operating system
Debian/Ubuntu
PHP engine version
None
Web server
None
Database engine version
MySQL
Is this bug present after an update or on a fresh install?
None
Are you using the Nextcloud Server Encryption module?
None
What user-backends are you using?
Configuration report
List of activated Apps
Nextcloud Signing status
Nextcloud Logs
No response
Additional info
nextcloud:29.0.0
mysql:8.4.0
postgres:16.3
The text was updated successfully, but these errors were encountered: