fix: added type to createConnectionParams, removed safeguard [restore db conversions]

[Delogon]

• Resolves: [Bug]: occ db:convert-type broken since PrimaryReadReplicaConnection support added #45257

Summary

This should fix the db conversion bug. $type was added as a parameter to createConnectionParams in the ConnectionFactory.php. There is also a check if the type value is set if not the default is loaded from the config. There is also an additional type check to make sure mysql.utf8mb4 is only used with mysql. The parameters of createConnectionParams in Connection Factory has also been updated, and the safeguard has been removed. The solution was provided in the issue comments by @Queuecumber, @normen, @lightonflux and me.

TODO

• An error occurs when using group folders because the table 'oc_group_folders_acl' is missing in MariaDB see [Bug]: occ db:convert-type broken since PrimaryReadReplicaConnection support added #45257

Checklist

• Code is properly formatted
• Sign-off message is added to all commits
• Tests (unit, integration, api and/or acceptance) are included
• Screenshots before/after for front-end changes
• Documentation (manuals or wiki) has been updated or is not required
• Backports requested where applicable (ex: critical bugfixes)

[WhiskyDelta]

/backport to stable30

[WhiskyDelta]

/backport to stable29

[Delogon]

@juliushaertl @joshtrichards cloud you review this? Or do you know who could?

[joshtrichards]

This is a proposed alternative to #46908 (not yet reviewed - just noting since that one seemed to have blockers which this will need to make sure it's addressing).

[joshtrichards]

@Delogon Please stop rebasing / re-merging with master. It's unnecessary at this juncture and just keeps resetting all the tests. :)

[github-actions]

Hello there,
Thank you so much for taking the time and effort to create a pull request to our Nextcloud project.

We hope that the review process is going smooth and is helpful for you. We want to ensure your pull request is reviewed to your satisfaction. If you have a moment, our community management team would very much appreciate your feedback on your experience with this PR review process.

Your feedback is valuable to us as we continuously strive to improve our community developer experience. Please take a moment to complete our short survey by clicking on the following link: https://cloud.nextcloud.com/apps/forms/s/i9Ago4EQRZ7TWxjfmeEpPkf6

Thank you for contributing to Nextcloud and we hope to hear from you soon!

(If you believe you should not receive this message, you can add yourself to the blocklist.)

[WhiskyDelta]

Is this going to be fixed or should we maybe correct the documentation and remove this feature?
Converting Database Type - Nextcloud documentation

[SnejPro]

Is this going to be fixed or should we maybe correct the documentation and remove this feature? Converting Database Type - Nextcloud documentation

I think removing this feature is a very bad option. The officially suggested method to deploy Nextcloud is with AIO right now. But for the migration it is mandatory to convert the database from mysql to postgres. So if this feature is removed in the future, it should be announced a certain time in advance to give users the opportunity to migrate.
I know the issue is already there, but for now i do not see another option than solving it.

But there should be an information in the documentation that this feature is currently broken and therefore disabled.

[SnejPro]

I used this PR to migrate my installation from mariadb to postgres a few hours ago. For now everything works fine.

[FlorianFritz]

Hi, I tried to convert my mariadb to pgsql with this code today. It failed with following exceptions when converting the "oc_accounts" table

In ExceptionConverter.php line 62:

[Doctrine\DBAL\Exception\UniqueConstraintViolationException (1062)]
An exception occurred while executing a query: SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '0f97f6aa-3975-4b81-935f-de4a1bac7a2d' for ke
y 'PRIMARY'

In Exception.php line 28:

[Doctrine\DBAL\Driver\PDO\Exception (1062)]
SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '0f97f6aa-3975-4b81-935f-de4a1bac7a2d' for key 'PRIMARY'

In Statement.php line 130:

[PDOException (23000)]
SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '0f97f6aa-3975-4b81-935f-de4a1bac7a2d' for key 'PRIMARY'

When I check my mysql database there is only one occurance of the uid.

MariaDB [nextcloud]> SELECT * FROM oc_accounts;
[...]
| 0f97f6aa-3975-4b81-935f-de4a1bac7a2d | {"displayname":{"value":"nextcloud","scope":"v2-federated", [...]
[...]

So '0f97f6aa-3975-4b81-935f-de4a1bac7a2d' is the 'uid' of the "nextcloud" user.

When I count this 'uid' I only get one result (as expected)

MariaDB [nextcloud]> SELECT count() FROM oc_accounts WHERE uid="0f97f6aa-3975-4b81-935f-de4a1bac7a2d";
+----------+
| count() |
+----------+
| 1 |
+----------+
1 row in set (0.000 sec)

Is this related or should I open a new issue?

[karlsandreas]

For those in need to convert their db. I used this PR on 30.0.2 to migrate from MySQL to PostgreSQL without any problems.

[HexagonCoder]

Can someone please review and fix this so it can be merged? Bug #45257 has been in need of fixing for many months. Earlier this year I needed to convert a NC DB from MariaDB to Postgres and had to put in a lot of extra time and effort to work around this bug, and now, when I am converting the DB of another NC instance, I am shocked to find that it still hasn't been fixed.

Please, this is an important feature. Can someone take care of it soon?

I don't want to sound demanding or ungrateful, I really appreciate everyone's time they put into developing FOSS, I just want to stress how important this feature is and how frustrating it is to users that the bug still hasn't been fixed.

Thank you and happy new year everyone!

[xalt7x]

I've tried mySQL > PostgreSQL conversion on the freshly installed Nextcloud instance

Environment:
Debian 12
Mariadb 10.11.6-MariaDB-0+deb12u1 (Debian package)
PostgreSQL 15+248 (Debian package), also tried PostgreSQL 17
Nextcloud v30.0.4 + patches from this PR

sudo -u www-data php /var/www/nextcloud/occ db:convert-type --all-apps --password="${old_dbpassword}" pgsql oc_nextcloud 127.0.0.1 nextcloud_database

1. If the installation of “recommended” apps was skipped, the database converts succesfully!
2. If the "Talk" app was previously installed - script informs

The following tables will not be converted:
oc_talk_commands
Continue with the conversion (y/n)?

I've tried different options with this:
a. If I agree to continue (y)
conversion proceeds without visible errors and server starts with PostgreSQL
b. If I disable Talk app, remove it and restart server
script still complains
c. If I remove app and try to drop oc_talk_commands table
it breaks conversion

• oc_talk_rooms
  0/2 [>---------------------------] 0% < 1 sec/< 1 sec
  In ExceptionConverter.php line 68:

  An exception occurred while executing a query: SQLSTATE[42703]: Undefined column: 7 ERROR:
  column "last_activity" of relation "oc_talk_rooms" does not exist
  LINE 1: ...pe", "password", "active_since", "active_guests", "last_acti...

If after that I try to drop oc_talk_rooms - conversion breaks due to duplicated keys

An exception occurred while executing a query: SQLSTATE[23505]: Unique violation: 7 ERROR: duplicate key value violates unique c
onstraint "oc_storages_pkey"
DETAIL: Key (numeric_id)=(1) already exists.

d. If prior to running script I remove Talk app and drop both oc_talk_commands and oc_talk_rooms tables conversion proceeds without visible errors and server starts with PostgreSQL.

---

So I guess, it can be re-enabled for instances without apps.
As for the "Talk" and other apps, I'm not entirely sure...

mistake