Fix DB unit tests

[Kehrlann]

Context

@fhanik and I have noticed that some unit tests did not respect the profile that was passed in, and always ran against an HSQL database, or were ignored

When you ran:

./gradlew '-Dspring.profiles.active=postgresql,default' test

without a postgres database running, the tests would pass, which should not be the case when a database is involved.

This PR only addresses tests, by fixing this issue, fixing failing tests and adding tooling.

Consequences of the PR

• Higher confidence when making db-related changes
• Running against postgresql or mysql is now twice as slow. On my m1 mac, it goes from "just under 5 minutes" to "in the 10-12 minute range"
• The tests now run against the actual DB and you may have a problem with database names - they are uaa_1, uaa_2, etc ; all the way up to 24. Sometimes your tests may complain that uaa_33 or something similar does not exist. In order to avoid most problems, please run your gradle tests with --no-daemon. If you want more details, head over to docs/testing.md.

Fixes

The tests in question:

• Any test with @WithDatabaseContext ; because that annotation had @ActiveProfiles("default"). We removed it.
• TableAndColumnNormalizationTest, again annotated with @ActiveProfiles("default")
• And possibly some, but not all, the tests annotated with @DefaultTestContext

When we re-enabled the multi-profile support, we had issues in some tests, both with Postgres and MySQL. We have fixed those issues. The main issues were:

• The column name groups was improperly quoted in MySQL
• Database connection leaks, both in prod and in the tests
• Some tests cannot work on certain databases
• Some tests have casing and sorting issues
• There are timezone issues specifically with MySQL when the host running your tests is ahead of UTC (e.g. Europe/Paris) ; this is addressed by coercing the MySQL container to the same TZ as your host.

Addition: scripts/docker-compose.yml

We added a docker compose file to have MySQL and Postgres running locally. Using run-unit-tests.sh was not working well on our ARM-based macs. This is not expected to be the final setup, but a first step to help us run tests fast.

We will converge startup scripts later on.

Addition: @EnabledWithProfile / @DisabledWithProfile

To selectively turn off some tests based on profiles ; instead of relying on custom use of Assume.assumeXXX and Assumptions.assumeXXX which are harder to locate in the code.

See the javadoc for their usage.

Addition: docs/testing.md

Some findings on testing have been recorded in docs/testing.md.

[fhanik]

New findings

1. Tests have database connection leaks. Many tests use JdbcTemplate.getDataSource().getConnection() and never call connection.close()

2. The @WithTimeZone cause some tests to fail. Need to be investigated. For now, this annotation is limited to mysql

3. Columns have mismatched data types causing failure in later versions of MySQL (users.id is a padded char while authz_approvals.user_id is a varchar.

4. Because tests didn't run against MySQL or PostgreSQL, we found many places where the word groups was used in queries and cause BadSqlGrammarException

5. All the MockMvc tests also had hardcoded to use in memory HSQLDB. This was fixed.

6. MySQL has in later versions deprecated PAD_CHAR_TO_FULL_LENGTH so we have converted users.id and oauth_client_details.created_by to VARCHAR

Items Left to Do

1. We can add maildev to docker-compose.yml (not to conflict with the in memory SMTP server SimpleSmtpServer that the integration tests run
2. We should enhance the LDAP image in docker-compose.yml to properly configure and seed the LDAP server - see scripts/integration-tests.sh
3. RateLimiter for cargo is causing 429 error on my Macbook M4
4. /etc/hosts requires the following entrie

127.0.0.1 testzone1.localhost
127.0.0.1 testzone2.localhost
127.0.0.1 testzone3.localhost
127.0.0.1 testzone4.localhost
127.0.0.1 testzonedoesnotexist.localhost
127.0.0.1 oidcloginit.localhost
127.0.0.1 testzoneinactive.localhost

5. Verify that hsqldb isn't unintentionally used in any tests

6. Fix remaining integration tests - current status has IT failures (assuming rate limits have been adjusted) - and one of them relates to LDAP preloading data (bullet 2 above) LdapIntegrationTests.test_LDAP_Custom_User_Attributes_In_ID_Token

7. is CHAR to VARCHAR a breaking change?

Notes

./gradlew \
  -Dspring.profiles.active=mysql,default \
  :cloudfoundry-identity-uaa:test \
  --tests org.cloudfoundry.identity.uaa.login.LoginMockMvcTests

This previously passed even if I shut down MySQL database. proving that the DB was not tested.

[Kehrlann]

Update:

The @WithTimeZone cause some tests to fail. Need to be investigated. For now, this annotation is limited to mysql

This was fixed with a new composed annotation, @DatabaseTestUTC

Verify that hsqldb isn't unintentionally used in any tests

Verified that it's not used in unit tests, by removing the hsqldb dependency entirely.

[Kehrlann]

Side note, there was a flake earlier on:

DeprecatedUaaTokenServicesTests#testCreateAccessTokenRefreshGrantSomeScopesAutoApproved, for both variants:

• https://github.com/cloudfoundry/uaa/actions/runs/12318536514/job/34383499756?pr=3186#step:5:8381
• https://github.com/cloudfoundry/uaa/actions/runs/12318536514/job/34383499756?pr=3186#step:5:10149

Errors were:

    java.lang.AssertionError: 
    Expected: should be valid for <is <43200>>
         but:  but was <43199>
        at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
        at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:6)
        at org.cloudfoundry.identity.uaa.oauth.DeprecatedUaaTokenServicesTests.testCreateAccessTokenRefreshGrantSomeScopesAutoApproved(DeprecatedUaaTokenServicesTests.java:925)

And:

    java.lang.AssertionError: 
    Expected: Refresh token should be valid for null
         but:  was <2591999>
        at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
        at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:6)
        at org.cloudfoundry.identity.uaa.oauth.DeprecatedUaaTokenServicesTests.testCreateAccessTokenRefreshGrantSomeScopesAutoApproved(DeprecatedUaaTokenServicesTests.java:930)

[strehle]

maybe was not good enough documented, but if I had to use a DB in past I setup the docker for it via

docker pull cfidentity/uaa-mysql-8 && pushd ~/work/uaa && docker run -p 3306:3306 --name db --rm --env MYSQL_ROOT_HOST=% --env DB=mysql --env RUN_TESTS=false --volume $PWD:/root/uaa --privileged --tty --interactive --shm-size=1G cfidentity/uaa-mysql-8 /root/uaa/scripts/integration-tests.sh mysql,default

because this uses the same images which run in all of our tests

I guess for your Mac/Processor this is not possible... thus a plain Linux is always the best choice for development

[Kehrlann]

Thanks for the trick running the tests.

It is possible to run on my Mac, it's just slower than a plain x86 proc.

The reason for the docker-compose is mostly to be able to debug DB tests inside my IDE :)

[strehle]

Thanks for the trick running the tests.

It is possible to run on my Mac, it's just slower than a plain x86 proc.

The reason for the docker-compose is mostly to be able to debug DB tests inside my IDE :)

yeah the docker-compose helps to better understand the setup. for ldap we need similar thing. https://github.com/cloudfoundry/uaa?tab=readme-ov-file#connecting-uaa-to-local-ldap-server

I only start DBs, when we have new columns with an index on it, then we check with explain and SQL if the index is really used or not... for all other cases... only a new column and no index I dont need the real DB

[strehle]

Ok for me

[strehle]

debug DB tests inside my IDE :)

if I started a DB on my machine and map ports , e.g. -p 3306:3306

I can debug then everything and can access my tables with a GUI like DBeaver, but docker-compose is ok

[duanemay]

👍