Running latest wf27, junit5 and arquillian1.7 seems does not execute the test method, while previously used combination wf14, junit4 and arquillian1.5 did

[abregar]

Hi,
I am porting some integration tests based on following combination wildfly14.0.1, java8, junit4.12, arquillian 1.5.0.Final to some latest known good releases wildfly27 java17 junit5.9.1 arquillian 1.7.0.Alpha12. At first, after solving some of the changed dependencies, all looked good - all tests were passing, green.

Then something interesting was observed - some tests, which should not pass, just did pass. Then tried plain simple minimal TestIT class, just some injection, logging and explicit fail(). And the test passed again. Looking closer, shrinkwrapped deployment is created, deployed to managed wildfly27 instance, .. but then seems that no code from test is executed. And deployment is then properly undeployed. This i.e. is the test which just passes:

`
@ExtendWith(ArquillianExtension.class)
public class GreetIT {
private final static Logger LOGGER = Logger.getLogger(GreetIT.class.getName());

@Deployment
public static JavaArchive createDeployment() {
    return ShrinkWrap.create(JavaArchive.class).addClass(GreetingMessage.class).addClass(GreetingService.class)
        .addAsManifestResource(EmptyAsset.INSTANCE, "beans.xml");
}

@Inject
GreetingService service;

@Test
@DisplayName("testing hello")
public void should_create_greeting() {
    System.out.println("test::Hello");
    LOGGER.log(Level.INFO, "test::Hello::logger");
    var message = service.buildGreetingMessage("Hello");
    assertTrue(message.getMessage().startsWith("Hello"),"message matches");
    fail("fail explicitly");
}

`

In my opinion, I would say that deployment part is ok, just the test method is somehow not executed at all.

I was looking to your JIRA to find something on topic, but closest is something on TestNG - https://issues.redhat.com/projects/WFARQ/issues/WFARQ-129, but nothing is reported in the context of latest JUnit5 and Arquillian 1.7.x (which supports Jakarta EE10).
Unfortunately, there is also no issues on topic reported at https://github.com/arquillian/arquillian-core/, but I would say that there must be something with integration against wildfly, but just don't know where to start looking. (With knowledge that previous combination mentioned above, with wildfly 14 and same test scenario, worked flawlessly as expected).

I do kind of apoligise to abuse discussion where the proper track would be issue, but nevertheless, I would like to discuss first where the potential solution to this issue might be. Is it my side setup, wildfly-arquillian integration or arquillian-junit5?

As those tests in previous usage iteration were a great deal in our project, I would also contribute here, but also need some shed of (en)lightenment where possible to start, at least looking or testing.

Thanks in advance for any clue!

[jamezp]

If I add that test to https://github.com/wildfly/wildfly-arquillian/tree/main/integration-tests/junit5-tests it definitely fails for me. Mine is pretty simple;

package org.wildfly.arquillian.integration.test.junit5;

import jakarta.inject.Inject;
import org.jboss.arquillian.container.test.api.Deployment;
import org.jboss.arquillian.junit5.ArquillianExtension;
import org.jboss.shrinkwrap.api.ShrinkWrap;
import org.jboss.shrinkwrap.api.asset.EmptyAsset;
import org.jboss.shrinkwrap.api.spec.JavaArchive;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;

/**
 * @author <a href="mailto:jperkins@redhat.com">James R. Perkins</a>
 */
@ExtendWith(ArquillianExtension.class)
public class GreetIT {
    @Deployment
    public static JavaArchive createDeployment() {
        return ShrinkWrap.create(JavaArchive.class).addClass(Greeter.class)
                .addAsManifestResource(EmptyAsset.INSTANCE, "beans.xml");
    }

    @Inject
    Greeter service;

    @Test
    @DisplayName("testing hello")
    public void should_create_greeting() {
        Assertions.assertNotNull(service);
        Assertions.fail("fail explicitly");
    }
}

Result:

[INFO] --- maven-failsafe-plugin:2.22.2:integration-test (default) @ junit5-tests ---
[INFO] 
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.wildfly.arquillian.integration.test.junit5.GreetIT
[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 5.425 s <<< FAILURE! - in org.wildfly.arquillian.integration.test.junit5.GreetIT
[ERROR] should_create_greeting  Time elapsed: 0.268 s  <<< FAILURE!
org.opentest4j.AssertionFailedError: fail explicitly
	at deployment.test-2193d3f7-2acb-42dd-859e-becc22901aa5.war//org.wildfly.arquillian.integration.test.junit5.GreetIT.should_create_greeting(GreetIT.java:48)

[INFO] 
[INFO] Results:
[INFO] 
[ERROR] Failures: 
[ERROR]   GreetIT.should_create_greeting:48 fail explicitly
[INFO] 
[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0

[abregar]

Ty for taking time to look into this. Briefly I cant see the significant difference in my project or the one you referenced, but the only one - I am using container remote

<groupId>org.wildfly.arquillian</groupId>
<artifactId>wildfly-arquillian-container-remote</artifactId>
<scope>test</scope>
</dependency>

May I ask if you can validate the 'remote' container, too?
Meantime I will also try the managed container.

[jamezp]

Sorry, I had to re-setup my environment and yesterday got away from me. I still get the same failure with the wildfly-arquillian-container-remote dependency.

[abregar]

Ty again taking time second time. Your reply gave precious reference point to compare what could possible be the cause.
Seem that it is the following dependency

<groupId>org.arquillian.ape</groupId>
<artifactId>arquillian-ape-sql-container-dbunit</artifactId>
<version>2.0.0-alpha.7</version>

My project uses this dbunit at some places. Fun fact - even if not used and only dependency is added - tests just start to behave the way described above.

Looking into mentioned project, it seems abandoned, but author Bartosz Majsak says he's willing to help with some PR (arquillian/arquillian-extension-persistence#195 (comment))

So, questions - @jamezp are you willing to provide some hints what would be the mentioned dbunit artifact's issue. At brief, I see it is certainly still using non-jakarta Arquillian core. Actually - https://github.com/arquillian/arquillian-extension-persistence/blob/2.0.0-alpha.7/pom.xml telling pretty old 1.4.0.Final.

Or, is of your awareness - is there any alternative to dbunit project already, which is still maintained and aligned with jakarta compatible Arquillian?

Thank you in advance, a lot!

[jamezp]

I'm actually not familiar with dbunit. However, I know with JPA at least you can fill a database with SQL statements. Not nearly as user friendly though. I suppose one option could be to use a Docker/podman image with a pre-defined database.

[abregar]

May I ask one favor, as you are the expert on topic here. Can you please give this culprited 'org.arquillian.ape' dependency one debug run and feedback approximate opinion what would be the cause for the observed behavior?
From what I have scratched the surface of all projects (arquillian, wildfly-arquillian, ape, dbunit), I see potential issue(s) in context of using junit4 versus junit5 and new jakarta's introduced package naming, but .. problem space is to broad and it would be really nice to have some narrowed down contexts.

Thank you in advance, a lot!

[jamezp]

I don't really know anything about org.arquillian.ape. If you've got a reproducer I could look, but I'm not sure how to set any of that up.

[uenala]

Hi, I managed to get my setup (Wildfly29-preview, Junit5, Java17/JakartaEE10) to work with wildfly-arquillian-container-remote by using recent versions of arquillian-artifacts with the following dependency-configuration:
...

<dependencyManagement>
<dependencies>
<dependency>
         <groupId>org.jboss.arquillian</groupId>
          <artifactId>arquillian-bom</artifactId>
          <version>1.7.1.Final</version>
          <type>pom</type>
          <scope>import</scope>
</dependency>

<dependency>
          <groupId>org.wildfly.arquillian</groupId>
          <artifactId>wildfly-arquillian-container-remote</artifactId>
          <version>5.0.1.Final</version>
          <scope>test</scope>
</dependency>

<dependency>
          <groupId>org.jboss.arquillian.extension</groupId>
          <artifactId>arquillian-persistence-dbunit</artifactId>
          <version>1.0.0.Alpha7</version>
          <scope>test</scope>
              <exclusions>
                  <exclusion>
                      <groupId>log4j</groupId>
                      <artifactId>log4j</artifactId>
                  </exclusion>
                  <exclusion>
                      <groupId>org.jboss.arquillian.extension</groupId>
                      <artifactId>arquillian-transaction-spi</artifactId>
                  </exclusion>
              </exclusions>
</dependency>

<dependency>
          <groupId>org.dbunit</groupId>
           <artifactId>dbunit</artifactId>
          <version>2.6.0</version>
          <scope>test</scope>
</dependency>

<dependency>
          <groupId>org.jboss.arquillian.extension</groupId>
          <artifactId>arquillian-transaction-bom</artifactId>
          <version>2.0.0</version>
</dependency>
</dependencies>
</dependencyManagement>

<dependencies>

      <dependency>
            <groupId>org.wildfly.arquillian</groupId>
            <artifactId>wildfly-arquillian-container-remote</artifactId>
        </dependency>

        <dependency>
            <groupId>org.jboss.arquillian.protocol</groupId>
            <artifactId>arquillian-protocol-servlet-jakarta</artifactId>
        </dependency>

      <dependency>
            <groupId>org.jboss.arquillian.extension</groupId>
            <artifactId>arquillian-persistence-dbunit</artifactId>
        </dependency>

        <dependency>
            <groupId>org.jboss.arquillian.extension</groupId>
            <artifactId>arquillian-transaction-api</artifactId>
            <version>2.0.0</version>
        </dependency>
        <dependency>
            <groupId>org.jboss.arquillian.extension</groupId>
            <artifactId>arquillian-transaction-impl-base</artifactId>
            <version>2.0.0</version>
        </dependency>
        <dependency>
            <groupId>org.jboss.arquillian.extension</groupId>
            <artifactId>arquillian-transaction-spi</artifactId>
            <version>2.0.0</version>
        </dependency>
        <dependency>
            <groupId>org.jboss.arquillian.extension</groupId>
            <artifactId>arquillian-transaction-jta</artifactId>
            <version>2.0.0</version>
        </dependency>

    <dependency>
            <groupId>org.jboss.arquillian.junit5</groupId>
            <artifactId>arquillian-junit5-container</artifactId>
            <version>1.7.1.Final</version>
            <scope>test</scope>
     </dependency>

</dependencies>

...

Also make sure that you migrate properly from Junit4 to JUnit5. Hope this helps.