Writing Integration Tests for APIM

Some useful best practices to follow when writing integration tests for APIM.

• Avoid creating and using configuration files for each test case rather merge similar configuration files and use it for all test cases.
  • For example: If 3 test cases needs to change values in api-manager.xml/deployment.yaml, merge all 3 changes in one file rather than using 3 different api-manager.xml/deployment.yaml
• Group test cases which needs configuration deployment and server restarts.
• Use @BeforeTest annotation in each group to deploy merged configuration files and do a single server restart before executing the tests of that group. Use @AfterTest to restore all the deployed configurations after executing all the tests in that group. For example: The following example class will be in one group in the testng.xml

 public class SampleDeployer {
        @BeforeTest
         public void deployAndRestart() {
          //Deploy merged configs and restart the server
         }
         @AfterTest
          Public void restore() {
              //Restore the configurations
          }
}

• Use restAPIPublisher and restAPIStore variables defined and initialized in the base class, if we are implementing the tests with the usermode defined in the test class which we are implementing. That is, when we are not creating new users and new roles. In the case of having new users and roles, we need to initialize the above two variables again in our test class for the newly created users.
• Include super.cleanUp() method in the @AfterClass method in each and every test class. This will make sure that all the APIs, Subscriptions and Applications are deleted if they are missed.
• Use the common methods defined such as createPublishAndSubscribeToAPI(), createAndPublishAPI() when creating, publishing and subscribing APIs in test classes. This is to avoid code duplication.
• Avoid adding common test methods (testAddAPI(), testLifecycleChange()) in the test classes of other use cases such as APIImportExportTestCase rather use the methods mentioned in the previous point in the setEnvironment() method (@BeforeClass)
• In case it is impossible to use @BeforeTest and @AfterTest, use the @BeforeClass annotation to initialize the environment and other instances. Use @AfterClass to clean up all resources used inside the test and make sure to revert the system to its initial state.
• Use the newly introduced test user modes in the new tests as follows. Assume that the test case is CORSHeadersTestCase.

    @DataProvider
    public static Object[][] userModeDataProvider() {
        return new Object[][] { new Object[] { TestUserMode.SUPER_TENANT_ADMIN },
                new Object[] { TestUserMode.TENANT_ADMIN },
                new Object[] { TestUserMode.SUPER_TENANT_USER_STORE_USER },
                new Object[] { TestUserMode.SUPER_TENANT_EMAIL_USER },
                new Object[] { TestUserMode.TENANT_EMAIL_USER },
        };
    }

    @Factory(dataProvider = "userModeDataProvider")
    public CORSHeadersTestCase(TestUserMode userMode) {
        this.userMode = userMode;
    }

• Use assert statements instead of System.out.print() while running tests.
• Include Javadocs at the top of the class, and make sure to include a description under the @Test annotation.
• All test method names should start with the prefix test, e.g., testAddResorce, testVerifyAvailabilty, etc.
• All test class names should end with the suffix TestCase, e.g., ThrottlingTestCase
• If you are writing a test case for a public JIRA, prefix the class name with the JIRA issue ID, e.g. APIM1234TestCase
• Group test methods by product, e.g., wso2.am, wso2.esb, etc.
• Use testNg assert appropriately. Writing test cases without using assert will result in false positive test results.
• Don't use dependsOn and priority annotations together in the same test class, as this will lead to testNG issues.
• Avoid using the same test method name in different classes with the dependsOnMethod annotation, as this makes it hard to follow your code.
• Priority annotation works fine from 1 - 10. If you are going to go beyond 10, use the testNG dependency model.

References https://docs.wso2.com/display/TA100/Best+Practices+for+Writing+Integration+Tests http://testng.org/doc/documentation-main.html