Skip to content

Latest commit

 

History

History
115 lines (76 loc) · 7 KB

PluginTESTINGGuidelines.md

File metadata and controls

115 lines (76 loc) · 7 KB
Raw

Zowe CLI Plug-in Testing Guidelines

This document is intended to be a living summary of conventions and best practices for running tests against Zowe CLI plug-ins.

Contents

Automated Tests

Plug-ins have two groups of tests that can be run separately:

  • Unit tests
  • System and integration tests.

All automated test suite file names should end in .test.ts.

To run all automated tests, use the command npm run test.

Unit tests

Unit tests test the logic of your code without interacting with any back end servers or modifying the file system. Dependencies, web APIs, file system functions, and so on should all be mocked with the Jest mocking functionality.

Unit tests are run with the npm script test:unit which is run via the command npm run test:unit.

Unit tests are stored under the directory __tests__ in the project root. You should duplicate the directory structure of file you are unit testing within this folder. For instance, if you are testing the file src/api/MyAPI.ts, you should create the file __tests__/api/MyAPI.test.ts.

System and Integration Tests

System and integration tests are stored under the __tests__/__system__ directory. These tests execute commands from your plugin and communicate with the relevant back end server, if any.

Note: You must run npm run build before you can successfully run the system/integration tests, because the plugin must be able to be installed from compiled source during the tests.

In addition to Node.js, you must have a means to execute ".sh" (bash) scripts (required for running integration tests). On Windows, you can install "Git Bash" (bundled with the standard Git installation - check "Use Git and Unix Tools from Windows Command Prompt" installation option).

After downloading/installing the prerequisites ensure you can perform the following (and receive success responses):

  1. node --version
  2. npm --version
  3. On Windows: where sh

To run the plug-in system tests, you require a configured properties file with proper system information present.

A dummy properties file is present in the __tests__/__resources__/properties folder, example_properties.yaml. Using this file as a template, create a custom_properties.yaml file within the same directory.

You can modify example_properties.yaml to include whatever properties are necessary for running your system tests, such as connection information to back end servers. If you update example_properties.yaml, please provide comments to explain each property or group of properties, and update the ITestPropertiesSchema.ts interface to reflect the new or changed properties. The ITestPropertiesSchema file is an interface used from your test to allow compile-time checking of references to test properties, so it should always be up to date with example_properties.yaml.

Important! NEVER CHECK-IN A CONFIGURED PROPERTIES FILE, AS SECURITY PRINCIPALS AS WELL AS OTHER CRITICAL INFORMATION IS PRESENT!

You can run the system tests by issuing: npm run test:system.

If the custom_properties.yaml file cannot be found or loaded, an error with relevant details will be thrown.

Types of System and Integration Tests

There are two main types of system tests that you will want to write for your plug-in. Remember to end all of your test files in both categories with either .system.test.ts or .integration.test.ts.

API system tests, stored under the __tests__/__system__/api directory, exercise the Typescript APIs you write for your plugin, by connecting to a real instance of any back end APIs.

CLI tests use .sh shell script files in order to issue commands exposed by your plug-in. The philosophy behind these tests is that the environment should be as close as possible to how the end user will issue your commands. So even though it may be possible to install your plug-in as a stand-alone CLI for testing purposes, we recommend that you test your plug-in commands installed into Zowe CLI.

You run these scripts using the runCliScript utility available in TestUtils.ts.

So an example script file would be:

#!/bin/bash
set -e # fail the script if we get a non zero exit code
directory=$1 
zowe bp-sample list directory-contents "$directory"

Automated Test Utilities

A number of test utilities are available to you to aid in writing automated tests for your plug-in.

TestEnvironment.ts (in the @zowe/cli-test-utils package) contains functionality for creating an isolated environment from which to test your CLI. A new directory under __tests__/__results__/data is created, and logs, installed plugins, and profiles are stored under this directory. Your CLI home is automatically set to this new directory when using the runCliScript function from TestUtils.ts, so any plugins or profiles that you use with your globally installed copy of Zowe CLI will not interfere with running the tests.

In your system test files, set up and clean up your test environment with a call to TestEnvironment.setUp and TestEnvironment.cleanUp as shown in the following example. After you call TestEnvironment.setUp, you can access the values of the system properties, and also use the working directory

Example use of TestEnvironment.ts:

let testEnvironment: ITestEnvironment<ITestPropertiesSchema>; 
let myTestFile: string;
describe ("my tests", =>{ 

    // Create the unique test environment
    beforeAll(async () => {
        testEnvironment = await TestEnvironment.setUp({
            installPlugin: true, // install our plugin into the test environment working directory so that we can issue plugin commands
            testName: "fail_command", // this influences the name of the generated directory 
            tempTestProfiles: ["zosmf"] // create a zosmf profile from the settings in the custom_properties.yaml file
        });

        // access any of the test properties
        const user = testEnvironment.systemTestProperties.zosmf.user;

        // access the working directory from your test environment to save temporary test-related files 
        myTestFile = testEnvironment.workingDir +"/myTestFile.txt";
    });
    afterAll(async () => {
        await TestEnvironment.cleanUp(TEST_ENVIRONMENT);
         // cleans up installed plugins and created profiles but doesn't delete the directory
         // in case you need to reference any of the files after the tests run 
    });
    //... your tests here ... 
});

For more information, JSDoc documentation is available on the functions and interface in TestEnvironment.ts and ISetupEnvironmentParms.ts.

If your plugin introduces any new types of configuration profiles, see and update TempTestProfiles.ts (along with example_properties.yaml and ITestPropertiesSchema.ts) so that your tests can automatically create and delete profiles based on your properties. // TODO comments have been placed in TempTestProfiles in locations you may need to update.