Releases: wso2/testgrid
TestGrid 0.9.0-M13
We are pleased to announce the release of TestGrid Milestone 13
TestGrid provides enterprise customers with the confidence that the products and fixes they get from WSO2 are tested against all product scenarios against a wide-array of infrastructures and deployments, hence is production-ready.
Here’s what really happens in TestGrid:
-
Provision an infrastructure on AWS and other infrastructure providers via Infrastructure as Code (IAC) scripts. We use the IAC scripts provided by WSO2 for this purpose.
-
Do WSO2 product deployments on top the provisioned infrastructure. We do that by running Configuration -Management scripts (Puppet/Ansible) provided for WSO2 products.
-
Run Jmeter/Selenium/TestNG scenarios against the deployment to certify the created deployment pattern against the given infrastructure combination.
-
In the end, Testgrid generates report that shows the outcome of the scenario tests. All is good if it is green; start worrying if it is red! :)
With this release, we provide the following,
- Added support for IS 5.4.1.
- Authentication and authorization mechanism for TestGrid APIs.
- Added alert mechanism for CI/CD pipeline stages.
- Support to add CentOS 7 from cloudformation scripts.
- Fixed the Report displaying duplicate test scenarios.
- Introduced build API into PROD.
TestGrid has been tested against following infrastructures:
- Cloud Provider: AWS
- Spawn instances with AMI images in any AWS region
- Any database type AWS RDS supports (MySQL, MariaDB, PostgreSQL, SQL Server and Oracle)
Known Issues: https://github.com/wso2/testgrid/issues
TestGrid 0.9.0-M12
We are pleased to announce the release of TestGrid Milestone 12
TestGrid provides enterprise customers with the confidence that the products and fixes they get from WSO2 are tested against all product scenarios against a wide-array of infrastructures and deployments, hence is production-ready.
Here’s what really happens in TestGrid:
-
Provision an infrastructure on AWS and other infrastructure providers via Infrastructure as Code (IAC) scripts. We use the IAC scripts provided by WSO2 for this purpose.
-
Do WSO2 product deployments on top the provisioned infrastructure. We do that by running Configuration -Management scripts (Puppet/Ansible) provided for WSO2 products.
-
Run Jmeter/Selenium/TestNG scenarios against the deployment to certify the created deployment pattern against the given infrastructure combination.
-
In the end, Testgrid generates report that shows the outcome of the scenario tests. All is good if it is green; start worrying if it is red! :)
With this release, we provide the following,
- Single Sign-On for Testgrid dashboard via apps.wso2.com
- New design for the configuration file for infra/deploy/scenario repos that describe what that repo does.
This is not really testgrid specific. It will simply describe the repo. This is similar to travis.yml of Travis CI and Jenkinsfile of Jenkins2.
- View results of JMeter tests in the downloadable log file
- Run multiple Testgrid jobs in parallel
- Bug fixes
TestGrid has been tested against following infrastructures:
- Cloud Provider: AWS
- Spawn instances with AMI images in any AWS region
- Any database type AWS RDS supports (MySQL, MariaDB, PostgreSQL, SQL Server and Oracle)
Known Issues: https://github.com/wso2/testgrid/issues
TestGrid 0.9.0-M11
We are pleased to announce the release of TestGrid Milestone 11
TestGrid provides enterprise customers with the confidence that the products and fixes they get from WSO2 are tested against all product scenarios against a wide-array of infrastructures and deployments, hence is production-ready.
TestGrid is capable of the following,
- Create and destroy infrastructure and deployment on AWS via CloudFormation scripts
- Run JMeter/Selenium/TestNG scenarios against the deployment
- And, finally, generate a report based on outcomes of the scenario testing
- You can also view archived reports
With this release, we provide,
- Generation of infrastructure combinations via value-sets in a database.
- A job history view in the dashboard
- Download and view logs in the dashboard
- Selenium execution support for scenario scripts
- Testgrid dashboard is now accessible through AppStore with SSO. (Experimental Feature)
And Bug fixes
TestGrid has been tested against following infrastructures:
- Cloud Provider: AWS
- Spawn instances with AMI images in any AWS region
- Any database type AWS RDS supports (MySQL, MariaDB, PostgreSQL, SQL Server and Oracle)
Known Issues: https://github.com/wso2/testgrid/issues
TestGrid 0.9.0-M10
We are pleased to announce the release of TestGrid Milestone 10
TestGrid provides enterprise customers with the confidence that the products and fixes they get from WSO2 are tested against all product scenarios against a wide-array of infrastructures and deployments, hence is production-ready.
TestGrid is capable of the following,
- Create and destroy infrastructure and deployment on AWS via CloudFormation scripts
- Run JMeter/Selenium/TestNG scenarios against the deployment
- And, finally, generate a report based on outcomes of the scenario testing
- You can also view archived reports
With this release, we provide,
- A summary page inclusive of the test run log and test summary.
- Live log display when the test plan is in pending state (similar to Jenkins).
- Archiving of test run logs and reports to AWS S3.
- Log4j XML is now packed with TestGrid distribution and therefore needs not to be given as a parameter.
- External dependencies are decoupled from the TestGrid distribution.
- TestGrid execution script to support test command executions and debug.
And Bug fixes
TestGrid has been tested against following infrastructures:
- Cloud Provider: AWS
- Spawn instances with AMI images in any AWS region
- Any database type AWS RDS supports (MySQL, MariaDB, PostgreSQL, SQL Server and Oracle)
Known Issues: https://github.com/wso2/testgrid/issues
TestGrid 0.90-M9
We are pleased to announce the release of TestGrid Milestone 8
TestGrid provides enterprise customers the confidence that the products and fixes they get from WSO2 are tested against all product scenarios against a wide-array of infrastructures and deployments, hence is production-ready.
TestGrid is capable of the following,
- Create and destroy infrastructure and deployment on AWS via CloudFormation scripts
- Run Jmeter/Selenium/TestNG scenarios against the deployment
- And, finally generate a report based on outcomes of the scenario testing
- You can also view archived reports
With this release, we provide,
- API to create test plans in TestGrid
- A Refined dashboard
- Support execution of Deployment scripts
- HTTPS enabled access to TestGrid deployment
TestGrid has been tested against following infrastructures:
- Cloud Provider: AWS
- Spawn instances with AMI images in any AWS region
- Any database type AWS RDS supports (MySQL, MariaDB, PostgreSQL, SQL Server and Oracle)
Known Issues: https://github.com/wso2-incubator/wso2-test-grid/issues
TestGrid 0.90-M8
We are pleased to announce the release of TestGrid Milestone 8
TestGrid provides enterprise customers the confidence that the products and fixes they get from WSO2 are tested against all product scenarios against a wide-array of infrastructures and deployments, hence is production-ready.
TestGrid is capable of the following,
- Create and destroy infrastructure and deployment on AWS via CloudFormation scripts
- Run Jmeter/Selenium/TestNG scenarios against the deployment
- And, finally generate a report based on outcomes of the scenario testing
- You can also view archived reports
With this release, we provide,
- Hostname for TestGrid dashboard.
- Report generation for each test plan in the test run artifacts location.
- Improvements on shell script execution.
- Documentation of continuous delivery flow of TestGrid.
TestGrid has been tested against following infrastructures: - Cloud Provider: AWS
- Spawn instances with AMI images in any AWS region
- Any database type AWS RDS supports (MySQL, MariaDB, PostgreSQL, SQL Server and Oracle)
Known Issues: https://github.com/wso2-incubator/wso2-test-grid/issues
TestGrid 0.90-M7
We are pleased to announce the release of TestGrid Milestone 7
TestGrid provides enterprise customers the confidence that the products and fixes they get from WSO2 are tested against all product scenarios against a wide-array of infrastructures and deployments, hence is production-ready.
TestGrid is capable of the following,
- Create and destroy infrastructure and deployment on AWS via CloudFormation scripts
- Run Jmeter/Selenium/TestNG scenarios against the deployment
- And, finally generate a report based on outcomes of the scenario testing
- You can also view archived reports
With this release, we provide,
- Adding npm build of TestGrid dashboard to WAR distribution.
- Dummy infrastructure and deployment creation scripts that work against CF.
- Continuous Delivery for TestGrid
- Simplified logging structure to facilitate traceability.
TestGrid has been tested against following infrastructures: - Cloud Provider: AWS
- Spawn instances with AMI images in any AWS region
- Any database type AWS RDS supports (MySQL, MariaDB, PostgreSQL, SQL Server and Oracle)
Known Issues: https://github.com/wso2-incubator/wso2-test-grid/issues
TestGrid 0.90-M6
We are pleased to announce the release of TestGrid Milestone 6
TestGrid provides enterprise customers the confidence that the products and fixes they get from WSO2 are tested against all product scenarios against a wide-array of infrastructures and deployments, hence is production-ready.
TestGrid is capable of the following,
- Create and destroy infrastructure and deployment on AWS via CloudFormation scripts
- Run Jmeter/Selenium/TestNG scenarios against the deployment
- And, finally generate a report based on outcomes of the scenario testing
- You can also view archived reports
With this release, now we can:
- A Dashboard to view the test-run status with history along with detailed logs.
- Test-Run status of each and every product channel
- Drill down view to know precise cause and failure reason in case of a test run failure
TestGrid has been tested against following infrastructures:
- Cloud Provider: AWS
- Spawn instances with AMI images in any AWS region
- Any database type AWS RDS supports (MySQL, MariaDB, PostgreSQL, SQL Server and Oracle)
Known Issues: https://github.com/wso2-incubator/wso2-test-grid/issues
TestGrid 0.9-M5
Whats New
- TestGrid logging framework to capture errors in infra, deployment and test scenarios
- Improved report format. Please do find the generated reports for the latest IS-5.4.0 build attached herewith. The 3 reports attached are generated for the same test scenarios but highlight each different axis (Deployment axis - "wso2is-5.4.0-LTS - deployment perspective.pdf", Infrastructure axis - "wso2is-5.4.0-LTS - Infra perspective.pdf", Scenario axis - "wso2is-5.4.0-LTS - Scenario perspective.pdf"). Please do note that the scenario description and proper scenario name is not yet reflected in these reports. Furthermore, the reports will include only the test failure information. If it is required to have all the test information this is still possible via the "--showSuccess" flag.
Bug Fixes And Improvements
All the issues addressed in this milestone are reported in the following location:
Known issues
All open issues pertaining to TestGrid are reported in the following location:
TestGrid 0.9-M4
Whats New
- Supporting environment setup scripts for test scenarios (pre/post scripts).
- Define infrastructure combinations using a single infrastructure script (instead of multiple scripts as before).
- Securing credentials in Jenkins pipeline script.
Bug Fixes And Improvements
All the issues addressed in this milestone are reported in the following location:
Known issues
All open issues pertaining to TestGrid are reported in the following location: