TestGrid 0.9.0-m25

We are pleased to announce the release of TestGrid Milestone 25

TestGrid provides enterprise customers with the confidence that the products and fixes they get from WSO2 are tested against all product scenarios against a wide-array of infrastructures and deployments, hence is production-ready.

Here’s what really happens in TestGrid:

• Provision an infrastructure on AWS and other infrastructure providers via Infrastructure as Code (IAC) scripts. We use the IAC scripts provided by WSO2 for this purpose.

• Do WSO2 product deployments on top the provisioned infrastructure. We do that by running Configuration -Management scripts (Puppet/Ansible) provided for WSO2 products.

• Run JMeter/Selenium/TestNG scenarios against the deployment to certify the created deployment pattern against the given infrastructure combination.

• In the end, Testgrid generates a report that shows the outcome of the scenario tests. All is good if it is green; start worrying if it is red! :)

With this release, we provide the following,

• Add report downloading feature into TestGrid dashboard.
• Modified TESTGRID dashboard URL patterns as following way.
  Homepage / Job listing page
  https://<host>:<port>
  Results of deployment patterns for a specific product displaying Page
  https://<host>:<port>/ <product-name>
  All result for a specific infra combination displaying page
  https://<host>:<port>/<product-name>/<deployment-pattern-name>/<testplan-id>/infra
  Test result displaying and live log displaying page
  https://<host>:<port>/<product-name>/<deployment-pattern-name>/test-plans/<testplan-id>
• Fixed TestGrid dashboard direct accessing issue.
• Externalize JMeter engine from the TestGrid
• Improved wso2apim-2.2.0-LTS error rate to 0%.
• Bug fixes and improvements

List of resolved features and issues can be tracked at :
https://github.com/wso2/testgrid/milestone/18

TestGrid has been tested against following infrastructures:

• Cloud Provider: AWS and K8S-on-Openstack
• Spawn instances with AMI images in any AWS region
• Any database type AWS RDS supports (MySQL, MariaDB, PostgreSQL, SQL Server and Oracle)

Known Issues: https://github.com/wso2/testgrid/issues

TestGrid 0.9.0-m22

We are pleased to announce the release of TestGrid Milestone 22

TestGrid provides enterprise customers with the confidence that the products and fixes they get from WSO2 are tested against all product scenarios against a wide-array of infrastructures and deployments, hence is production-ready.

Here’s what really happens in TestGrid:

• Provision an infrastructure on AWS and other infrastructure providers via Infrastructure as Code (IAC) scripts. We use the IAC scripts provided by WSO2 for this purpose.

• Do WSO2 product deployments on top the provisioned infrastructure. We do that by running Configuration -Management scripts (Puppet/Ansible) provided for WSO2 products.

• Run JMeter/Selenium/TestNG scenarios against the deployment to certify the created deployment pattern against the given infrastructure combination.

• In the end, Testgrid generates a report that shows the outcome of the scenario tests. All is good if it is green; start worrying if it is red! :)

With this release, we provide the following,

• Improved UI in TestGrid dashboard
• Improved live logs view in TestGrid dashboard
• Improved load time and size of the final report
• Bug fixes

List of resolved features and issues can be tracked at :
https://github.com/wso2/testgrid/milestone/23

TestGrid has been tested against following infrastructures:

• Cloud Provider: AWS and K8S-on-Openstack
• Spawn instances with AMI images in any AWS region
• Any database type AWS RDS supports (MySQL, MariaDB, PostgreSQL, SQL Server and Oracle)

Known Issues: https://github.com/wso2/testgrid/issues

TestGrid 0.9.0-m21

We are pleased to announce the release of TestGrid Milestone 21

TestGrid provides enterprise customers with the confidence that the products and fixes they get from WSO2 are tested against all product scenarios against a wide-array of infrastructures and deployments, hence is production-ready.

Here’s what really happens in TestGrid:

• Provision an infrastructure on AWS and other infrastructure providers via Infrastructure as Code (IAC) scripts. We use the IAC scripts provided by WSO2 for this purpose.

• Do WSO2 product deployments on top the provisioned infrastructure. We do that by running Configuration -Management scripts (Puppet/Ansible) provided for WSO2 products.

• Run JMeter/Selenium/TestNG scenarios against the deployment to certify the created deployment pattern against the given infrastructure combination.

• In the end, Testgrid generates a report that shows the outcome of the scenario tests. All is good if it is green; start worrying if it is red! :)

With this release, we provide the following,

• Integrated IS 5.5.0, IS 5.3.0, APIM 2.2.0 and EI 6.1.1 into TestGrid
• Run product tests on MariaDB 10.0.32
• Support for PostgreSQL 10.3
• Support for Oracle-SE 11.2
• Bug fixes and improvements

List of resolved features and issues can be tracked at :
https://github.com/wso2/testgrid/milestone/22

TestGrid has been tested against following infrastructures:

• Cloud Provider: AWS and K8S-on-Openstack
• Spawn instances with AMI images in any AWS region
• Any database type AWS RDS supports (MySQL, MariaDB, PostgreSQL, SQL Server and Oracle)

Known Issues: https://github.com/wso2/testgrid/issues

TestGrid 0.9.0-m20

We are pleased to announce the release of TestGrid Milestone 20

TestGrid provides enterprise customers with the confidence that the products and fixes they get from WSO2 are tested against all product scenarios against a wide-array of infrastructures and deployments, hence is production-ready.

Here’s what really happens in TestGrid:

• Provision an infrastructure on AWS and other infrastructure providers via Infrastructure as Code (IAC) scripts. We use the IAC scripts provided by WSO2 for this purpose.

• Do WSO2 product deployments on top the provisioned infrastructure. We do that by running Configuration -Management scripts (Puppet/Ansible) provided for WSO2 products.

• Run JMeter/Selenium/TestNG scenarios against the deployment to certify the created deployment pattern against the given infrastructure combination.

• In the end, Testgrid generates a report that shows the outcome of the scenario tests. All is good if it is green; start worrying if it is red! :)

With this release, we provide the following,

• Developed EI cloudformation script to start a single node setup on AWS.
• Integrate EI cloudformation deployment scripts into TestGrid.
• Enhanced the dashboard UI by adding meaningful test case scenario names.
• Improved the report view.
• Bug fixes and improvements

List of resolved features and issues can be tracked at :
https://github.com/wso2/testgrid/milestone/21

TestGrid has been tested against following infrastructures:

• Cloud Provider: AWS and K8S-on-Openstack
• Spawn instances with AMI images in any AWS region
• Any database type AWS RDS supports (MySQL, MariaDB, PostgreSQL, SQL Server and Oracle)

Known Issues: https://github.com/wso2/testgrid/issues

TestGrid 0.9.0-m19

We are pleased to announce the release of TestGrid Milestone 19

TestGrid provides enterprise customers with the confidence that the products and fixes they get from WSO2 are tested against all product scenarios against a wide-array of infrastructures and deployments, hence is production-ready.

Here’s what really happens in TestGrid:

• Provision an infrastructure on AWS and other infrastructure providers via Infrastructure as Code (IAC) scripts. We use the IAC scripts provided by WSO2 for this purpose.

• Do WSO2 product deployments on top the provisioned infrastructure. We do that by running Configuration -Management scripts (Puppet/Ansible) provided for WSO2 products.

• Run Jmeter/Selenium/TestNG scenarios against the deployment to certify the created deployment pattern against the given infrastructure combination.

• In the end, Testgrid generates report that shows the outcome of the scenario tests. All is good if it is green; start worrying if it is red! :)

With this release, we provide the following,

• Integrated cloudformation APIM deployment scripts into TestGrid
• Introduce new AMIs which supports JDKS including Oracle JDK 8,9,10 and OpenJDK 8,9,10
• Add capability of product testing on Oracle JDK 8,9,10 and OpenJDK 8,9,10
• Run product tests on CentOS 7.4
• Support OS version as a infrastructure parameter
• Bug fixes and improvements

List of resolved features and issues can be tracked at :
https://github.com/wso2/testgrid/milestone/20

TestGrid has been tested against following infrastructures:

• Cloud Provider: AWS and K8S-on-Openstack
• Spawn instances with AMI images in any AWS region
• Any database type AWS RDS supports (MySQL, MariaDB, PostgreSQL, SQL Server and Oracle)

Known Issues: https://github.com/wso2/testgrid/issues

TestGrid 0.9.0-m18

We are pleased to announce the release of TestGrid Milestone 18

TestGrid provides enterprise customers with the confidence that the products and fixes they get from WSO2 are tested against all product scenarios against a wide-array of infrastructures and deployments, hence is production-ready.

Here’s what really happens in TestGrid:

• Provision an infrastructure on AWS and other infrastructure providers via Infrastructure as Code (IAC) scripts. We use the IAC scripts provided by WSO2 for this purpose.

• Do WSO2 product deployments on top the provisioned infrastructure. We do that by running Configuration -Management scripts (Puppet/Ansible) provided for WSO2 products.

• Run Jmeter/Selenium/TestNG scenarios against the deployment to certify the created deployment pattern against the given infrastructure combination.

• In the end, Testgrid generates report that shows the outcome of the scenario tests. All is good if it is green; start worrying if it is red! :)

With this release, we provide the following,

• Add product testing on MariaDB, PostgreSQL, SQL Server and Oracle database types with AWS infrastructure
• Introduced new testgrid statuses with icons
• Introduced a new command to finalize the statuses of testplans with errors
• Increased availability of AWS resource space by avoiding snapshots creation on infrastructure deletion
• Simplified test case failure message in final report and TestGrid Dashboard
• Re-structure TestGrid product job directory.
• Bug fixes

List of resolved features and issues can be tracked at :
https://github.com/wso2/testgrid/milestone/19

TestGrid has been tested against following infrastructures:

• Cloud Provider: AWS and K8S-on-Openstack
• Spawn instances with AMI images in any AWS region
• Any database type AWS RDS supports (MySQL, MariaDB, PostgreSQL, SQL Server and Oracle)

Known Issues: https://github.com/wso2/testgrid/issues

TestGrid 0.9.0-m17

We are pleased to announce the release of TestGrid Milestone 17

TestGrid provides enterprise customers with the confidence that the products and fixes they get from WSO2 are tested against all product scenarios against a wide-array of infrastructures and deployments, hence is production-ready.

Here’s what really happens in TestGrid:

• Provision an infrastructure on AWS and other infrastructure providers via Infrastructure as Code (IAC) scripts. We use the IAC scripts provided by WSO2 for this purpose.

• Do WSO2 product deployments on top the provisioned infrastructure. We do that by running Configuration -Management scripts (Puppet/Ansible) provided for WSO2 products.

• Run Jmeter/Selenium/TestNG scenarios against the deployment to certify the created deployment pattern against the given infrastructure combination.

• In the end, Testgrid generates report that shows the outcome of the scenario tests. All is good if it is green; start worrying if it is red! :)

With this release, we provide the following,

• Add support to Open JDK 8/9 and Oracle JDK 9 with AWS infrastructure
• Integrate AWS cloud-formation stack creation logs to TestGrid logs
• Add option to send workspace as a property into scenario test-scripts
• Add option to request wum-credentials by cloudformation scripts
• Introduce Pending icon to TestGrid dashboard
• Bug fixes

List of resolved features and issues can be tracked at :
https://github.com/wso2/testgrid/milestone/17

TestGrid has been tested against following infrastructures:

• Cloud Provider: AWS and K8S-on-Openstack
• Spawn instances with AMI images in any AWS region
• Any database type AWS RDS supports (MySQL, MariaDB, PostgreSQL, SQL Server and Oracle)

Known Issues: https://github.com/wso2/testgrid/issues

TestGrid 0.9.0-M16

We are pleased to announce the release of TestGrid Milestone 16

TestGrid provides enterprise customers with the confidence that the products and fixes they get from WSO2 are tested against all product scenarios against a wide-array of infrastructures and deployments, hence is production-ready.

Here’s what really happens in TestGrid:

• Provision an infrastructure on AWS and other infrastructure providers via Infrastructure as Code (IAC) scripts. We use the IAC scripts provided by WSO2 for this purpose.

• Do WSO2 product deployments on top the provisioned infrastructure. We do that by running Configuration -Management scripts (Puppet/Ansible) provided for WSO2 products.

• Run Jmeter/Selenium/TestNG scenarios against the deployment to certify the created deployment pattern against the given infrastructure combination.

• In the end, Testgrid generates report that shows the outcome of the scenario tests. All is good if it is green; start worrying if it is red! :)

With this release, we provide the following,

• An AMI-mapper for AWS extension point to find suitable AMI for a given set of infrastructure combination
• Gracefully handle failures of infrastructure creation and deployment
• UI stability improvements

List of resolved features and issues can be tracked at :
https://github.com/wso2/testgrid/milestone/15

TestGrid has been tested against following infrastructures:

• Cloud Provider: AWS and K8S-on-Openstack
• Spawn instances with AMI images in any AWS region
• Any database type AWS RDS supports (MySQL, MariaDB, PostgreSQL, SQL Server and Oracle)

Known Issues: https://github.com/wso2/testgrid/issues

TestGrid 0.9.0-M15

We are pleased to announce the release of TestGrid Milestone 15

TestGrid provides enterprise customers with the confidence that the products and fixes they get from WSO2 are tested against all product scenarios against a wide-array of infrastructures and deployments, hence is production-ready.

Here’s what really happens in TestGrid:

• Provision an infrastructure on AWS and other infrastructure providers via Infrastructure as Code (IAC) scripts. We use the IAC scripts provided by WSO2 for this purpose.

• Do WSO2 product deployments on top the provisioned infrastructure. We do that by running Configuration -Management scripts (Puppet/Ansible) provided for WSO2 products.

• Run Jmeter/Selenium/TestNG scenarios against the deployment to certify the created deployment pattern against the given infrastructure combination.

• In the end, Testgrid generates report that shows the outcome of the scenario tests. All is good if it is green; start worrying if it is red! :)

With this release, we provide the following,

• Introduce job-config.yaml to feed data into Testgrid commands
• Facilitate higher availability of Jenkins Prod environment by adding a new EBS volume with higher capacity
• A comprehensive documentation on setting up a Jenkins slave to run K8S
• Improved SSO session expiry handling and avoid caching expired session data in Dev and Prod dashboards
• Run scenario tests for product IS on Kubernetes deployment on OpenStack infrastructure

TestGrid has been tested against following infrastructures:

• Cloud Provider: AWS and K8S-on-Openstack
• Spawn instances with AMI images in any AWS region
• Any database type AWS RDS supports (MySQL, MariaDB, PostgreSQL, SQL Server and Oracle)

Known Issues: https://github.com/wso2/testgrid/issues

TestGrid 0.9.0-M14

We are pleased to announce the release of TestGrid Milestone 14

TestGrid provides enterprise customers with the confidence that the products and fixes they get from WSO2 are tested against all product scenarios against a wide-array of infrastructures and deployments, hence is production-ready.

Here’s what really happens in TestGrid:

• Provision an infrastructure on AWS and other infrastructure providers via Infrastructure as Code (IAC) scripts. We use the IAC scripts provided by WSO2 for this purpose.

• Do WSO2 product deployments on top the provisioned infrastructure. We do that by running Configuration -Management scripts (Puppet/Ansible) provided for WSO2 products.

• Run Jmeter/Selenium/TestNG scenarios against the deployment to certify the created deployment pattern against the given infrastructure combination.

• In the end, Testgrid generates report that shows the outcome of the scenario tests. All is good if it is green; start worrying if it is red! :)

With this release, we provide the following,

• Introduced testgrid.yaml config file to make infra/deployment/scenario repos testgrid-ready.
• Added support to execute kubernetes-is and kubernetes-apim.
• Added ability to run multiple AWS stacks simultaneously from the same cloudformation script.
• Introduced a central configuration file for testgrid instead of depending on env variables.
• Improved the extension architecture design.
• Added support for OpenStack as the infrastructure

TestGrid has been tested against following infrastructures:

• Cloud Provider: AWS and K8S-on-Openstack
• Spawn instances with AMI images in any AWS region
• Any database type AWS RDS supports (MySQL, MariaDB, PostgreSQL, SQL Server and Oracle)

Known Issues: https://github.com/wso2/testgrid/issues