The Quality Insights Toolkit (QIT) is an initiative by WooCommerce designed to provide current and prospective extension developers in the Woo Marketplace with managed automated tests.
Summary
The goal of the Quality Insights Toolkit (QIT) is to provide extension developers with managed automated tests and easy ways to integrate these tests into development workflows. We’ve designed and developed three core tools within this toolkit to achieve this goal:
- Viewer: A UI-based test runner and test results viewer, available in your WooCommerce dashboard.
- CLI: A terminal tool which allows for running tests against published and development versions of an extension.
- GitHub Workflows: GitHub workflow files that allow running tests regularly with QIT as part of a GitHub development workflow.
Timeframe
We have launched the initial version of this tool to developers currently in the Woo Marketplace where they will see this new option in the woo.com vendor dashboard. Additionally, we will begin incorporating QIT into the product submission process.
We are continuing to work on making the tool accessible to all developers in the community.
Background
We’ve identified compatibility testing as one of the critical areas to focus on with quality tooling. Primarily, we wanted to provide an easy way to be able to test against upcoming versions of WooCommerce, PHP and WordPress. The QIT was also designed to test compatibility between extensions by allowing users to activate them alongside each other. For example, if you develop multiple extensions in the Woo Marketplace, the tool should be able to activate them all together and report back on any PHP errors or warnings that might be triggered.
With these points in mind, we’ve identified the following benefits that QIT can provide:
Benefits as an extension developer
- You can enjoy out-of-the-box tests for all the existing and upcoming extensions you sell on the Marketplace.
- Your extensions will undergo continuous testing, including against new releases of PHP, WooCommerce, and WordPress.
- You can have increased peace of mind knowing that you’ll be alerted if the tests catch anything with a future release or an edge case.
- You can use our GitHub Workflows integration guide to plug our tool into your CI/CD pipeline.
- You can easily integrate the testing tool with your development workflow using the terminal tool.
- You can run some tests with other extensions active at the same time, including those that may be known to cause issues when activated alongside yours.
Benefits for users in the WooCommerce ecosystem
- Users can enjoy increased reliability when updating WooCommerce extensions on the Marketplace.
- Users can have confidence to update PHP, WordPress, or WooCommerce versions without fear of breaking their extensions
- There will be less potential security issues for users of the WooCommerce Ecosystem.
- Users can have more assurance that all extensions will continue to work as expected when activated alongside each other, ensuring a seamless experience.
Resources
We’re still actively developing the QIT in anticipation of our launch and are finalizing our public documentation. While we finish out the documentation, we wanted to provide a list of the types of tests that will be included at launch:
- End-to-end tests: Uses the WooCommerce Core end-to-end tests to verify the core Critical Flows for WooCommerce continue to function as expected.
- API tests: Tests the WooCommerce REST API using the WooCommerce API Core Tests.
- Activation tests: Activates the extension against a freshly created shop and captures any PHP errors or warnings that may occur on activation.
- Security tests: Runs an experimental security scanner against a given extension. It uses a set of PHPCS WordPress Security rules.
- PHPStan tests: Runs level 0 PHPStan checks against the extension.
Beta opportunities
If you have an extension in the Woo Marketplace and would like to join our beta program, please let us know!