End-to-end testing WooCommerce

It is important for developers to be able to easily and quickly test new versions of WooCommerce. The Core Testing Checklist has 30 items in it, and is a manual process. Running through the whole thing is tedious and time consuming, so automating the process would be a big improvement.

End-to-end (e2e) testing tests whether the whole flow of an application is working as expected, and will be an important part in ensuring changes don’t break anything. It automates the manual process of going through all of the different customer-facing and admin-facing pages, performing actions on those pages, and verifying those actions do what is expected.

We have developed an e2e testing suite that should dramatically reduce the time it takes to run through tests before big releases. It works by navigating around a site with ChromeDriver. ChromeDriver simulates a real user by moving the mouse around, clicking on things, and entering input into forms. The results of these actions get compared to the expected results, and the test suite can then determine whether the action was successful. Since this is programmatic, testing can be completed much faster than humanly possible.

The initial release of the testing suite covers some of our core testing checklist, but not yet all of it. Developers are welcome and encouraged to contribute new tests

Usage

You can use the test suite included in the WooCommerce GitHub repository for testing the base WooCommerce plugin. Follow the Getting Started guide to set up your environment for e2e testing.

You can also use the tests for verifying that an extension didn’t break core WooCommerce functionality. To do this, activate the extension before running the tests. If the tests pass, it will verify that the activated extension did not break functionality covered by the tests.

Please note: this may not work well with some extensions that drastically change the WooCommerce flow or change certain html elements because the tests expect to be able to click on specific elements and have them act in a predetermined way.

Adding end-to-end tests to extensions.

The WooCommerce test suite is built on top of a library we’ve developed for simplifying WooCommerce e2e testing. The library contains a collection of JavaScript objects that can be used for performing common actions on WooCommerce screens, and the objects can be easily extended to add new, custom user actions. You can write test suites for your extensions and dramatically cut down on the amount of time you spend manually testing!

Here is a simple test example using the library:

[code language="javascript"]

// Create a new Shop page object.
const shopPage = new ShopPage(
driver,
{
url: manager.getPageUrl( '/shop' )
}
);

// Click the "Add to Cart" button for "Flying Ninja".
shopPage.addProductToCart( 'Flying Ninja' );

// Create a new Cart page object.
const cartPage = new CartPage(
driver,
{
url: manager.getPageUrl( '/cart' )
}
);

// Verify the cart has the items.
assert.eventually.ok( cartPage.hasItem( 'Flying Ninja' ) );

[/code]

To help you get started with extension e2e testing, we’ve written a helpful tutorial that introduces the library, setting up your environment, writing tests for customer-facing screens, and writing tests for admin-facing screens. Read the tutorial to get started! If you run into any questions about objects or their methods, there is also API documentation.

Contributing

If you encounter any bugs in the library or want to contribute, you can do that here.


Keep yourself in the loop!

This field is hidden when viewing the form
This field is hidden when viewing the form
This field is hidden when viewing the form


12 responses to “End-to-end testing WooCommerce”

  1. Bryan M Avatar

    This will be helpful. Another note, for anyone using WPEngine, a few months ago I discovered if I used their “Staging” area, to copy my live sent, then make some changes in test environment, it could be handy. The problem? Woo detects that it’s a copy, and prompts you as such. BUT, and this is a big one, the copy site decided to fire off a bunch of old email receipts from Woo, to customers from months ago!!! Not good, they thought we charged them a 2nd time for products they already bought. Not sure if this helps, but something to note as many sites are on WPEngine…

    Cheers
    Bryan

    1. That sounds completely unrelated to this blog post?

  2. Itโ€™s great to see work being done on these. e2e tests are the only automated way to solve the single biggest issue for WooCommerce store owners (and extension developers) – is my store/plugin working with this update?

    Thatโ€™s why weโ€™ve been working on something very similar at Prospress the last few months.

    Can you share more about the longer term plans (or hopes and dreams) for the test suite?

    Knowing plans for the official e2e tests could help us avoid wasting time working to solve the same problem in a different way. Weโ€™d much prefer to align our efforts and contribute to make one suite better than spend that effort creating a duplicate suite.

    1. First priorities are documentation, increasing test coverage, and adding more page objects (tax settings screens, shipping settings screens, etc.).

      Long-term direction is pretty open at the moment, so let us know if there is a specific use-case you’re working on. Ideally, the framework will be used as a base for any sort of WooCommerce-based e2e testing.

      I agree with you. Duplicate projects will be less maintained and less feature-rich than one project used and maintained by multiple organizations, so it makes sense to align effort if possible.

      1. Sounds good. We’ve been focusing on a different use case to the current core e2e tests – testing real stores. We’re coming to the problem from the store owner’s perspective and writing end-to-end tests to run against their store (either a live version or a copy in staging or similar) to help them see if everything is working with the normal purchase flow.

        The biggest differences to the core e2e tests that come from that are that we’ve (so far) focused more on testing the front-end, customer facing application than the back-end, administration tasks. Most of the objects used for testing that front-end flow, like products etc. are also dynamic and can be changed so that we can run the same tests with different products on different stores.

        I think the work we’re doing (and expect to continue to do a lot of over the coming years) could be very complimentary to the work being done for core e2e tests. We’ll chat about how we can work more closely with these e2e tests internally, but I hope we will be able to integrate and build on top of them rather than duplicating the effort.

  3. Really interesting, do you know if there is anything similar also for WordPress Core features? I am thinking that could be interesting for WooCommerce as well, to know if a release is breaking existing standard WordPress functionality.

    1. There is wp-e2e-page-objects, which has generic WordPress page objects for e2e testing. The WooCommerce library uses it as a dependency, but wp-e2e-page-objects can be used on its own.

  4. Hi, I had a few observations going through the guides for both E2E and unit testing, however the github woocommerce wiki pages don’t appear to welcome comments or contributions, so I logged here:
    https://jonmoblog.wordpress.com/2017/04/27/automated-testing-for-woocommerce-intro/

  5. Hi, Do you have any further thoughts/advice/guidance/examples on extending the testing to plugins?
    (I didn’t find any examples amongst the other https://github.com/woocommerce/ projects, if there was any reference example that would be good.)

    I’m working on the Hyyan WooCommerce Polylang Integration update for WooCommerce3 compatibility.
    Since it covers the entire functionality of wooCommerce, as well has having over 50 options of its own controlling the level of synchronization of data between languages, the number of test cases that should exist is enormous, let’s say it exceeds the capacities of a spare time unfunded project, so any tips much appreciated.

    Ideally I would want to inherit from and extend the WooCommerce tests,
    For example every time there is a test action that affects the inventory, we should test the other language version of the product to see if the inventory is synchronised (or not if the synchronisation option is off).

    Also:
    – the entire test suite should then be run in reverse, that is to say running against the secondary language version of the site and confirming that the data in the primary language is synchronized appropriately
    – both sets of tests (primary language run and secondary language run) should then be rerun for different synchronization settings (Synchronize stock or not is 1 of 50+ settings)
    – ideally a third language test should also be done to confirm there is no code that is expecting only 2 languages

  6. Ideally I would want to inherit from and extend the WooCommerce tests,
    For example every time there is a test action that affects the inventory, we should test the other language version of the product to see if the inventory is synchronised (or not if the synchronisation option is off). are also dynamic and can be changed so that we can run the same tests with different products on different stores.

  7. Ideally I would want to inherit from and extend the WooCommerce tests,
    For example every time there is a test action that affects the inventory, we should test the other language version of the product to see if the inventory is synchronised (or not if the synchronisation option is off).
    Also:
    – the entire test suite should then be run in reverse, that is to say running against the secondary language version of the site and confirming that the data in the primary language is synchronized appropriately
    – both sets of tests (primary language run and secondary language run) should then be rerun for different synchronization settings (Synchronize stock or not is 1 of 50+ settings)
    – ideally a third language test should also be done to confirm there is no code that is expecting only 2 languages

Leave a Reply

Your email address will not be published. Required fields are marked *