The new CRUD classes in WooCommerce 2.7

High order volume is one of the best problems a store can have, but it can really slow down your site’s performance. That’s why our team’s main focus this year (and going into 2017) is performance and scalability.

We’ve put in motion a plan spanning several releases to help us tackle scalability head on, and you can read some of our discussions about orders in particular in this issue.

The first step, and something we’ve invested a lot of time in for WooCommerce 2.7, is abstracting the way we store and retrieve WC data. Why? Because right now we have no knowledge of how developers write data to the database for things like orders and products. A developer can use update_post_meta  calls, direct database writes with the $wpdb class, or the REST API as some examples. If we want to optimize where this data is stored, we first need to ensure developers use a single method of sending this data.

Retrieving data is also fragmented at present. You can use get_posts, $wpdb, WP_Query… all of which require you to know what type of data you’re trying to query.

Ideally, developers should not need to know or care about where and how their data gets stored. Be it meta data, or term data, it should not matter. This is why we’re abstracting everything.

What is CRUD?

CRUD is an abbreviation of the 4 basic operations you can do to a database or resource – Create, Read, Update, Delete.

The benefits:

  • We define the structured data for each resource.
  • We control the flow of data, and any validation needed.
  • As a developer, you don’t need to know the internals of the data you’re working with, just the names.
  • Once abstracted, the data can be moved elsewhere e.g. custom tables, without affecting existing code.
  • We can use the same code for updating things in admin as we do in the REST API and CLIs – everything is unified.
  • Less code = less change to malfunction and more unit test coverage.

Which resources will have a CRUD system in place?

We’re implementing CRUD in:

  • Products (in progress)
  • Customers
  • Orders
  • Order items
  • Coupons

These are the main types of data in WooCommerce and are currently each a custom post type, except for customers which are users and user meta.

Admin won’t be the only part of WooCommerce using the new CRUD operations – our REST API will too. This will make it all easier to maintain and more testable.

In addition, we’re working on making our CLI use the REST API, so again, this consolidates our code base and makes everything much more maintainable.

Examples of using the CRUD

Let’s take an existing example. I want to update an order’s address. With the current code base I’d end up doing something like the following:

$order_id = 100;
update_post_meta( $order_id, '_billing_first_name', 'Fred' );
update_post_meta( $order_id, '_billing_last_name', 'Flintstone' );

I need to know the post ID, the meta keys for each field, and I need to handle all validation myself.

Now the CRUD example:

$order = wc_get_order( 100 );
$order->set_billing_first_name( 'Fred );
$order->set_billing_last_name( 'Flintstone' );
$order->save();

I know an order has a billing first and last name property, but I don’t need to know that it’s post meta, nor that the keys are _billing_first_name and _billing_last_name. If those keys changed in the future, this code would be unaffected.

Getting data from the database follows a similar pattern. Old:

$order_id = 100;
$billing_name = get_post_meta( $order_id, '_billing_first_name', true ) . ' ' . get_post_meta( $order_id, '_billing_last_name', true );

New:

$order = wc_get_order( 100 );
$billing_name = $order->get_billing_first_name() . ' ' . $order->get_billing_last_name();

Where possible, the CRUD classes should handle mostly data reads and writes. Anything template related, such as getting HTML, should be moved to a template function. This is also something we’re working on in 2.7.

Examples of querying resources

Because we intend to move the data to a custom table in the future, it’s important that we offer a way to query the data without going through WordPress itself. Again, the developer should not need to know that orders are a custom post type named shop_order – this is irrelevant.

$customer_orders = get_posts( array(
    'numberposts' => 10,
    'meta_key'    => '_customer_user',
    'meta_value'  => get_current_user_id(),
    'post_type'   => wc_get_order_types( 'view-orders' ),
    'post_status' => array_keys( wc_get_order_statuses() )
) );

vs.

$customer_orders = wc_get_orders( array( 
    'customer' => get_current_user_id(), 
    'limit'    => 10,
) );

How will this affect existing plugins?

If you do anything with product, customer, orders, and coupons, you will be affected in some way. Even if you do a simple update meta call. This won’t break immediately, but your code will not be future proof. As soon as the schema changes in another update, your code will fail.

Helping out and testing

2.7 is scheduled for the new year. We will be releasing the first beta within the next month.

Orders, customers and coupons are all found in the master branch on Github here.

Products are still being built. Progress can be tracked in this PR.

It is very important, especially if you’re a plugin developer, that you get involved and start testing these classes as soon as possible. If you do anything with orders/products/customers, you need to be aware of the changes,  and if you need further changes or spot issues let us know as soon as possible. 

Feedback, code review, and PRs/contribution are welcome and encouraged on Github. We’d like to thank the developers working on this with us in advance. This will improve WooCommerce for everyone in the long run, so as a community let’s make it happen!


35 responses to “The new CRUD classes in WooCommerce 2.7”

  1. Interesting changes, they will surely help with abstraction. The key question, of course, is “how to keep backward compatibility” (a must in most cases) without writing dozens of “if version is 2.7” conditions?

    1. As pointed out, meta still works but won’t (maybe) forever. Our WC_Data can though handle custom meta that is not defined.

  2. Second issue that I can think of is how to handle custom meta. Will new classes be built around magic methods, or will it be necessary to handle custom meta in a different way?

  3. Luke Cavanagh Avatar
    Luke Cavanagh

    This is great news.

  4. Gurdeep Sembi Avatar
    Gurdeep Sembi

    This is really really good news! It will allow us to focus on delivering added functionality with our plugins and if it helps performance then that’s brilliant!

  5. Great move! This is a good step to bring non-wp devs closer.

  6. Love the changes from a developer point of view, but from view of my customers – I think they going to be very upset about things breaking, if the current way stops working in the future.

    1. If the old way were kept forever (meta) we wouldn’t be able to do the performance improvements we plan. So devs will need to eventually shift over.

      1. If the old way were kept forever (meta) we wouldn’t be able to do the performance improvements we plan.

        Uhh, that’s called back-compat 🙂 It shouldn’t matter whether it’s the best practice, it should still work, always or it’s not backward compatible. Period.

        Now obviously, you don’t have to make it easy to do it the old way, and there’s certainly roadblocks – deprecation notices, _doing_it_wrong(), etc – you can throw up to discourage the old ways, but I’m not sure basically saying “we’re going to break back-compat” for a widely extended, extremely popular platform is the wisest move. Besides, your abstraction layer should be able to just handle the old way, but again, discourage it without breaking.

        1. We’re talking about moving where data is stored here. If we moved it to a custom table for example, and you continued to use meta, clearly some things won’t work such as reports querying that data. That’s why this abstraction has to come first. If you don’t use the abstraction layer I don’t really see much that can be done for you, and if that has to be supported forever why bother with any of this at all?

        2. Oh and not saying we should immediately break everything either, this abstraction is the first step as I said. Any breaking schema changes (moving tables as I mentioned) wouldn’t come until a major 3 release. If we can support meta somehow (troll the update post meta hooks and look for relevant data?) we should, but only if it doesn’t further impact performance negatively. WC needs to scale to grow.

          1. I’m on board with the abstraction layer; I think it’s a great move. However, I also think it’s a tad irresponsible for a product of WooCommerce’s size and reach to intentionally break backward-compatibility. Back-compat breaks become necessary for a variety of reasons, this doesn’t feel like one of them. It feels more convenient than necessary.

            If you’re moving away from meta, great, filter the core meta methods to check for certain keys and handle it seamlessly is all I’m saying. Back-compat is sometimes hard but not impossible. The advance notice is much appreciated (from a developer perspective), we’ll see how the customer base votes with their wallets.

          2. Just wanted to point out, since you mentioned our “decision” in your post – nothing has been set in stone yet. The crud part is not implementing tables and the mention of breakage is purely hypothetical. Sorry if unclear.

            Regardless, even with hooks as you point out, depending on how a developer does things some cases are still impossible to support. Direct Wpdb update of meta anyone? So I think a warning in advance is fair.

          3. Gurdeep Sembi Avatar
            Gurdeep Sembi

            I tend to agree with Mike here but Drew raises some good points.

            From my point of view as a developer, I know the biggest gripe/reservations our larger clients have had is around the scalability of WC (I know with the right server setup etc anything is possible), but it is always compared to Magento and I think this is a necessary step in order to really make WC scalable, the post-meta structure is just not going to cut it long term.

            So in summary, I’d accept the short-term pain (will probably lead to a cull of non-supported plugins) for the considerable long-term gain for the WC ecosystem. Anything that can be done to mitigate the transition pain would be greatly appreciated, but 100% coverage of edge conditions shouldn’t be a requisite as I think the WC team should be focused on improving the product more than trying to catch every edge case. This early warning puts it on the radar for plugin devs to test with the beta when it is available to make sure their plugin is still compatible.

          4. hey Guys, I think its great that woocommerce will be improving and more scalable in the future. But I think there will be alot of pain for small store owners.

            Alot of freelancers like me build stores and add custom functionality (not always in plugins, often in the theme itself). Store owners that have had custom functionality added functionality over the years, will possibly require alot of updating of code (that uses things like update meta, if it were to change and not be backwards compatible).

            This will be a potentially be a big cost for store owners, which is not problem for bigger stores. But for small stores maybe doing less that 60 or 70 k a year, its alot of money for functionality they have already paid once for.

          5. Presumably these store owners want to grow sales also, so there are benefits either way. Hopefully they were explained to that code may not work with all future versions, as with any platform, and understand what updating software entails, or have a developer to handle it for them 🙂

  7. […] when I read on the WooCommerce development blog about upcoming changes in 2.7+, I was at first encouraged that they’re embracing an abstraction layer for meta […]

  8. Anything that improve order_items & order_itemmeta is most welcome. If you can quit the previous approach, which, to my understanding, assumed that things (like adding items to an order) is supposed to be done only in a certain context (like a $_POST call), you will make plenty of developers happy 🙂

  9. In general, this is really great news. So much so that I would love to conceptually see it make its way in WordPress core.

    That said, I am disappointed with some of the specifics. First is the use of functions like `wc_get_order()` rather than a factory method on a `WooCommerce` class, e.g. `WooCommerce::get_order( 100 )`. The benefit being to limit the number of functions in the global namespace and make it easier to discover functionality for those that use autocompleting editors like PhpStorm. It would be really awesome if you could embrace using object dispensers using this approach, and plan to deprecate the functions over time.

    Further, I would be great if you could to consider dropping the `get_` from methods that are effectively property accessors that do not take any arguments, e.g.:

    `$billing_name = $order->billing_first_name() . ‘ ‘ . $order->billing_last_name();`

    The benefits being to make using them more natural and less verbose. `get_` could still be used when arguments are needed, e.g. `$orders->get_order_items( $args )`. We use this set of conventions in WPLib and it works brilliantly.

    Finally, it would be great if for things that effectively load from disk `WooCommerce::get_order( 100 )` were named to differentiate (conceptually) accessing from memory vs. loading from disk, e.g. `WooCommerce::load_order( 100 )`

    Of course WordPress overloads `get_*()` horribly: `get_posts()` vs. `get_header()` vs. `get_blog_info()` which all do different things, especially `get_header()`. So I get it, currently you are just being consistent with an inconsistent standard. Would be nice if someone would lead the way though.

  10. This is a great news from the developer view. Adding an abstraction layer between the api calls and the database helps a lot in optimizing and refactoring the code. But please try to keep the backward compatibility as much as you can. WooCommerce didn’t work quite well on this.

  11. CRUD stands for Create, Read, Update & Delete. Sounds cool…! Really Wonderful News. Thanks for updating us.

  12. Please have product variations in mind when you roll out crud classes. Product variations right now need WP_queries for more complex scenarios, for example if you want to get all orders that contain a specific variation of a product. So please consider these edge-cases when implementing CRUD.

  13. Sounds good. We are still very happy to have choosen for WooCommerce years ago. Our main bottleneck has allways been the import of products, although much has improved and it is certainly not a show stopper anymore. Our toy store carries 8500 different products and a full reload via CSV import still takes quite some time. We normally run incremental merges based upon timestamps in the DB that feeds the site via CSV import. So any improvement in that area will be much appreciated. Keep up the good work! Thanks.

  14. The thing is, 2.5 to 2.6 means making changes to clients stuff, and when they’re on a shoe string it’s just not viable, so this is just adding to the fact that they will stay on 2.5 forever, even if there are security risks…
    It seems to me that making a break and say 2.6 onwards is a new product and then keep updating the 2.5 branch with security updates would be a better move, otherwise you’re just invalidating our choice of e-commerce in our clients eyes, or we’ve got to rework things, or our clients get less secure?
    Maybe you should make the 2.5 a ‘lite’ version and apply all the advance stuff into a new version?
    Just my 2p’s worth of the frustration of building a brilliant system only for it to break a year later!

    1. Whether it’s providing compatibility updates or security fixes, sounds like this is something that needs working into client contracts? If there ever was a major security release it’s likely we would have to back port it, but never features and enhancements.

      1. Hi Mike, don’t get me wrong I’m not saying don’t advance the product, but it does seem that over the last couple of years even subtle template changes have made more work for us, especially when you change a template but don’t change the version number (you do, believe me), or deprecate something and I can’t find a replacement without re-writing a whole new plugin.

        The way you want to go sounds like it would alleviate many of the problems we have, but sometimes it would just be nice to say, “there you are all settled for a year”, not every few months go through the whole “what’s changed” routine, it may not affect Woocommerce but it surely affects us poor solution providers…

  15. concerro Avatar

    I’m not a developer, but I do have woocommerce installed on a few sites. Would there be a way to create a plugin that could transfer the old data to the new(in testing) method once/if it becomes finalized so that things breaking does not happen?

    1. Data doesn’t change in this update. only the way in which your plugin accesses and writes data.

  16. […] WooCommerce team’s main focus for 2016 was performance and scalability. This required a huge code refactoring project. Almost everything was touched. As expected, this had repercussions on our plugin too. Even from […]

  17. Will it be possible to update products by SKU and not only by Product ID?

    1. There are methods to lookup by SKU..

  18. We needed to put additional information in to the customer note field (not the order note). There was no existing method that I could find, I hope there comes a crud field for this too, so I can remove this table dependend code:

    $wpdb->update($table,
    array(‘post_excerpt’ => $note),
    array (‘ID’ => $order_id));

  19. […] You can read more about the new CRUD classes here. […]

  20. […] WooCommerce 2.7, we’ve done quite a bit of deprecation mostly due to the new CRUD system. We want developers to make use of the new CRUD objects where possible, but we don’t want to […]

  21. […] our CRUD introduction post, it’s important to standardize accessing and writing data so that we can make improvements in […]

Leave a Reply

Your email address will not be published. Required fields are marked *