Over the last couple of weeks I have been working on a series of pull requests aimed at improving build times and decreasing watch process memory usage. The WooCommerce monorepo is notoriously demanding during development and I am happy to say that things have come down to much more reasonable levels.
Just to lead with it, this project dropped cold build time by 60%, watch ready time by 75%, and watch memory usage by 84%.
Performance improvements by PR
All of these pull requests were tested on an M4 Max with 48 GB of RAM and macOS 26.
| Pull Request | Cold Build | Warm Build | Watch Ready | Watch Memory |
|---|---|---|---|---|
| Baseline | 96s | 6.3s | 132s | 24.4 GB |
| #64876 | 59s | 5.3s | 39s | 18.8 GB |
| #65168 | 25.2s | 5.1s | 33s | 16.6 GB |
| #65210 | 12.8s | 4.0s | 30s | 7.3 GB |
| #65254 | 37.1s | 4.9s | 33s | 3.9 GB |
The cold build regression on the last PR comes from moving the packages into the admin webpack build. This means that Babel has to transpile the TypeScript, which is noticeably slower than esbuild. In the future I can move it to esbuild but there are bigger wins to be had elsewhere right now.
Removing duplicate build work
Special thanks to @kalessil for making himself available to review all of these pull requests!
- Only Emit CJS for Publishing #64876
One of the first things I noticed was that we’re building both ESM and CommonJS modules in our primary workflow. Even though we distribute both, in our bundler, we’re only actually consuming the ESM build. I addressed this by setting up a separate build command for prepack that builds both for publishing.
The next thing I wanted to do was migrate us away from the TypeScript compiler to use something like esbuild. First, as a prerequisite, I needed to move type-checking to a separate lint step and emission to a separate publish step since we wouldn’t be doing it as part of the build anymore. Once that was finished I was able to move all of our package builds over to esbuild and that was a pretty big performance win.
- Centralize
esbuildConfiguration #65422
As a little side-quest I went back and cleaned up the mess I had created when migrating to esbuild. In the interest of keeping the previous PR’s focus small, each package contained a nearly-identical build.mjs file that kicked off the build. I created a new @woocommerce/internal-build package and merged the @woocommerce/internal-ts-config and @woocommerce/internal-style-build packages into it. This will serve as a home for all build-related scripts and configuration files moving forward.
- Build Packages With Admin/Blocks #65254
With everything else complete, I was finally able to get to the catalyst for all of the changes in this project. The entire reason our watch:build command originally used 24.4 GB of memory was because it required running 128 processes. There were processes for ESM, CJS, webpack, and each came with wireit monitors and a PNPM kickoff process. This was necessary because our Admin and Blocks webpack builds consumed transpiled ESM rather than bundling from source.
This pull request changes that, but it means we are no longer able to take advantage of our previous build caching infrastructure. The webpack filesystem cache makes it so you will never even notice, however, our E2E CI jobs now take an additional minute to build. This also means that we’re using Babel to transpile the packages and lose some cold build performance compared to esbuild.
CI throughput is the next performance target
With this rework done, I’m planning to move onto improving the performance of our CI workflow.
Our job matrix sharding approach tends to exhaust all of our GitHub Action workers. This means that, even if I improved the performance of each job, waiting 20+ minutes for a worker makes it meaningless. I’m going to move us towards running tasks in parallel on the same workers so that we can increase overall CI throughput. I’d also like to take a look at improving our E2E suite speed by using multisite (or something similar) to run each test suite in parallel in the same environment.
Leave a Reply