Skip to content

Speed up PHPUnit and avoid E2E setup timeouts - #579

Merged
JanJakes merged 2 commits into
trunkfrom
fix-e2e-browser-setup
Oct 28, 2026
Merged

JanJakes merged 2 commits into
trunkfrom
fix-e2e-browser-setup

Conversation

@JanJakes

@JanJakes JanJakes commented Oct 28, 2026 •

Copy link
Copy Markdown
Member

Summary

E2E jobs sometimes exhaust their 20-minute timeout downloading Playwright system packages. This PR removes that setup step and speeds up WordPress PHPUnit.

  • Avoid browser installation timeouts by using the official Docker image matching WordPress’s installed Playwright version.
  • Pull Docker images faster and more reliably. The Playwright and WordPress images download in parallel. A stuck Playwright image pull stops after 3 minutes, and a failed one is retried, up to 3 attempts. Setup now creates WordPress’s .env file early, so the parallel pull uses the same settings as the test environment.
  • Show browser server logs when an E2E job fails or times out.
  • Speed up WordPress PHPUnit by enabling CLI OPcache and tracing JIT.
  • Remove unused Docker Buildx setup from the four WordPress test workflows.

Test order, expected failures, error reporting, and job timeouts stay unchanged.

Measured speedups

GitHub-hosted runners vary in speed. During these measurements, the same PHPUnit job took up to about 70% longer on slower runners. To compare fairly, each PHPUnit time is divided by the webpack build time from the same job. The build runs before the tests, and this PR does not change it.

Mode Runs PHPUnit ÷ build time Change
Without OPcache (trunk and other PRs) 10 34.1 —
OPcache 3 29.0 ~15% faster
OPcache and tracing JIT 3 25.5 ~25% faster

Comparing only runs on the slower runners, PHPUnit took 9m46s on average without OPcache and 7m49s with OPcache and JIT, about 20% faster. The plugin PHPUnit suites take about 10 seconds, too short to show a difference.

These are estimates from 16 runs on the same day, all on WordPress 7.1.4.

All runs
Run Mode PHPUnit Build Ratio
40716913795 OPcache and JIT 7m59.1s 18.0s 26.6
40712439242 OPcache and JIT 7m38.4s 17.8s 25.8
40704150768 OPcache and JIT 5m16.3s 13.1s 24.2
40701867316 OPcache 8m05.5s 17.0s 28.6
40694469161 OPcache 8m43.9s 17.7s 29.6
40690272800 OPcache 8m41.0s 18.1s 28.8
40716362956 Without OPcache 7m03.8s 11.9s 35.6
40712857322 Without OPcache 10m02.3s 18.1s 33.3
40709850513 Without OPcache 6m17.2s 12.6s 29.8
40708807510 Without OPcache 9m33.9s 16.8s 34.3
40703990906 Without OPcache 6m19.5s 10.1s 37.6
40703886731 Without OPcache 6m27.3s 10.4s 37.4
40701787710 Without OPcache 5m46.9s 12.7s 27.2
40697656411 Without OPcache 8m07.6s 13.7s 35.7
40689426132 Without OPcache 7m23.4s 12.2s 36.5
40669664710 Without OPcache 9m42.9s 17.3s 33.6

@coderabbitai

coderabbitai Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 19 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 2 included reviews currently available.

Learn how review limits work.

Review configuration:

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 142dc5bc-66df-497d-8107-40286fdd37c4
Commits

Reviewing files that changed from the base of the PR and between ec510fb and 84eb24a.

Files selected for processing: 8
  • .github/actions/playwright/action.yml
  • .github/actions/playwright/docker-compose.yml
  • .github/workflows/end-to-end-tests.yml
  • .github/workflows/sqlite-plugin-phpunit-tests.yml
  • .github/workflows/wp-tests-end-to-end.yml
  • .github/workflows/wp-tests-phpunit.yml
  • composer.json
  • wp-setup.sh
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@JanJakes JanJakes changed the title Avoid E2E timeouts during Playwright setup Speed up PHPUnit and avoid E2E setup timeouts Oct 28, 2026
@JanJakes
JanJakes force-pushed the fix-e2e-browser-setup branch 3 times, most recently from e7329d8 to a6f85a1 Compare October 28, 2026 15:05
@JanJakes
JanJakes marked this pull request as ready for review October 28, 2026 15:08
Run E2E browsers in the official image matching the installed Playwright version. Use Docker Compose for readiness and a composite action for test execution and cleanup, avoiding Ubuntu package downloads.

Enable CLI OPcache with a 256 MB cache for WordPress PHPUnit while preserving test diagnostics and file-change checks. Remove unused Buildx setup from the WordPress test workflows.

https://playwright.dev/docs/docker#remote-connection
Use a 64 MB JIT buffer for the WordPress and SQLite plugin PHPUnit suites. Keep the existing OPcache and test settings unchanged.
@JanJakes
JanJakes force-pushed the fix-e2e-browser-setup branch from a6f85a1 to 84eb24a Compare October 28, 2026 15:31
@JanJakes
JanJakes merged commit 84bff0c into trunk Oct 28, 2026
18 checks passed
@JanJakes
JanJakes deleted the fix-e2e-browser-setup branch October 28, 2026 15:55
@JanJakes JanJakes mentioned this pull request Nov 3, 2026
JanJakes added a commit that referenced this pull request Nov 3, 2026
## Release `3.1.3`

Version bump and changelog update for release `3.1.3`.

**Changelog draft:**
* Fix `BINARY` and `VARBINARY` column defaults
([#596](#596))
* Quote SQLite identifiers in `LOCK TABLES`, `SHOW TABLES`, and compound
primary key indexes
([#599](#599))
* Name lexer tokens after the MySQL grammar and default to MySQL 8.4.10
([#607](#607))
* Fix MySQL lexer correctness issues
([#584](#584))
* Require MySQL proxy authentication and default to loopback
([#540](#540))
* Speed up PHPUnit and avoid E2E setup timeouts
([#579](#579))
* Fix reserved prefix checks for case variants and foreign keys
([#580](#580))

**Full changelog:**
v3.1.2...release/v3.1.3

## Next steps

1. **Review** the changes in this pull request.
2. **Push** any additional edits to this branch (`release/v3.1.3`).
3. **Merge** this pull request to complete the release.

Merging will automatically build the plugin ZIP, create a [GitHub
release](https://github.com/WordPress/sqlite-database-integration/releases),
and deploy to
[WordPress.org](https://wordpress.org/plugins/sqlite-database-integration/).

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->

## Summary by CodeRabbit

* **Bug Fixes**
* Improved handling of binary column defaults and SQLite identifier
quoting.
  * Corrected MySQL lexer tokens, defaults, and lexer behavior.
* Updated MySQL proxy authentication and loopback defaults, test timing,
and reserved-prefix checks.
* **Release**
  * Updated the plugin and driver version to 3.1.3.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
JanJakes added a commit that referenced this pull request Nov 6, 2026
## Summary

The WordPress PHPUnit job takes about 10 minutes, and most of that time
is PHPUnit itself. This PR speeds it up with two small changes that only
affect the tests:

- **Turn off PHP's garbage collector for PHPUnit.** While tests run, PHP
keeps searching a large amount of memory for objects it can free, and it
rarely finds much. In a short-lived test process, this isn't worth the
time.
- **Use the low bcrypt cost in class fixtures too.**
`WP_UnitTestCase::set_up()` lowers the bcrypt cost for each test, but
`wpSetUpBeforeClass()` runs before it. Users created there still hash
passwords at PHP's default cost, and those hashes took most of the time
spent in class fixtures.

Test order and the list of expected failures stay unchanged.

**Memory use goes up.** A full WordPress PHPUnit run now peaks at about
4.2 GB instead of about 0.5 GB. This fits the 16 GB CI runner, but a
local Docker setup needs that much memory available for a full run. The
WordPress test bootstrap already removes the PHP memory limit, so no new
limit setting is needed. The SQLite plugin suites use the same command
and use very little memory either way.

The bcrypt issue is in WordPress's own test code, so it could also be
fixed there. Until then, `wp-setup.sh` adds the filter, as it already
does for other test changes.

## Why

PHPUnit takes about 7.5 minutes of the job on slower runners, and the
job is by far the slowest one in CI. Profiling showed that the SQLite
database itself is under 5% of that time, so these changes target where
the time actually goes.

## Measured speedups

**On CI:** GitHub-hosted runners vary a lot in speed, so each run is
compared with baseline runs on a runner of similar speed. As in #579,
the webpack build time from the same job shows the runner's speed. The
baseline is 14 recent runs without this change, 12 of which match these
two speeds.

| Runner | Baseline PHPUnit | This PR | Change | PHPUnit ÷ build time |
Whole job |
| --- | ---: | ---: | ---: | ---: | ---: |
| Slower (build about 14s) | 464s (10 runs) |
[286s](https://github.com/WordPress/sqlite-database-integration/actions/runs/42244888948/attempts/3)
| **~38% faster** | 32.9 → 21.2 | ~10m50s → 7m35s |
| Faster (build about 9s) | 311s (2 runs) |
[172s](https://github.com/WordPress/sqlite-database-integration/actions/runs/42244888948/attempts/2)
| **~45% faster** | 35.8 → 19.0 | ~7m30s → 5m13s |

These are two runs of this PR, one on each runner speed. Test results
match the baseline runs: 30,900 tests, with the same errors and
failures, and the expected failures check passes. Peak memory went from
515 MB to 4.18 GB.

**Locally:** the full suite on native PHP 8.3 (not Docker), in seconds.
Each row adds to the one above it. The list of failing tests was
identical in all three runs.

| Mode | PHPUnit | Change |
| --- | ---: | ---: |
| Current settings | 197.7s | — |
| Garbage collector off | 153.8s | **~22% faster** |
| Garbage collector off and low bcrypt cost in fixtures | 126.6s |
**~36% faster** |


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

<!-- coderabbit-ai-generated:v1 -->

## Summary by CodeRabbit

* **Chores**
* Updated PHP test execution settings and WordPress test setup. These
changes affect the project’s testing configuration; no end-user-facing
changes are included.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant