Skip to content

[6.x] Batch asset reference updates when moving or deleting many assets - #15866

Open
ryanmitchell wants to merge 1 commit into
statamic:6.xfrom
ryanmitchell:batch-folder-move-reference-updates
Open

ryanmitchell wants to merge 1 commit into
statamic:6.xfrom
ryanmitchell:batch-folder-move-reference-updates

Conversation

@ryanmitchell

Copy link
Copy Markdown
Contributor

Fixes #13813.

Moving, renaming or deleting an asset folder moves or deletes each asset in it, and every asset queued its own UpdateAssetReferences job. Each of those jobs scans all the content on the site (entries, terms, globals, users), so a folder of N assets meant N full scans. On content-heavy sites the jobs time out and the references are never updated. Selecting many assets in the CP and moving or deleting them had the same problem.

This PR adds UpdateAssetReferences::batch():

UpdateAssetReferences::batch(fn () => $assets->each->move('destination'));
  • While a batch is in progress, the listener's shouldQueue() (which Laravel calls when the event is dispatched) records each saved or deleted asset's original and new paths instead of queueing a job.
  • When the batch finishes, the references are updated in a single pass over the content, in a new UpdateBatchedAssetReferences job. Large batches are split into chunks of 500 moves, chained so they run in order. The jobs only carry paths, so the payload stays small.
  • The moves are applied to each item in the order they happened, so paths that are moved more than once, or swapped, end up correct.
  • Nested batches are handled by the outermost one. If the callback throws partway, the assets that did move still get their references updated.
  • Nothing is disabled and re-enabled, so a reference updater that has been disabled stays disabled.

AssetFolder::move() and AssetFolder::delete() run inside a batch, as do the MoveAsset, MoveAssetFolder, RenameAssetFolder and Delete actions.

Each item's data is also JSON-encoded once per pass rather than once per move, by reusing the pre-filter from #15089 through two new static helpers on DataReferenceUpdater.

Benchmarks

Stache, 10,000 entries, renaming a folder with 20 assets:

Time
Before 17.5s
After 5.4s

Renaming a folder with 500 assets across 5,000 entries now takes 4.1s, of which 1.6s is moving the files themselves. Before, it would take 500 full scans.

Notes

  • AssetReferencesUpdated is still dispatched once per asset whose references were updated. For a deleted asset, the event gets an asset made with the container and the original path.
  • A single scan can still exceed a worker timeout on a very large site. That's unchanged by this PR, but it's now one scan instead of one per asset.

🤖 Generated with Claude Code

Moving or deleting an asset folder moved or deleted each asset in it, and
every asset queued its own UpdateAssetReferences job. Each job scans all the
content on the site, so a folder of N assets meant N full scans, which times
out on content-heavy sites.

UpdateAssetReferences::batch() collects the asset moves and deletions made
inside a callback (via the listener's shouldQueue hook) and updates their
references in a single pass, queued as one chained job per 500 moves. Asset
folder moves and deletes, and the bulk move, move folder, rename folder and
delete actions, now run inside a batch.

Moves are applied to each item in the order they happened, and each item's
data is encoded once per pass rather than once per move.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Renaming an asset folder causes the UpdateAssetReferences job to time out and relinking does not occur.

1 participant