fix(project): Add polling fallback for file watching in containers - #1511
Draft
RandomByte wants to merge 1 commit into
Draft
fix(project): Add polling fallback for file watching in containers#1511RandomByte wants to merge 1 commit into
RandomByte wants to merge 1 commit into
Conversation
Some container setups (Docker over overlayfs, bind mounts, network
filesystems) do not surface filesystem events: @parcel/watcher's
subscribe() resolves and then stays silent, with no error to catch. The
incremental build derives "what changed" solely from watcher events, so a
silent watcher breaks rebuilds and live reload entirely.
Parcel offers no polling subscription, so add one in a new pollingWatcher.js
module that all three watcher consumers now call instead of @parcel/watcher
directly. It exposes a subscribe() matching @parcel/watcher's exact
contract. The choice between the native watcher and polling is made once per
process: UI5_WATCH_MODE=polling|native forces it, otherwise a startup test
subscribes with the native watcher to a temp dir on the same filesystem as
the real watch, writes a file, and falls back to polling if no event arrives
within 1500 ms.
The polling path walks the tree and diffs an mtimeMs+size snapshot every
250 ms (rescheduling itself so a slow walk cannot overlap the next one),
emitting the same {type, path} events the native watcher would. Errors flow
through the callback so each consumer's existing recovery path fires
unchanged. The 250 ms interval stays below WATCHER_BURST_SETTLE_MS (550 ms)
so downstream event batching still works. Ignore globs reuse micromatch and
skip ignored directories so node_modules is never walked.
WatchHandler, ProjectDefinitionWatcher, and projectGraphSettleWatcher swap
their parcelWatcher.subscribe call for the new module; callback bodies,
ignore globs, and settle windows are untouched.
The walk competes with the initial build for file descriptors. On a large
tree under a low ulimit the build can use them all, so readdir/stat fail with
EMFILE/ENFILE/EAGAIN. Reporting these would make every consumer tear down and
re-subscribe, so they are retried on the next tick and only reported once
they persist past a grace window. Because polling resamples the whole tree
every interval, a skipped walk loses nothing: the next walk rebuilds the
snapshot. To see the shortage directly, the walk reads node:fs/promises
rather than graceful-fs, which would retry those errors internally and hide
the signal.
Fixes: #1479
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Some container setups (Docker over overlayfs, bind mounts, network filesystems) do not surface filesystem events: @parcel/watcher's subscribe() resolves and then stays silent, with no error to catch. The incremental build derives "what changed" solely from watcher events, so a silent watcher breaks rebuilds and live reload entirely.
Parcel offers no polling subscription, so add one in a new pollingWatcher.js module that all three watcher consumers now call instead of @parcel/watcher directly. It exposes a subscribe() matching @parcel/watcher's exact contract. The choice between the native watcher and polling is made once per process: UI5_WATCH_MODE=polling|native forces it, otherwise a startup test subscribes with the native watcher to a temp dir on the same filesystem as the real watch, writes a file, and falls back to polling if no event arrives within 1500 ms.
The polling path walks the tree and diffs an mtimeMs+size snapshot every 250 ms (rescheduling itself so a slow walk cannot overlap the next one), emitting the same {type, path} events the native watcher would. Errors flow through the callback so each consumer's existing recovery path fires unchanged. The 250 ms interval stays below WATCHER_BURST_SETTLE_MS (550 ms) so downstream event batching still works. Ignore globs reuse micromatch and skip ignored directories so node_modules is never walked.
WatchHandler, ProjectDefinitionWatcher, and projectGraphSettleWatcher swap their parcelWatcher.subscribe call for the new module; callback bodies, ignore globs, and settle windows are untouched.
The walk competes with the initial build for file descriptors. On a large tree under a low ulimit the build can use them all, so readdir/stat fail with EMFILE/ENFILE/EAGAIN. Reporting these would make every consumer tear down and re-subscribe, so they are retried on the next tick and only reported once they persist past a grace window. Because polling resamples the whole tree every interval, a skipped walk loses nothing: the next walk rebuilds the snapshot. To see the shortage directly, the walk reads node:fs/promises rather than graceful-fs, which would retry those errors internally and hide the signal.
Fixes: #1479