Skip to content

fix(project): Add polling fallback for file watching in containers - #1511

Draft
RandomByte wants to merge 1 commit into
mainfrom
fix/fs-watch-in-containers
Draft

fix(project): Add polling fallback for file watching in containers#1511
RandomByte wants to merge 1 commit into
mainfrom
fix/fs-watch-in-containers

Conversation

@RandomByte

Copy link
Copy Markdown
Member

Some container setups (Docker over overlayfs, bind mounts, network filesystems) do not surface filesystem events: @parcel/watcher's subscribe() resolves and then stays silent, with no error to catch. The incremental build derives "what changed" solely from watcher events, so a silent watcher breaks rebuilds and live reload entirely.

Parcel offers no polling subscription, so add one in a new pollingWatcher.js module that all three watcher consumers now call instead of @parcel/watcher directly. It exposes a subscribe() matching @parcel/watcher's exact contract. The choice between the native watcher and polling is made once per process: UI5_WATCH_MODE=polling|native forces it, otherwise a startup test subscribes with the native watcher to a temp dir on the same filesystem as the real watch, writes a file, and falls back to polling if no event arrives within 1500 ms.

The polling path walks the tree and diffs an mtimeMs+size snapshot every 250 ms (rescheduling itself so a slow walk cannot overlap the next one), emitting the same {type, path} events the native watcher would. Errors flow through the callback so each consumer's existing recovery path fires unchanged. The 250 ms interval stays below WATCHER_BURST_SETTLE_MS (550 ms) so downstream event batching still works. Ignore globs reuse micromatch and skip ignored directories so node_modules is never walked.

WatchHandler, ProjectDefinitionWatcher, and projectGraphSettleWatcher swap their parcelWatcher.subscribe call for the new module; callback bodies, ignore globs, and settle windows are untouched.

The walk competes with the initial build for file descriptors. On a large tree under a low ulimit the build can use them all, so readdir/stat fail with EMFILE/ENFILE/EAGAIN. Reporting these would make every consumer tear down and re-subscribe, so they are retried on the next tick and only reported once they persist past a grace window. Because polling resamples the whole tree every interval, a skipped walk loses nothing: the next walk rebuilds the snapshot. To see the shortage directly, the walk reads node:fs/promises rather than graceful-fs, which would retry those errors internally and hide the signal.

Fixes: #1479

Some container setups (Docker over overlayfs, bind mounts, network
filesystems) do not surface filesystem events: @parcel/watcher's
subscribe() resolves and then stays silent, with no error to catch. The
incremental build derives "what changed" solely from watcher events, so a
silent watcher breaks rebuilds and live reload entirely.

Parcel offers no polling subscription, so add one in a new pollingWatcher.js
module that all three watcher consumers now call instead of @parcel/watcher
directly. It exposes a subscribe() matching @parcel/watcher's exact
contract. The choice between the native watcher and polling is made once per
process: UI5_WATCH_MODE=polling|native forces it, otherwise a startup test
subscribes with the native watcher to a temp dir on the same filesystem as
the real watch, writes a file, and falls back to polling if no event arrives
within 1500 ms.

The polling path walks the tree and diffs an mtimeMs+size snapshot every
250 ms (rescheduling itself so a slow walk cannot overlap the next one),
emitting the same {type, path} events the native watcher would. Errors flow
through the callback so each consumer's existing recovery path fires
unchanged. The 250 ms interval stays below WATCHER_BURST_SETTLE_MS (550 ms)
so downstream event batching still works. Ignore globs reuse micromatch and
skip ignored directories so node_modules is never walked.

WatchHandler, ProjectDefinitionWatcher, and projectGraphSettleWatcher swap
their parcelWatcher.subscribe call for the new module; callback bodies,
ignore globs, and settle windows are untouched.

The walk competes with the initial build for file descriptors. On a large
tree under a low ulimit the build can use them all, so readdir/stat fail with
EMFILE/ENFILE/EAGAIN. Reporting these would make every consumer tear down and
re-subscribe, so they are retried on the next tick and only reported once
they persist past a grace window. Because polling resamples the whole tree
every interval, a skipped walk loses nothing: the next walk rebuilds the
snapshot. To see the shortage directly, the walk reads node:fs/promises
rather than graceful-fs, which would retry those errors internally and hide
the signal.

Fixes: #1479
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

UI5 Build Cache does not work in case process is executed in a container

1 participant