317dd3ad fix(core): combine Windows watcher recursive root and rescan recovery (#36912)
## Current Behavior
On Windows the Nx daemon registers one non-recursive `notify` watch per
directory, for both the source watcher and the outputs watcher. That
costs 2 kernel handles per directory (measured slope 2.000008/dir in
#36563, corroborated at 2.12/dir on a second harness), memory that grows
linearly with directory count, and a volume-wide file-operation slowdown
while the daemon runs. A large workspace reaches ~192,000 handles and
~1.6 GB with a 34s watcher startup.
Separately, when the kernel cannot deliver every file event, Nx loses
the difference silently and permanently. On Linux an inotify queue
overflow arrives as notify's rescan flag and Nx discards it (45-61% of a
forced burst lost with no signal). On Windows the loss is structurally
invisible: notify 8 never raises rescan there, and Nx handles rescan on
no platform. The daemon's view of the workspace diverges from disk until
someone runs `nx reset`.
## Expected Behavior
This branch combines the two open PRs that together fix both halves, so
the whole Windows watcher slice can be tested as one unit:
- **#36811 (christopher-buss)** widens the recursive-root watch strategy
from macOS to Windows: one registration, constant handle and memory
cost, no volume-wide tax. In an A/B on real hardware this is 205 handles
flat against 3,423 on base.
- **#36566 (Stephen/Jupiter Jayna)** makes kernel event loss visible and
bounded: the native watcher recognises notify's rescan flag and emits a
single rescan event; the daemon re-walks the workspace, diffs against
its known file map, and synthesizes exactly the missed events through
the normal recomputation path. A no-change rescan preserves the cached
graph.
The two compose cleanly (both cherry-picks applied without conflict;
`cargo check -p nx` green). #36811's recursive root is what makes the
rescan recovery necessary on Windows, and the recovery is what makes the
recursive root safe.
On top of the two combined PRs, the slice adds:
- **Recovery correctness.** The rescan marker accompanies the
accumulated batch rather than replacing it, so ignore-file and
`server-process.json` events in the same overflow burst still reach the
daemon's intercepts (which read the raw array ahead of per-path
routing). The recovery walk and diff run inside the workspace context,
so a rescan no longer materialises the full file list across the napi
boundary twice.
- **`notify` bumped to `9.0.0-rc.5`** (notify-rs/notify#964), the first
release that raises the rescan flag on Windows. Under notify 8 the
recovery path is Linux-only. Pinned exactly because it is a prerelease.
- **Ignore parity with the walk.** The recursive root makes the runtime
filterer the only ignore gate on Windows. It now enforces the hardcoded
ignores (`node_modules`, `.git`, `.nx/cache`, `.yarn/cache`) as an
unbeatable veto, and reads the same sources `create_walker` does:
`.gitignore`, `.git/info/exclude`, the global `core.excludesFile`, and
nested `.nxignore`. Precedence matches the ignore crate in both
directions, so the higher class wins at any depth and the deeper file
wins within a class. Neither side reads `.ignore`, which is a ripgrep
convention nx never chose. Without this the watcher would admit files
the rescan walk drops and report them deleted on every overflow.
- **Fewer stats, fewer walks.** Per-path stats are taken lazily and only
for ambiguous event kinds (the elimination lands on Linux; on Windows
notify has already stat'd). Rescan markers are coalesced across a
sustained storm so recovery re-walks once, not once per flush.
Two user-visible behavior changes fall out of this:
- **Editing a watched ignore file restarts the daemon.** The native
filterer's ignore rules are fixed when the watcher starts, so a change
to a watched `.gitignore` or `.nxignore` now stops the daemon and it
restarts on the next command with the rules reloaded. That holds when
the edit is only recovered through a rescan after an overflow.
Previously an ignore-file edit could go unreflected until the next
daemon start.
- **One full re-hash on the first run after upgrade.** The files-archive
format changes (`nx_files.nxt` → `nx_files_v2.nxt`), because an mtime
match alone can reuse a stale hash for a file rewritten within the
gather window. The first daemon run after upgrading discards the old
archive and pays one full workspace re-hash; steady state is unchanged.
Rust watch tests: 31/31 on macOS locally. Linux and Windows run on CI.
## Related Issue(s)
Addresses #36563. Combines #36566 and #36811.
Co-authored-by: Jupiter Jayna <636020+sdjayna@users.noreply.github.com>
Co-authored-by: Christopher Buss
<christopher.buss+github@protonmail.com>
---------
Co-authored-by: Stephen Jayna <Stephen.Jayna@GBM-CX167GGF4C.local>
Co-authored-by: christopher-buss <christopher.buss+github@protonmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: nx-cloud[bot] <71083854+nx-cloud[bot]@users.noreply.github.com>
Co-authored-by: FrozenPandaz <jasonjean1993@gmail.com> b616fa1d fix(core): detect changes in paths that git escapes (#36660)
## Current Behavior
When a changed file lives under a directory whose name contains
non-ASCII characters, affected detection silently finds nothing.
Changing `apps/cart/src/app/Einkäufe/test.tsx` and running `nx show
projects --affected` returns no projects.
Git C-quotes any path containing non-ASCII bytes when `core.quotepath`
is enabled, which is the default:
```
"apps/cart/src/app/Eink\303\244ufe/test.tsx" <- umlaut path, quoted and octal-escaped
apps/cart/src/app/plain/test.tsx <- ASCII path, untouched
```
`parseGitOutput` split that output on newlines and trimmed each entry,
so the quoted literal was passed downstream unchanged. It matched no
project root, so nothing was reported as affected.
The reporter noted that setting `git config core.quotepath false` works
around it, but Nx should not depend on that setting.
## Expected Behavior
Affected detection works regardless of `core.quotepath`, and `cart` /
`cart-e2e` are reported as affected.
This passes `-z` to the three git readers (`getUncommittedFiles`,
`getUntrackedFiles`, `getFilesUsingBaseAndHead`) and splits
`parseGitOutput` on NUL instead of newline.
### Why `-z` rather than `core.quotepath=false`
`git -c core.quotepath=false` only suppresses escaping of non-ASCII
bytes. Paths containing `"` or `\` are still C-quoted:
```
$ git -c core.quotepath=false diff --name-only HEAD
"apps/cart/src/app/back\\slash/test.tsx"
"apps/cart/src/app/quo\"te/test.tsx"
$ git diff --name-only -z HEAD | tr '\0' '\n'
apps/cart/src/app/back\slash/test.tsx
apps/cart/src/app/quo"te/test.tsx
```
So `core.quotepath=false` would have fixed umlauts while leaving a
second, subtler bug in place. `-z` disables path quoting entirely.
Two details worth flagging for review:
- `-z` **must precede non-option arguments** for `git diff` (otherwise
`fatal: option '-z' must come before non-option arguments`), so it is
placed per-call-site rather than centralised in `parseGitOutput`.
- Git emits a trailing NUL, so the empty-entry filter is retained.
Dropping the now-redundant `.trim()` additionally fixes paths with
leading or trailing spaces, which the previous code silently corrupted.
All consumers of the `parseFiles` seam benefit: `show projects
--affected`, `affected`, `release plan`/`plan-check`, and `format`.
## Tests
Adds `packages/nx/src/utils/command-line-utils.real.spec.ts`, following
the existing `.real.spec.ts` convention of exercising **real temporary
git repositories** rather than mocking `child_process`. This was
deliberate — a mocked test cannot catch the `-z` argument-order
constraint above. Four cases (uncommitted, untracked, base/head, and
paths containing `"` / trailing spaces); all four fail before the fix
and pass after. The quote/trailing-space case is skipped on Windows,
where those characters are not legal in a path segment.
Validation: `nx lint nx` passes, prettier clean, and a full jest run
over `src/utils`, `src/command-line/affected`, `src/command-line/show`,
and `src/project-graph/file-utils` is identical to pristine `master`
apart from the 4 new passing tests.
Also ports the e2e case from #35731 into
`e2e/nx/src/affected-graph.test.ts`. The unit specs stop at
`parseFiles`, so nothing here covered the step where the reported
symptom actually surfaced, matching a changed path against a project
root.
## Also in this PR: prettier is spawned with argv, not a shell string
Reading paths with `-z` makes a second, pre-existing bug **reachable**,
so it is fixed here rather than deferred.
`writeWithPrettier` / `checkWithPrettier` interpolated each pattern into
a command string for `execSync`/`exec`, quoting it with `quoteForShell`.
That quoting is complete on POSIX but a no-op on Windows:
```ts
const escaped =
process.platform !== 'win32' ? pattern.replace(/([\\"`$])/g, '\\$1') : pattern;
return `"${escaped}"`;
```
On Windows a path containing `"` closes the quoting and the rest of the
path is parsed as shell. Before `-z`, git handed those paths back
C-quoted and the `fileExists` filter dropped them, so the hole was
unreachable. It is reachable now.
Both entry points now spawn with `execFileSync`/`execFile` and an argv
array, removing the shell entirely along with `quoteForShell` and the
escaping it needed. Nothing globbed through the shell before — patterns
were quoted, so the shell never expanded them, and prettier does its own
globbing.
### Why `chunkify`'s `measure` parameter is deleted
This is the change most likely to read as unrelated, so to be explicit:
`measure` existed **only** to compensate for the quoting removed above,
and has no other caller in the repo.
`chunkify` splits the file list into batches that fit under the OS
command-line limit. Before oxfmt, `format.ts` quoted patterns *before*
chunking, so the array held the exact strings that would reach the
command line and the sizing was correct for free.
[#35089](https://github.com/nrwl/nx/pull/35089) (built-in oxfmt support)
had to stop pre-quoting, because oxfmt spawns with `execFile` and needs
raw paths. Prettier's quoting moved down to its spawn site, which
inverted the order — chunk first, quote later — leaving `chunkify`
sizing `apps/foo.ts` while the command line would carry `"apps/foo.ts"`,
silently eating the headroom the budget reserves. `measure` was added in
that same commit so the caller could say "size against the quoted
length".
With prettier on `execFile` too, both consumers take paths as-is,
on-the-wire length is raw length again, and there is nothing left to
compensate for. So `measure` and its three spec cases are removed rather
than left as dead API surface.
### Tests for this half
- `quote-for-shell.spec.ts` is **deleted** — it asserted a shell-quoting
property that no longer exists.
- `prettier-argv.spec.ts` **replaces** that coverage: it asserts paths
containing `"`, backticks, `$(…)`, `$HOME` and `\$` reach prettier as
literal argv entries. Both cases fail if the quoting is reintroduced
(verified by mutation) and pass without it.
- `check-with-prettier.spec.ts`'s mock moves from `exec(cmd, opts, cb)`
to `execFile(file, args, opts, cb)`.
Validation for this half: `tsc --noEmit` clean on every touched file; a
jest run over `src/utils/formatters`, `src/command-line/format` and
`src/command-line/init` is identical to pristine `master` (the three
oxfmt-dependent suites fail on both).
## Related Issue(s)
Fixes #35722
Supersedes #35731, which reported and fixed the same bug. That branch
predates the move of `parseGitOutput` to an argv array, and it does not
cover the `"`/backslash class or the prettier spawn downstream. Its e2e
case is carried over here, with credit.
<!-- polygraph-session-start -->
---
<p><a
href="https://app.trypolygraph.com/orgs/6a061dcb561c062131116eca/sessions/Fix-nrwl-nx-issue-35722---UTF-8-path-detection-with-git-19db98e8">View
Polygraph session ↗</a></p>
<!-- polygraph-session-end -->
---------
Co-authored-by: nx-cloud[bot] <71083854+nx-cloud[bot]@users.noreply.github.com>
Co-authored-by: Christina Huege <11138584+chrstnbrn@users.noreply.github.com> b616fa1d fix(core): detect changes in paths that git escapes (#36660)
## Current Behavior
When a changed file lives under a directory whose name contains
non-ASCII characters, affected detection silently finds nothing.
Changing `apps/cart/src/app/Einkäufe/test.tsx` and running `nx show
projects --affected` returns no projects.
Git C-quotes any path containing non-ASCII bytes when `core.quotepath`
is enabled, which is the default:
```
"apps/cart/src/app/Eink\303\244ufe/test.tsx" <- umlaut path, quoted and octal-escaped
apps/cart/src/app/plain/test.tsx <- ASCII path, untouched
```
`parseGitOutput` split that output on newlines and trimmed each entry,
so the quoted literal was passed downstream unchanged. It matched no
project root, so nothing was reported as affected.
The reporter noted that setting `git config core.quotepath false` works
around it, but Nx should not depend on that setting.
## Expected Behavior
Affected detection works regardless of `core.quotepath`, and `cart` /
`cart-e2e` are reported as affected.
This passes `-z` to the three git readers (`getUncommittedFiles`,
`getUntrackedFiles`, `getFilesUsingBaseAndHead`) and splits
`parseGitOutput` on NUL instead of newline.
### Why `-z` rather than `core.quotepath=false`
`git -c core.quotepath=false` only suppresses escaping of non-ASCII
bytes. Paths containing `"` or `\` are still C-quoted:
```
$ git -c core.quotepath=false diff --name-only HEAD
"apps/cart/src/app/back\\slash/test.tsx"
"apps/cart/src/app/quo\"te/test.tsx"
$ git diff --name-only -z HEAD | tr '\0' '\n'
apps/cart/src/app/back\slash/test.tsx
apps/cart/src/app/quo"te/test.tsx
```
So `core.quotepath=false` would have fixed umlauts while leaving a
second, subtler bug in place. `-z` disables path quoting entirely.
Two details worth flagging for review:
- `-z` **must precede non-option arguments** for `git diff` (otherwise
`fatal: option '-z' must come before non-option arguments`), so it is
placed per-call-site rather than centralised in `parseGitOutput`.
- Git emits a trailing NUL, so the empty-entry filter is retained.
Dropping the now-redundant `.trim()` additionally fixes paths with
leading or trailing spaces, which the previous code silently corrupted.
All consumers of the `parseFiles` seam benefit: `show projects
--affected`, `affected`, `release plan`/`plan-check`, and `format`.
## Tests
Adds `packages/nx/src/utils/command-line-utils.real.spec.ts`, following
the existing `.real.spec.ts` convention of exercising **real temporary
git repositories** rather than mocking `child_process`. This was
deliberate — a mocked test cannot catch the `-z` argument-order
constraint above. Four cases (uncommitted, untracked, base/head, and
paths containing `"` / trailing spaces); all four fail before the fix
and pass after. The quote/trailing-space case is skipped on Windows,
where those characters are not legal in a path segment.
Validation: `nx lint nx` passes, prettier clean, and a full jest run
over `src/utils`, `src/command-line/affected`, `src/command-line/show`,
and `src/project-graph/file-utils` is identical to pristine `master`
apart from the 4 new passing tests.
Also ports the e2e case from #35731 into
`e2e/nx/src/affected-graph.test.ts`. The unit specs stop at
`parseFiles`, so nothing here covered the step where the reported
symptom actually surfaced, matching a changed path against a project
root.
## Also in this PR: prettier is spawned with argv, not a shell string
Reading paths with `-z` makes a second, pre-existing bug **reachable**,
so it is fixed here rather than deferred.
`writeWithPrettier` / `checkWithPrettier` interpolated each pattern into
a command string for `execSync`/`exec`, quoting it with `quoteForShell`.
That quoting is complete on POSIX but a no-op on Windows:
```ts
const escaped =
process.platform !== 'win32' ? pattern.replace(/([\\"`$])/g, '\\$1') : pattern;
return `"${escaped}"`;
```
On Windows a path containing `"` closes the quoting and the rest of the
path is parsed as shell. Before `-z`, git handed those paths back
C-quoted and the `fileExists` filter dropped them, so the hole was
unreachable. It is reachable now.
Both entry points now spawn with `execFileSync`/`execFile` and an argv
array, removing the shell entirely along with `quoteForShell` and the
escaping it needed. Nothing globbed through the shell before — patterns
were quoted, so the shell never expanded them, and prettier does its own
globbing.
### Why `chunkify`'s `measure` parameter is deleted
This is the change most likely to read as unrelated, so to be explicit:
`measure` existed **only** to compensate for the quoting removed above,
and has no other caller in the repo.
`chunkify` splits the file list into batches that fit under the OS
command-line limit. Before oxfmt, `format.ts` quoted patterns *before*
chunking, so the array held the exact strings that would reach the
command line and the sizing was correct for free.
[#35089](https://github.com/nrwl/nx/pull/35089) (built-in oxfmt support)
had to stop pre-quoting, because oxfmt spawns with `execFile` and needs
raw paths. Prettier's quoting moved down to its spawn site, which
inverted the order — chunk first, quote later — leaving `chunkify`
sizing `apps/foo.ts` while the command line would carry `"apps/foo.ts"`,
silently eating the headroom the budget reserves. `measure` was added in
that same commit so the caller could say "size against the quoted
length".
With prettier on `execFile` too, both consumers take paths as-is,
on-the-wire length is raw length again, and there is nothing left to
compensate for. So `measure` and its three spec cases are removed rather
than left as dead API surface.
### Tests for this half
- `quote-for-shell.spec.ts` is **deleted** — it asserted a shell-quoting
property that no longer exists.
- `prettier-argv.spec.ts` **replaces** that coverage: it asserts paths
containing `"`, backticks, `$(…)`, `$HOME` and `\$` reach prettier as
literal argv entries. Both cases fail if the quoting is reintroduced
(verified by mutation) and pass without it.
- `check-with-prettier.spec.ts`'s mock moves from `exec(cmd, opts, cb)`
to `execFile(file, args, opts, cb)`.
Validation for this half: `tsc --noEmit` clean on every touched file; a
jest run over `src/utils/formatters`, `src/command-line/format` and
`src/command-line/init` is identical to pristine `master` (the three
oxfmt-dependent suites fail on both).
## Related Issue(s)
Fixes #35722
Supersedes #35731, which reported and fixed the same bug. That branch
predates the move of `parseGitOutput` to an argv array, and it does not
cover the `"`/backslash class or the prettier spawn downstream. Its e2e
case is carried over here, with credit.
<!-- polygraph-session-start -->
---
<p><a
href="https://app.trypolygraph.com/orgs/6a061dcb561c062131116eca/sessions/Fix-nrwl-nx-issue-35722---UTF-8-path-detection-with-git-19db98e8">View
Polygraph session ↗</a></p>
<!-- polygraph-session-end -->
---------
Co-authored-by: nx-cloud[bot] <71083854+nx-cloud[bot]@users.noreply.github.com>
Co-authored-by: Christina Huege <11138584+chrstnbrn@users.noreply.github.com> b616fa1d fix(core): detect changes in paths that git escapes (#36660)
## Current Behavior
When a changed file lives under a directory whose name contains
non-ASCII characters, affected detection silently finds nothing.
Changing `apps/cart/src/app/Einkäufe/test.tsx` and running `nx show
projects --affected` returns no projects.
Git C-quotes any path containing non-ASCII bytes when `core.quotepath`
is enabled, which is the default:
```
"apps/cart/src/app/Eink\303\244ufe/test.tsx" <- umlaut path, quoted and octal-escaped
apps/cart/src/app/plain/test.tsx <- ASCII path, untouched
```
`parseGitOutput` split that output on newlines and trimmed each entry,
so the quoted literal was passed downstream unchanged. It matched no
project root, so nothing was reported as affected.
The reporter noted that setting `git config core.quotepath false` works
around it, but Nx should not depend on that setting.
## Expected Behavior
Affected detection works regardless of `core.quotepath`, and `cart` /
`cart-e2e` are reported as affected.
This passes `-z` to the three git readers (`getUncommittedFiles`,
`getUntrackedFiles`, `getFilesUsingBaseAndHead`) and splits
`parseGitOutput` on NUL instead of newline.
### Why `-z` rather than `core.quotepath=false`
`git -c core.quotepath=false` only suppresses escaping of non-ASCII
bytes. Paths containing `"` or `\` are still C-quoted:
```
$ git -c core.quotepath=false diff --name-only HEAD
"apps/cart/src/app/back\\slash/test.tsx"
"apps/cart/src/app/quo\"te/test.tsx"
$ git diff --name-only -z HEAD | tr '\0' '\n'
apps/cart/src/app/back\slash/test.tsx
apps/cart/src/app/quo"te/test.tsx
```
So `core.quotepath=false` would have fixed umlauts while leaving a
second, subtler bug in place. `-z` disables path quoting entirely.
Two details worth flagging for review:
- `-z` **must precede non-option arguments** for `git diff` (otherwise
`fatal: option '-z' must come before non-option arguments`), so it is
placed per-call-site rather than centralised in `parseGitOutput`.
- Git emits a trailing NUL, so the empty-entry filter is retained.
Dropping the now-redundant `.trim()` additionally fixes paths with
leading or trailing spaces, which the previous code silently corrupted.
All consumers of the `parseFiles` seam benefit: `show projects
--affected`, `affected`, `release plan`/`plan-check`, and `format`.
## Tests
Adds `packages/nx/src/utils/command-line-utils.real.spec.ts`, following
the existing `.real.spec.ts` convention of exercising **real temporary
git repositories** rather than mocking `child_process`. This was
deliberate — a mocked test cannot catch the `-z` argument-order
constraint above. Four cases (uncommitted, untracked, base/head, and
paths containing `"` / trailing spaces); all four fail before the fix
and pass after. The quote/trailing-space case is skipped on Windows,
where those characters are not legal in a path segment.
Validation: `nx lint nx` passes, prettier clean, and a full jest run
over `src/utils`, `src/command-line/affected`, `src/command-line/show`,
and `src/project-graph/file-utils` is identical to pristine `master`
apart from the 4 new passing tests.
Also ports the e2e case from #35731 into
`e2e/nx/src/affected-graph.test.ts`. The unit specs stop at
`parseFiles`, so nothing here covered the step where the reported
symptom actually surfaced, matching a changed path against a project
root.
## Also in this PR: prettier is spawned with argv, not a shell string
Reading paths with `-z` makes a second, pre-existing bug **reachable**,
so it is fixed here rather than deferred.
`writeWithPrettier` / `checkWithPrettier` interpolated each pattern into
a command string for `execSync`/`exec`, quoting it with `quoteForShell`.
That quoting is complete on POSIX but a no-op on Windows:
```ts
const escaped =
process.platform !== 'win32' ? pattern.replace(/([\\"`$])/g, '\\$1') : pattern;
return `"${escaped}"`;
```
On Windows a path containing `"` closes the quoting and the rest of the
path is parsed as shell. Before `-z`, git handed those paths back
C-quoted and the `fileExists` filter dropped them, so the hole was
unreachable. It is reachable now.
Both entry points now spawn with `execFileSync`/`execFile` and an argv
array, removing the shell entirely along with `quoteForShell` and the
escaping it needed. Nothing globbed through the shell before — patterns
were quoted, so the shell never expanded them, and prettier does its own
globbing.
### Why `chunkify`'s `measure` parameter is deleted
This is the change most likely to read as unrelated, so to be explicit:
`measure` existed **only** to compensate for the quoting removed above,
and has no other caller in the repo.
`chunkify` splits the file list into batches that fit under the OS
command-line limit. Before oxfmt, `format.ts` quoted patterns *before*
chunking, so the array held the exact strings that would reach the
command line and the sizing was correct for free.
[#35089](https://github.com/nrwl/nx/pull/35089) (built-in oxfmt support)
had to stop pre-quoting, because oxfmt spawns with `execFile` and needs
raw paths. Prettier's quoting moved down to its spawn site, which
inverted the order — chunk first, quote later — leaving `chunkify`
sizing `apps/foo.ts` while the command line would carry `"apps/foo.ts"`,
silently eating the headroom the budget reserves. `measure` was added in
that same commit so the caller could say "size against the quoted
length".
With prettier on `execFile` too, both consumers take paths as-is,
on-the-wire length is raw length again, and there is nothing left to
compensate for. So `measure` and its three spec cases are removed rather
than left as dead API surface.
### Tests for this half
- `quote-for-shell.spec.ts` is **deleted** — it asserted a shell-quoting
property that no longer exists.
- `prettier-argv.spec.ts` **replaces** that coverage: it asserts paths
containing `"`, backticks, `$(…)`, `$HOME` and `\$` reach prettier as
literal argv entries. Both cases fail if the quoting is reintroduced
(verified by mutation) and pass without it.
- `check-with-prettier.spec.ts`'s mock moves from `exec(cmd, opts, cb)`
to `execFile(file, args, opts, cb)`.
Validation for this half: `tsc --noEmit` clean on every touched file; a
jest run over `src/utils/formatters`, `src/command-line/format` and
`src/command-line/init` is identical to pristine `master` (the three
oxfmt-dependent suites fail on both).
## Related Issue(s)
Fixes #35722
Supersedes #35731, which reported and fixed the same bug. That branch
predates the move of `parseGitOutput` to an argv array, and it does not
cover the `"`/backslash class or the prettier spawn downstream. Its e2e
case is carried over here, with credit.
<!-- polygraph-session-start -->
---
<p><a
href="https://app.trypolygraph.com/orgs/6a061dcb561c062131116eca/sessions/Fix-nrwl-nx-issue-35722---UTF-8-path-detection-with-git-19db98e8">View
Polygraph session ↗</a></p>
<!-- polygraph-session-end -->
---------
Co-authored-by: nx-cloud[bot] <71083854+nx-cloud[bot]@users.noreply.github.com>
Co-authored-by: Christina Huege <11138584+chrstnbrn@users.noreply.github.com> 29200561 feat(core): share one workspace walk across processes through the files archive (#36980)
## Current Behavior
Every process that constructs a `WorkspaceContext` walks and hashes the
whole workspace. Two things pay for that more than once per graph build:
- **Plugin workers.** Any `createNodes` that hashes through the
workspace context (every first-party plugin, via
`calculateHashForCreateNodes`) builds its own context in its own process
and walks again. The host has already walked and written `nx_files.nxt`
by the time it sends the first hook. Measured on ocean CI
(`ubuntu-latest`, 12.4k files), that was 9 to 15 concurrent walks per
graph build at 1.1 to 1.8 s each.
- **Parallel hosts.** Two daemonless `nx` runs overlapping in one
checkout both walk. The daemonless graph cache already shares through
`project-graph.lock`; the walk underneath it does not.
The walk itself is the block behind NXC-4932: the host's `multiGlob`
waits synchronously on the walker, and on a small runner the hasher runs
two threads, so a cold walk is 4 to 9 seconds on a fresh runner and over
20 under disk contention.
## Expected Behavior
One walk per checkout at a time, and none in plugin workers.
- The walk runs under `nx_files.lock`, beside the archive, using
`FileLock` (which gains `try_lock` and `wait_blocking` for Rust callers
on a plain thread; the napi surface is unchanged). A process that finds
the lock held waits for the holder, then loads the archive the holder
wrote. It trusts that archive only if it was written after the wait
began, so a holder that died mid-walk leaves the waiter to walk itself,
the same fallthrough `project-graph.ts` has for a stale graph cache. The
wait is bounded, 60 seconds by default and `NX_WORKSPACE_WALK_WAIT_MS`
to change it (capped at an hour, `0` never waits), and the bound covers
the whole `acquire_files` call rather than each attempt. That default is
longer than any walk measured so far. A holder that outlasts it
(suspended, on a stalled filesystem, or walking a workspace that takes
longer than a minute) costs the waiter the whole wait and then a walk of
its own, so a workspace whose walk is legitimately that long should
raise it. Staging files are named with a random part beside the pid and
opened with `create_new`, and any left behind by a writer that died
mid-write are swept on the next write.
- `WorkspaceContext.fromArchive(root, cacheDir)` loads the archive
without walking, waiting for an in-progress walk first if there is one.
It walks only when there is no archive at all, so it cannot fail where
`new` succeeds.
- Plugin workers use `fromArchive`. Their host cannot send a hook before
its walk finished, because the file list it passes came from that walk,
so the archive is always there.
- The archive is written to a staging file and renamed into place, so
nothing that trusts it can read a partial one.
Reading the archive no longer deserializes it. `FilesArchive` keeps the
validated bytes (in an `AlignedVec`) and serves lookups and iteration in
place through rkyv's archived types: the selective hash looks each
walked file up in the view, and a process that loads the archive copies
each string once, straight into the sorted list. The owned map now
exists only as what a walk produces and the writer serializes, and the
walk drains it into the list rather than cloning it. Peak memory on load
is the archive bytes plus the list, not bytes plus a map plus the list.
Measured on the 150k-file fixture (10.9 MB archive), one fresh process
per run, five runs, median, against a 59 MB baseline for node plus the
binding. "before" is the archive code as of the second commit
(deserialize into a map, then clone into the list); "after" is this
branch. The load is forced with a glob that matches nothing, so the
numbers are the Rust-side footprint and do not include copying the file
list into V8:
| path | before | after |
| --- | --- | --- |
| load from archive (`fromArchive`, what workers and waiting hosts do) |
58 ms, 114 MB | 48 ms, 106 MB |
| selective-hash walk with the archive present (what a warm host does) |
313 ms, 151 MB | 288 ms, 138 MB |
| load, then `allFileData()` into V8 | 116 ms, 140 MB | 102 ms, 140 MB |
The archive path was never the large allocation: the last row shows the
V8 copy of the file list sets the process peak either way. What changed
is the part this PR touches, 15 percent less memory and 17 percent less
time on load, 14 percent less memory and 8 percent less time on the
walk.
A run that starts after the holder released still walks, with the
existing selective hash, since it has to detect changes. The daemon
keeps its existing RPC path for workers and is unaffected beyond taking
the lock during its own startup walk.
Measured on a 150k-file fixture with 25 plugin workers that hash through
the context, cold (no archive) on every run, two runs each, same
machine. The middle row is the published JS on the new binding, which
isolates the lock from the worker switch:
| configuration | processes that walked | wall time | slowest worker
walk |
| --- | --- | --- | --- |
| baseline (published binding and JS) | 26 | 9.7 s, 10.5 s | 3.7 s, 3.6
s |
| lock only (this binding, published JS) | 2 | 7.3 s, 6.2 s | 0.20 s,
0.18 s |
| lock and `fromArchive` (this PR) | 1 | 5.9 s, 5.6 s | none |
In the lock-only row the 24 other workers found the lock held, waited,
and loaded the archive the one walking worker wrote. Two hosts started
cold together: one walks (hash 4.5 s), the other reports `loaded ...
archive another process wrote`. A warm host still walks.
## Whole-command numbers, master vs this branch
[nx-graph-perf](https://github.com/AgentEnder/nx-graph-perf) on the same
150k-file fixture, `nx show projects --json`, five cycles per arm, both
arms built with the same toolchain. Master is 9e9dbb256b, this branch's
merge base, so the five commits here are the only difference. The tool
forces the daemon on, so a `--no-daemon` flag was added for the
CI-shaped pair. Ranges are in brackets.
Daemonless:
| phase | master | branch | delta |
| --- | --- | --- | --- |
| cold | 9.77 s [9.5 to 13.3] | 7.12 s [6.2 to 9.9] | -27% |
| warm | 5.13 s [5.0 to 5.4] | 1.81 s [1.8 to 2.0] | -65% |
| semi-warm | 5.20 s [5.1 to 5.8] | 1.88 s [1.8 to 2.1] | -64% |
Daemon on:
| phase | master | branch | delta |
| --- | --- | --- | --- |
| cold | 8.07 s [7.8 to 8.7] | 8.71 s [6.4 to 10.0] | +8% |
| warm | 258 ms | 256 ms | 0% |
| semi-warm | 1.89 s [1.7 to 2.1] | 1.65 s [1.0 to 1.8] | -13% |
The daemon cold ranges overlap fully and the daemon's own graph build is
3% faster inside those runs, so that reads as no change. Workers already
hash through the daemon's RPC in that mode and never build a local
context.
The daemonless warm gap is the walk count. The native trace for one warm
run shows master doing 26 walks (host plus 25 workers) and the branch
doing 1 walk plus 25 archive reads, and both arms produce the same
751-node graph. The host walks the tree alone in 160 ms. Each worker's
walk on master takes 3 s, because 25 of them run at once against one
directory tree, each with its own walker thread pool and its own
150k-entry file list. That is 3.6 s of every worker's `createNodes`
spent waiting on its own walk, against 0.39 s on the branch. Real
plugins do more than the walk inside `createNodes`, so the share is
smaller on a real repo; the ocean CI trace in NXC-4932 recorded 9 to 15
concurrent worker walks at 1.1 to 1.8 s each on a 12k-file tree, against
a natural single walk of 0.3 s.
Parallel hosts, the lock's own case. Three `nx show projects --affected
--json` clients started together, daemonless, three cycles, slowest
client per run. `--affected` is needed because plain `show projects`
clients queue on the project-graph lock and only one builds; with it,
each client loads every plugin in-process and spawns its own workers.
| phase | master | branch | delta |
| --- | --- | --- | --- |
| cold | 16.15 s [16.1 to 17.9] | 9.62 s [9.6 to 10.9] | -40% |
| warm | 7.83 s [7.8 to 8.3] | 4.28 s [4.2 to 4.8] | -45% |
| semi-warm | 7.94 s [7.9 to 9.7] | 4.71 s [4.3 to 4.8] | -41% |
The fastest-to-slowest client spread is about 1.5 s on both arms, the
graph-lock queue, which this PR leaves alone. What changes is what the
second and third clients do after it: on master their workers walk the
tree again, on the branch they read the archive the first client's walk
wrote.
## The host's own walk
The lock and `fromArchive` take the walks out of the workers, but the
host still walked with its event loop frozen, and that freeze is the
mechanism in NXC-4932. The last commit fixes it on the JS side.
`WorkspaceContext.ready()` is a napi `AsyncTask`. It waits on the
walker's condvar from a libuv thread and resolves once the file list
exists. The seven async readers in `workspace-context.ts`
(`getNxWorkspaceFilesFromContext`, `multiGlobWithWorkspaceContext`,
`hashWithWorkspaceContext` and the rest) await it before they call the
native method, so a walk in progress yields the event loop instead of
holding it. The sync readers still block, since they have no caller to
yield to.
`getPluginsSeparated` calls `startWorkspaceContext(root)` before it
spawns a worker, so the walk starts first and overlaps worker startup.
It does nothing when the daemon owns the context.
Measured on the same fixture, cold, with 30 worker processes, two runs
each. Each cell is how long a worker waited between listening and
receiving `load`:
| | p50 | p90 | max | workers over 200 ms |
| --- | --- | --- | --- | --- |
| before this commit | 31 ms, 16 ms | 4164 ms, 73 ms | 4186 ms, 4493 ms
| 5, 3 |
| after | 19 ms, 11 ms | 59 ms, 49 ms | 134 ms, 144 ms | 0, 0 |
The host's walk was 4.3 to 5.0 s in all four runs. Before, a worker that
connected during it waited for the whole walk. After, none waited longer
than 144 ms.
## Related Issue(s)
NXC-4932. Follow-up to #36977; the hasher's thread count is a separate
change.
---------
Co-authored-by: FrozenPandaz <jasonjean1993@gmail.com>
Co-authored-by: nx-cloud[bot] <71083854+nx-cloud[bot]@users.noreply.github.com> 29200561 feat(core): share one workspace walk across processes through the files archive (#36980)
## Current Behavior
Every process that constructs a `WorkspaceContext` walks and hashes the
whole workspace. Two things pay for that more than once per graph build:
- **Plugin workers.** Any `createNodes` that hashes through the
workspace context (every first-party plugin, via
`calculateHashForCreateNodes`) builds its own context in its own process
and walks again. The host has already walked and written `nx_files.nxt`
by the time it sends the first hook. Measured on ocean CI
(`ubuntu-latest`, 12.4k files), that was 9 to 15 concurrent walks per
graph build at 1.1 to 1.8 s each.
- **Parallel hosts.** Two daemonless `nx` runs overlapping in one
checkout both walk. The daemonless graph cache already shares through
`project-graph.lock`; the walk underneath it does not.
The walk itself is the block behind NXC-4932: the host's `multiGlob`
waits synchronously on the walker, and on a small runner the hasher runs
two threads, so a cold walk is 4 to 9 seconds on a fresh runner and over
20 under disk contention.
## Expected Behavior
One walk per checkout at a time, and none in plugin workers.
- The walk runs under `nx_files.lock`, beside the archive, using
`FileLock` (which gains `try_lock` and `wait_blocking` for Rust callers
on a plain thread; the napi surface is unchanged). A process that finds
the lock held waits for the holder, then loads the archive the holder
wrote. It trusts that archive only if it was written after the wait
began, so a holder that died mid-walk leaves the waiter to walk itself,
the same fallthrough `project-graph.ts` has for a stale graph cache. The
wait is bounded, 60 seconds by default and `NX_WORKSPACE_WALK_WAIT_MS`
to change it (capped at an hour, `0` never waits), and the bound covers
the whole `acquire_files` call rather than each attempt. That default is
longer than any walk measured so far. A holder that outlasts it
(suspended, on a stalled filesystem, or walking a workspace that takes
longer than a minute) costs the waiter the whole wait and then a walk of
its own, so a workspace whose walk is legitimately that long should
raise it. Staging files are named with a random part beside the pid and
opened with `create_new`, and any left behind by a writer that died
mid-write are swept on the next write.
- `WorkspaceContext.fromArchive(root, cacheDir)` loads the archive
without walking, waiting for an in-progress walk first if there is one.
It walks only when there is no archive at all, so it cannot fail where
`new` succeeds.
- Plugin workers use `fromArchive`. Their host cannot send a hook before
its walk finished, because the file list it passes came from that walk,
so the archive is always there.
- The archive is written to a staging file and renamed into place, so
nothing that trusts it can read a partial one.
Reading the archive no longer deserializes it. `FilesArchive` keeps the
validated bytes (in an `AlignedVec`) and serves lookups and iteration in
place through rkyv's archived types: the selective hash looks each
walked file up in the view, and a process that loads the archive copies
each string once, straight into the sorted list. The owned map now
exists only as what a walk produces and the writer serializes, and the
walk drains it into the list rather than cloning it. Peak memory on load
is the archive bytes plus the list, not bytes plus a map plus the list.
Measured on the 150k-file fixture (10.9 MB archive), one fresh process
per run, five runs, median, against a 59 MB baseline for node plus the
binding. "before" is the archive code as of the second commit
(deserialize into a map, then clone into the list); "after" is this
branch. The load is forced with a glob that matches nothing, so the
numbers are the Rust-side footprint and do not include copying the file
list into V8:
| path | before | after |
| --- | --- | --- |
| load from archive (`fromArchive`, what workers and waiting hosts do) |
58 ms, 114 MB | 48 ms, 106 MB |
| selective-hash walk with the archive present (what a warm host does) |
313 ms, 151 MB | 288 ms, 138 MB |
| load, then `allFileData()` into V8 | 116 ms, 140 MB | 102 ms, 140 MB |
The archive path was never the large allocation: the last row shows the
V8 copy of the file list sets the process peak either way. What changed
is the part this PR touches, 15 percent less memory and 17 percent less
time on load, 14 percent less memory and 8 percent less time on the
walk.
A run that starts after the holder released still walks, with the
existing selective hash, since it has to detect changes. The daemon
keeps its existing RPC path for workers and is unaffected beyond taking
the lock during its own startup walk.
Measured on a 150k-file fixture with 25 plugin workers that hash through
the context, cold (no archive) on every run, two runs each, same
machine. The middle row is the published JS on the new binding, which
isolates the lock from the worker switch:
| configuration | processes that walked | wall time | slowest worker
walk |
| --- | --- | --- | --- |
| baseline (published binding and JS) | 26 | 9.7 s, 10.5 s | 3.7 s, 3.6
s |
| lock only (this binding, published JS) | 2 | 7.3 s, 6.2 s | 0.20 s,
0.18 s |
| lock and `fromArchive` (this PR) | 1 | 5.9 s, 5.6 s | none |
In the lock-only row the 24 other workers found the lock held, waited,
and loaded the archive the one walking worker wrote. Two hosts started
cold together: one walks (hash 4.5 s), the other reports `loaded ...
archive another process wrote`. A warm host still walks.
## Whole-command numbers, master vs this branch
[nx-graph-perf](https://github.com/AgentEnder/nx-graph-perf) on the same
150k-file fixture, `nx show projects --json`, five cycles per arm, both
arms built with the same toolchain. Master is 9e9dbb256b, this branch's
merge base, so the five commits here are the only difference. The tool
forces the daemon on, so a `--no-daemon` flag was added for the
CI-shaped pair. Ranges are in brackets.
Daemonless:
| phase | master | branch | delta |
| --- | --- | --- | --- |
| cold | 9.77 s [9.5 to 13.3] | 7.12 s [6.2 to 9.9] | -27% |
| warm | 5.13 s [5.0 to 5.4] | 1.81 s [1.8 to 2.0] | -65% |
| semi-warm | 5.20 s [5.1 to 5.8] | 1.88 s [1.8 to 2.1] | -64% |
Daemon on:
| phase | master | branch | delta |
| --- | --- | --- | --- |
| cold | 8.07 s [7.8 to 8.7] | 8.71 s [6.4 to 10.0] | +8% |
| warm | 258 ms | 256 ms | 0% |
| semi-warm | 1.89 s [1.7 to 2.1] | 1.65 s [1.0 to 1.8] | -13% |
The daemon cold ranges overlap fully and the daemon's own graph build is
3% faster inside those runs, so that reads as no change. Workers already
hash through the daemon's RPC in that mode and never build a local
context.
The daemonless warm gap is the walk count. The native trace for one warm
run shows master doing 26 walks (host plus 25 workers) and the branch
doing 1 walk plus 25 archive reads, and both arms produce the same
751-node graph. The host walks the tree alone in 160 ms. Each worker's
walk on master takes 3 s, because 25 of them run at once against one
directory tree, each with its own walker thread pool and its own
150k-entry file list. That is 3.6 s of every worker's `createNodes`
spent waiting on its own walk, against 0.39 s on the branch. Real
plugins do more than the walk inside `createNodes`, so the share is
smaller on a real repo; the ocean CI trace in NXC-4932 recorded 9 to 15
concurrent worker walks at 1.1 to 1.8 s each on a 12k-file tree, against
a natural single walk of 0.3 s.
Parallel hosts, the lock's own case. Three `nx show projects --affected
--json` clients started together, daemonless, three cycles, slowest
client per run. `--affected` is needed because plain `show projects`
clients queue on the project-graph lock and only one builds; with it,
each client loads every plugin in-process and spawns its own workers.
| phase | master | branch | delta |
| --- | --- | --- | --- |
| cold | 16.15 s [16.1 to 17.9] | 9.62 s [9.6 to 10.9] | -40% |
| warm | 7.83 s [7.8 to 8.3] | 4.28 s [4.2 to 4.8] | -45% |
| semi-warm | 7.94 s [7.9 to 9.7] | 4.71 s [4.3 to 4.8] | -41% |
The fastest-to-slowest client spread is about 1.5 s on both arms, the
graph-lock queue, which this PR leaves alone. What changes is what the
second and third clients do after it: on master their workers walk the
tree again, on the branch they read the archive the first client's walk
wrote.
## The host's own walk
The lock and `fromArchive` take the walks out of the workers, but the
host still walked with its event loop frozen, and that freeze is the
mechanism in NXC-4932. The last commit fixes it on the JS side.
`WorkspaceContext.ready()` is a napi `AsyncTask`. It waits on the
walker's condvar from a libuv thread and resolves once the file list
exists. The seven async readers in `workspace-context.ts`
(`getNxWorkspaceFilesFromContext`, `multiGlobWithWorkspaceContext`,
`hashWithWorkspaceContext` and the rest) await it before they call the
native method, so a walk in progress yields the event loop instead of
holding it. The sync readers still block, since they have no caller to
yield to.
`getPluginsSeparated` calls `startWorkspaceContext(root)` before it
spawns a worker, so the walk starts first and overlaps worker startup.
It does nothing when the daemon owns the context.
Measured on the same fixture, cold, with 30 worker processes, two runs
each. Each cell is how long a worker waited between listening and
receiving `load`:
| | p50 | p90 | max | workers over 200 ms |
| --- | --- | --- | --- | --- |
| before this commit | 31 ms, 16 ms | 4164 ms, 73 ms | 4186 ms, 4493 ms
| 5, 3 |
| after | 19 ms, 11 ms | 59 ms, 49 ms | 134 ms, 144 ms | 0, 0 |
The host's walk was 4.3 to 5.0 s in all four runs. Before, a worker that
connected during it waited for the whole walk. After, none waited longer
than 144 ms.
## Related Issue(s)
NXC-4932. Follow-up to #36977; the hasher's thread count is a separate
change.
---------
Co-authored-by: FrozenPandaz <jasonjean1993@gmail.com>
Co-authored-by: nx-cloud[bot] <71083854+nx-cloud[bot]@users.noreply.github.com>