Overview - Nx

Nx

7 days30 days

Latest CI Pipeline Executions

Fuzzy

Failed
35172
918e368c chore(core): refactor init task runner"
1 hour ago by FrozenPandaz
Canceled
35172
69e5a543 chore(core): refactor init task runner"
1 hour ago by FrozenPandaz
Failed
35172
fbd56aab fix(core): bulk cache resolve in runDiscreteTasks with inline miss ids Do the bulk cache fetch for all discrete tasks in one resolveCachedTasks call, then fall through to runTaskDirectly for misses with inline groupId++ at the dispatch site. Hits don't consume a groupId slot — only tasks that actually execute do. GroupId accounting: - Batches: 0..N-1 (one per batch, unchanged) - Cache-hit batch: N (shared lifecycle id for all hits) - Miss tasks: N+1, N+2… (one per miss, inline at dispatch) The inline groupId++ inside the .then() callback is safe — JS increments are atomic within a microtask, so each miss gets a unique integer. Assignment order depends on scheduling but every id is unique, which is all lifecycle handlers need.
2 hours ago by FrozenPandaz
Canceled
35172
b1e42035 fix(core): preserve one-groupId-per-task in runDiscreteTasks My previous refactor introduced an extra groupId for a bulk cache-resolve phase, shifting every discrete task's id by one and changing the semantic from "each task has its own groupId" to "cache hits share one id, misses get individual ids." The bulk- resolve pretense wasn't buying much for runDiscreteTasks either — the programmatic API is typically called with a handful of tasks, not a thousand. Back to the original allocation: batches use groupId++, each discrete task uses groupId++, final counter = batches + discrete count. Matches the original code exactly. Per-task flow still uses the new split primitives internally — resolveCachedTasks for the check, runTaskDirectly for the run — but each task drives its own cache-or-run cycle under its own groupId, same as the old applyFromCacheOrRunTask behavior. The groupId allocation happens synchronously inside the .map() callback (not inside the async IIFE body), so each task's taskGroupId is stable and matches iteration order.
3 hours ago by FrozenPandaz
Canceled
35172
97f7a22b fix(core): revert closeGroup throw and overflow counter Undoing my own overengineering. The "throw on exhaustion" sanity check I added in an earlier cleanup was wrong — it assumed an invariant that doesn't hold — and the overflow-counter fix I followed up with was a solution to a problem I created. Back to the original shape: scan for a free slot, return its index, fall through to implicit undefined if nothing's free. The callers that matter (step 5 worker dispatch) are gated on `inFlightWorkers < parallelism` and never hit exhaustion. Other callers (cache-hit resolution, batch executor) tolerate undefined groupIds fine because lifecycle handlers treat groupId as opaque metadata. Fixes the "closeGroup called with no free slot" crash without the baggage of a second counter or type gymnastics.
3 hours ago by FrozenPandaz
Canceled
35172
b05b2691 fix(core): allow closeGroup to overflow when slots are claimed The "no free slot" throw I added as a sanity check was wrong. I assumed the coordinator always gates closeGroup on inFlightWorkers < parallelism, but that's only true for step 5 worker dispatch. Steps 2 (resolveCachedTasksBulk) and 4 (batch executor) also call closeGroup, and both can run when all parallelism slots are already claimed by in-flight workers. Concretely: worker completes → postRunSteps → scheduleNextTasksAndReleaseThreads wakes the coordinator BEFORE the worker's .finally() decrements inFlightWorkers and releases its slot. Coordinator then runs resolveCachedTasksBulk, calls closeGroup — all 8 slots still marked taken — throws "closeGroup called with no free slot (parallel=8)". Cache-hit resolution doesn't actually need a worker slot — it doesn't spawn a child process, just fires lifecycle. Same for batch executor fallback. These paths need a unique groupId for metadata routing, not a reservation in the worker pool. Fix: when the slot pool is exhausted, hand out an overflow id from a counter that counts down from Number.MAX_SAFE_INTEGER. Overflow ids are guaranteed outside both the worker slot range (0..parallel) and the continuous id range (parallel..parallel+n). openGroup is a no-op for overflow ids so they don't corrupt the pool. Step 5 worker dispatch still gates correctly and never reaches the overflow path, so workers still get strict slot reservation. Reported via the "closeGroup called with no free slot (parallel=8)" crash on a warm-cache run.
3 hours ago by FrozenPandaz
Canceled
35172
8d40565b cleanup(core): expose resolveCachedTasks as a primitive, drop bulk wrapper Follow-up to splitting cache check from task execution: instead of having a compound applyFromCacheOrRunTasks that hides "check cache, then run misses" behind one call, expose the two orthogonal primitives and let callers compose them. - resolveCachedTasks(tasks, groupId, doNotSkipCache): new public method. Filters to cacheable, bulk-fetches hits, fires scheduleTask lifecycle for hits not yet processed (coordinator gap-filler), runs preRunSteps + finalizeCacheHits + postRunSteps under the provided groupId, returns the hit results. Caller is responsible for running whatever wasn't hit. - runTaskDirectly: unchanged, still the "no cache check, just spawn and wait" primitive. - resolveCachedTasksBulk (coordinator private): shrunk to a thin wrapper — peek scheduler, filter to hashed discrete cacheable candidates, closeGroup/openGroup, delegate to resolveCachedTasks. - applyFromCacheOrRunTasks (the bulk wrapper I just added): deleted. init-tasks-runner composes resolveCachedTasks + runTaskDirectly directly. Each discrete task gets its own Promise<TaskResult[]> in the returned array, preserving the old runDiscreteTasks return shape exactly (one promise per task, not one bulk promise). Also drop the "set startTime = endTime = now before preRunSteps" block that was living in resolveCachedTasksBulk. Its comment claimed it "prevents negative durations in TUI" but preRunSteps unconditionally overrides startTime to a later now, producing endTime < startTime — i.e. the block was causing the exact bug its comment claimed to prevent. Removing it gives tiny positive durations for instant cache hits, which is correct TUI behavior.
3 hours ago by FrozenPandaz
Canceled
35172
7891a7e0 fix(core): split cache check from task execution applyFromCacheOrRunTask was doing the cache check inside a function whose callers had already done it once. On Nx Cloud workloads this meant two remote HTTP retrievals per confirmed miss — the first from the coordinator's resolveCachedTasksBulk, the second from applyFromCacheOrRunTask's own check. The fix is to make the cache-or-run contract explicit at the API level instead of hacking a skip flag. Three changes: 1. runTaskDirectly: new method that spawns and waits on a task's child process without any cache lookup. Handles env setup, preRunSteps, child spawn, result collection, postRunSteps, and the discreteTaskExitHandled bookkeeping. 2. applyFromCacheOrRunTasks (plural): new bulk entry point for callers that don't drive the coordinator loop themselves. Bulk-resolves cache hits in one shot, fires lifecycle around them under a shared groupId, and runs confirmed misses in parallel via runTaskDirectly with distinct groupIds. Replaces the per-task applyFromCacheOrRunTask (singular) that runDiscreteTasks was calling in a loop. 3. Coordinator step 5 now dispatches runTaskDirectly. To make this safe, resolveCachedTasksBulk's `<= 1` early-return bails are removed — it unconditionally checks the cache for every cacheable scheduled task when caching is enabled, guaranteeing the miss invariant that step 5 relies on. runDiscreteTasks (programmatic API) now benefits from the same bulk cache resolution the coordinator loop already got — one SQL call and one batch of parallel remote retrievals instead of N.
3 hours ago by FrozenPandaz
Canceled
35172
f5037265 cleanup(core): rewrite coordinator worker dispatch as try/catch/finally The fire-and-forget worker dispatch in the coordinator's step 5 was built from .catch(async e => ...).finally(...) on an unawaited Promise. Same semantics, but the async-catch pattern reads awkwardly and the final finally is easy to miss in a chain. Wrap the body in an async IIFE with linear try/catch/finally — identical control flow and closure captures, no behavior change. The `void` prefix signals intentional no-await for no-floating-promises lint rules.
3 hours ago by FrozenPandaz
Canceled
35172
a9e25064 cleanup(core): dedupe cache resolution + misc orchestrator cleanups Several small cleanups to task-orchestrator.ts: 1. Extract fetchCacheHits() and finalizeCacheHits() helpers. applyCachedResults and resolveCachedTasksBulk had ~60 lines of line-for-line duplicated logic: batch cache lookup + filter to successful entries, shouldCopyOutputsFromCache batch, copy outputs in parallel, derive status, print terminal output, build results. Both now share one CacheHit-tuple type and the two helpers. Also upgrades applyCachedResults from sequential output copies (await in for-loop) to parallel Promise.all — matches what resolveCachedTasksBulk was already doing. 2. completedTasks: object-as-map -> Map<string, TaskStatus>. Eight read sites were a mix of `[id] !== undefined` comparisons and truthy checks that worked only because no TaskStatus value is falsy. Map.has()/Map.get() make the intent explicit and align with the other state maps already on the class (processedTasks, runningDiscreteTasks, etc.). The public run() return type stays as a plain object via Object.fromEntries so defaultTasksRunner's public API doesn't change. 3. closeGroup(): fix lying return type. The `for` loop fell through without a return when no slot was free, silently handing back `undefined as number`. The caller does pair open/close properly so this should be unreachable, but failing loudly beats a silent type lie. Also types the previously-inferred `never[]` groups field as `boolean[]`. 4. Inline processTasks() into processAllScheduledTasks(). The wrapper was called from exactly one place; the indirection didn't buy anything. 5. Rename `_dbCacheEnabled` local to `usingDbCache` (inside the extracted finalizeCacheHits helper). The underscore prefix was cargo-culted to avoid shadowing the imported function name, which isn't actually a collision in TypeScript.
4 hours ago by FrozenPandaz
Canceled
35172
16f93e3d cleanup(core): pass projects directly to getExecutorForTask getExecutorForTask and getCustomHasher used to take the whole ProjectGraph and derive the projects record internally via a WeakMap cache keyed by graph identity. The WeakMap was a clever workaround for the hot-path cost of re-running readProjectsConfigurationFromProjectGraph per call, but it meant module-level mutable state, hidden lifetime invariants, and one extra lookup per call. Flip the dependency: the caller computes projects once at a higher scope (it already has projectGraph) and passes the record through explicitly. Callers: - TaskOrchestrator now holds `projects` as a readonly field, initialized from projectGraph once in class-field init. - TasksSchedule takes `projects` as a new constructor param from the orchestrator that instantiates it. - hashTask and hashTasks already computed projectsConfigurations for the custom hasher context — reuse that instead of running the walk again via the WeakMap. - hashTasksThatDoNotDependOnOutputsOfOtherTasks now computes projects once at function entry. - show-target/utils.hasCustomHasher uses the existing top-level readProjectsConfigurationFromProjectGraph import (the runtime require was dead weight added by a previous cleanup step). Delete the projectsForExecutorCache WeakMap and its unused readProjectsConfigurationFromProjectGraph import from tasks-runner/utils.ts. Also a small stylistic cleanup in DbCache.getBatch — replace the index-based for loop with tasks.entries() so each task binding is named instead of re-indexed.
4 hours ago by FrozenPandaz
Canceled
35172
dd4ed0b8 cleanup(core): drop singular get_files_for_outputs napi export The singular version and the batch version had 70+ lines of line-for-line duplicated logic — file/dir/glob partitioning, glob-set walking, directory walking, sort. Extract the shared body into a regular pub Rust fn that takes &Path, and make the napi-exported batch version par_iter over it. The singular napi export had no JS callers anywhere in the codebase — only two internal Rust callers in hash_task_output.rs (resolve_task_output_files and hash_task_output), which now pass &Path directly. Dropping the #[napi] annotation shrinks the native bindings surface by one function. Regenerated native-bindings.js and index.d.ts via a fresh build-native nx run. The two WASI binding files were already stale on this branch (missing getFilesForOutputsBatch and all the other batch additions the PR introduced — the WASI target can't build on stable Rust due to a nightly feature gate) so I only stripped the newly-dead getFilesForOutputs line from each to keep them from referencing a removed symbol. The broader WASI staleness is pre-existing and outside the scope of this fixup.
4 hours ago by FrozenPandaz
Canceled
35172
220400c6 fix(core): route applyCachedResults through DbCache.getBatch applyCachedResults was doing N sequential-ish this.cache.get(task) calls inside Promise.all — each one crossing the JS/Rust boundary and doing its own primary-key SQL. Switch to a single getBatch call so the batch executor's per-level cache check becomes one rarray SQL lookup instead of N individual ones. To make this safe, extend DbCache.getBatch itself to fall through to the remote cache for local misses — previously it was local-only, which meant a naive switch would silently regress remote-cache hit rates (reporting remote-only hits as misses and re-running tasks). This also closes the same latent gap in resolveCachedTasksBulk, which was already using getBatch without remote support — remote hits there used to have to fall all the way through to the worker dispatch path before being resolved. Add a single-task fast path in getBatch that delegates to get() so the n=1 callers (applyFromCacheOrRunTask -> applyCachedResults([task])) keep the direct primary-key SQL without paying rarray vtable overhead. DbCache.getBatch is async again — but now for a good reason: it genuinely does async work (HTTP retrievals) when remote misses exist. The earlier sync revert was correct at the time when the wrapper had no async work to do.
4 hours ago by FrozenPandaz
Canceled
35172
7b19402c cleanup(core): reuse build_cached_result in single-row get() The single-hash get() path was inlining the same CachedResult construction and terminal-output read that build_cached_result already encapsulates. Reuse it. Side benefit: the file read now happens AFTER the DB lock is released. Previously the row closure ran inside query_row, which holds the mutex — so concurrent single-get callers were serialized on disk I/O, not just on the DB query itself. Kept the direct `WHERE hash = ?1` SQL instead of routing through fetch_cache_rows() — for a single hash, primary-key lookup beats single-element rarray + vtable overhead.
4 hours ago by FrozenPandaz
Canceled
35172
2c3be547 cleanup(core): split get_batch into fetch + build helpers get_batch was doing four things inline: rarray binding, SQL execution, row collection, and per-hit CachedResult assembly. Split it into two private helpers: - fetch_cache_rows: one SQL round-trip, returns hash -> (code, size) - build_cached_result: read terminal output, assemble CachedResult Top-level get_batch is now ~10 lines of readable orchestration. Also reuse the existing get_task_outputs_path_internal helper instead of rebuilding the path inline.
4 hours ago by FrozenPandaz
Canceled
35172
37af0a85 cleanup(core): collect get_batch rows straight into HashMap rusqlite's MappedRows is an iterator of Result<T>, and collecting into HashMap<K, V> works when the row closure returns (K, V). Drops the intermediate Vec<(String, i16, i64)> that only existed so a HashMap<&str, ...> could borrow from it — now we own the String as the map key directly. Also lets the par_iter lookup use db_map.get(hash) instead of db_map.get(hash.as_str()), since HashMap::get is generic over Borrow.
5 hours ago by FrozenPandaz
Canceled
35172
6a857329 fix(core): use rarray for NxCache.get_batch IN clause Instead of building N positional placeholders (?1, ?2, ...) per call, bind the whole Vec as a single rarray parameter. The prepared statement text is now constant regardless of batch size, so SQLite's statement cache hits forever and we sidestep the 999/32766 per-statement parameter cap. Matches the pattern already in use by TaskHistory.get_flaky_tasks and get_estimated_task_timings. The rusqlite `array` feature is already enabled in Cargo.toml and the module is loaded per connection in db/initialize.rs, so no plumbing needed.
5 hours ago by FrozenPandaz
Canceled
35172
a26c89e1 docs(core): comment defensive task.hash filter in hashTasks Explain why we filter on task.hash before recordTaskDetails even though every task in `tasks` should have been hashed by the time we get here — custom hashers are user code and can theoretically resolve with a falsy value, which would violate the task_details schema downstream.
5 hours ago by FrozenPandaz
Failed
35172
585fba1d cleanup(core): revert DbCache.getBatch to sync It wraps a sync native call, so there's no async work to await. The orchestrator's `await this.cache.getBatch(...)` call site still type-checks against the DbCache | Cache union — await on Map | Promise<Map> collapses to Map. Saves a microtask per coordinator iteration on the warm-cache hot path and is more honest about what DbCache actually does. Cache.getBatch stays async since it awaits this.get() calls for its per-task fallback loop.
5 hours ago by FrozenPandaz
Succeeded
35248
71fc05b0 cleanup(core): trace jsonc parse errors and drop redundant comment Route the jsonc_parser error through tracing::trace! via inspect_err so parse failures aren't lost when the raw-bytes fallback kicks in — useful for debugging a workspace where a "JSON" input silently ended up hashed as bytes. Drop the adjacent "Requires valid UTF-8" comment since the from_utf8 call already says that.
5 hours ago by FrozenPandaz