zebra 0.0.1-preview.1
Zebra: an incremental build orchestrator for Zena and standalone workspaces
zebra#
An incremental script and service runner for Zena (zena-package.json), standalone Zebra (zebra.json), and Wireit (package.json) workspaces, built on zena-lang:workflow.
Zebra runs a dependency graph of scripts and long-running services with declared inputs, outputs, and dependencies, rerunning only what changed. For each script, Zebra content-hashes the declared inputs and asks the workflow engine what to do: nothing (when the previous run recorded the same fingerprints and declared outputs are intact), restore from the local content-addressed cache, or execute the command (sh -c in the package directory with node_modules/.bin on PATH).
Installation#
zebra is built into the zena CLI (zena build <target>). To use it as a library or standalone runner:
zena pkg add zena-lang:zebra
CLI Usage#
zena build [-j <jobs>] [-w|--watch] [-f|--force] [--no-cache] [-v|--verbose] [<target>] [-- <args>...]
zena run [-j <jobs>] [-w|--watch] [-f|--force] [--no-cache] [-v|--verbose] <target> [-- <args>...]
zena test [-j <jobs>] [-w|--watch] [--force] [--no-cache] [-v|--verbose] [-- <args>...]
zena build --migrate [--dry-run] [-f|--force] [<package-dir>...]
<target>can be a script name in the current package (zena build test,zena run check), a package directory (zena build ./packages/app, which runsbuild), or<package-dir>:<script>(zena build ./packages/app:test). With no<target>,zena buildruns the current package'sbuildscript andzena testruns itstestscript.-- <args>...appends shell-quoted arguments to the entry target's command only, running that entry step uncached without overwriting its saved freshness record.-j/--jobs <n>bounds parallel command execution (defaulting toZEBRA_PARALLEL,WIREIT_PARALLEL, orZENA_AVAILABLE_PARALLELISM).-w/--watchwatches input globs, lockfiles, and package manifests for changes, re-running affected steps incrementally while keeping unchanged services alive in aServicePoolacross rebuilds.-f/--forcebypasses freshness and cached outputs to rerun scripts unconditionally.--no-cachedisables restoring outputs from.zebra/blobs/when a step is not fresh.-v/--verboseprints every step (includingfreshandskipped) and any captured stdout/stderr on the rail.--migrateconverts Wireit blocks inpackage.jsonintozena-package.json(when present) orzebra.json.
Programmatic Usage#
import { buildTarget, reportBuild } from 'zena-lang:zebra/runner';
import { ScriptRef } from 'zena-lang:zebra/wireit-config';
let outcome = await buildTarget('.', new ScriptRef('packages/app', 'build'), {
parallelism: 4,
});
let exitCode = reportBuild(outcome);
Configuration (zena-package.json and zebra.json)#
Zena packages configure scripts and filesets directly in zena-package.json. Non-Zena projects can use a standalone zebra.json (with an optional "workspace": {"packages": [...]} at the workspace root). Both support // and /* ... */ comments:
{
"name": "app",
"dependencies": {
"lib": {"path": "../lib"},
},
"filesets": {
"assets": ["assets/**"],
},
"scripts": {
"build": {
"command": "zena build src/main.zena -o out/app.wasm",
"output": ["out/app.wasm"],
"dependencies": ["source", "assets"],
},
"test": {
"type": "zena:test",
"tests": "test/*_test.zena",
},
"fmt": "zena fmt --check .",
},
}
- Declarative
zena:teststeps and defaulttesttarget: Scripts inzena-package.jsonandzebra.jsoncan set"type": "zena:test"(with optional"tests": true,"tests": "test/*_test.zena", or"tests": ["test/*_test.zena", ...]— defaulting to["test/**/*_test.zena", "tests/**/*_test.zena"]when omitted — and optional"debug": truefor-g) to run Zena tests in-process throughTestStepRunner. Test pattern globs merge intofiles, andzena-package.json"zena:test"steps automatically depend onsource. When azena-package.jsonomits atestscript (and no fallbackpackage.jsondefinestest), Zebra synthesizes a"type": "zena:test"target overtest/**/*_test.zenaandtests/**/*_test.zena. - Automatic
sourcefileset: Every package withzena-package.jsonexposes an automaticsourcefileset covering its library.zenafiles (from"source", defaulting to"src"and excluding*_test.zena,test/**, andtests/**),zena-package.json, any"wit"dependencies, and thesourcefilesets of all"path"dependencies. Scripts opt into it via"dependencies": ["source"](or"../pkg:source"). - Named
filesets: Entries accept either a glob array shorthand ("wit": ["wit/**/*.wit"]) or{"files": [...], "dependencies": [...]}and compose transitively without spawning commands. - Caching and tracking: Task scripts cache automatically whenever
files,output, or a fileset dependency is declared, defaultingoutputto[]when omitted. Setting"cache": falseopts a step out of freshness and caching while keeping itsfilesglobs active in--watchmode. Omittingfiles,output, and fileset dependencies on a command step leaves it untracked so it runs every time. String entries ("fmt": "zena fmt --check .") are shorthands for always-run commands. - Locks:
locksdefaults to["zena-lock.json"]and may be set at the package root or overridden per script ("locks": []disables lockfile hashing). - Services: Scripts with
"service": trueor"service": {"readyWhen": {"lineMatches": "..."}}start on demand when a dependent executes, wait for readiness when configured, are shared across dependents, and are stopped as a Unix process group (SIGTERMthenSIGKILL) when dependents finish. Invoking a service directly as the top-level target runs its dependencies and streams the service in the foreground. - Wireit fallback: When a package directory has no
zebra.jsonand no"scripts"block inzena-package.json, Zebra falls back topackage.jsonWireit scripts while still serving filesets fromzena-package.json. Dependencies can cross freely betweenzena-package.json,zebra.json, andpackage.jsonpackages.
Cache Layout (.zebra/)#
Each package's <pkg>/.zebra/ directory stores:
state.json— freshness records for that package's scripts (the last cache key and output manifest).cache/<key>.json— cache key to output manifest.blobs/— content-addressed output bytes (sha256).
The workspace root's .zebra/ directory stores:
hashes.json— the(mtime, size)tosha256file hash memo shared across packages (with a 20ms racy-timestamp guard).plans/<hash>.plan—stat-verified cachedScriptConfigclosures keyed by target entry.
A dependency contributes the digest of its outputs to dependents' cache keys rather than its input fingerprint, giving early cutoff when a dependency reruns and produces identical outputs.
GitHub Actions Remote Cache#
Setting ZEBRA_CACHE=github layers GitHubActionsCache behind the local .zebra/ DiskCache (TieredCache). On a local cache miss, Zebra queries the GitHub Actions v2 CacheService Twirp JSON API, verifies downloaded blob sha256 digests, populates <pkg>/.zebra/, and materializes outputs into the workspace. On a build step execution, Zebra writes <pkg>/.zebra/ first and uploads the step's entry to GitHub Actions Cache (ZEBRA_CACHE_READ_ONLY=1 disables uploads; ZEBRA_CACHE=none disables both local and remote caching).
In GitHub Actions workflows, run .github/actions/setup-zebra-cache before build steps so ACTIONS_RESULTS_URL and ACTIONS_RUNTIME_TOKEN are served to run: steps over a 127.0.0.1 token custodian (ZEBRA_GITHUB_CUSTODIAN_URL).
Libraries#
zebra(zebra/main) — CLI entry point (main).zebra/runner—buildTarget,watchTarget,BuildOptions,WatchOptions,reportBuild,reportMigration,formatBuildReport, andBuildOutcome.zebra/zebra-config—zena-package.jsonandzebra.jsonparsing (ZebraWorkspace), synthesizedsourcefilesets, and Wireit migration (migrateWorkspaceToZebra,convertPackageJsonToZebraJson,mergeWireitIntoZenaPackageJson).zebra/wireit-config—package.jsonWireit parsing (WireitWorkspace) and shared script configuration types (ScriptConfig,ScriptRef,DependencySpec,ServiceConfig,EnvBinding,ConfigError,refKey,resolveDependency).zebra/plan-cache—.zebra/plans/snapshot serialization andstat-verified warm plan loading (loadOrCollectConfigs,PlanLoadStats,ConfigProbe).zebra/builder— TranslatesScriptConfigclosures intoworkflowsteps and execution callbacks (planScripts,StepExecution,lockfileCandidates).zebra/disk-cache,zebra/github-cache, andzebra/state-json—.zebra/blob/manifest storage (DiskCache,TieredCache,stateDirOf), GitHub Actions remote caching (GitHubActionsCache,GitHubCacheConfig,CacheMode,resolveCacheMode,resolveGitHubCacheConfig), and JSON serialization for state, manifests, and hash memos.zebra/fs-walkandzebra/fs-util— Input glob discovery (FileHasher,expandGlobs,hashFile,validateGlobs,splitBase) and workspace path utilities.zebra/shell— POSIX shell argument quoting (shellQuote,appendShellArgs). Zebra's terminal output useszena-lang:terminal(packages/terminal): colors, the Clack-style rail layout and the spinner.
License#
MIT
import {Future} from 'zena:async';
import {getEnv} from 'zena:cli';
import { HashSet } from 'zena:collections';
import {div} from 'zena:math';
import {Process, checkSignal, runWith, spawnWith} from 'zena:process';
import {Regex} from 'zena:regex';
import {monotonic} from 'zena:time';
import {StringBuilder, Unmanaged, adopt, disown} from 'zena:core';
import {Fingerprint} from 'workflow/fingerprint';
import {Graph} from 'workflow/graph';
import {OutputEntry, OutputManifest} from 'workflow/manifest';
import {
InputRef,
LeafSet,
RunContext,
ServiceHandle,
ServiceStep,
Step,
StepKey,
StepOrder,
StepOutputs,
TaskStep,
} from 'workflow/step';
import {deleteIfExists, joinWorkspace, withRoot} from './fs-util.zena';
import {FileHasher, expandGlobs, hashFile, validateGlobs} from './fs-walk.zena';
import {sha256HexOfString} from 'zena:crypto';
import {appendShellArgs} from './shell.zena';
import {
ConfigError,
DependencySpec,
EnvBinding,
EnvExternal,
EnvLiteral,
ScriptConfig,
ZENA_TEST_RULE_TYPE,
refKey,
} from './wireit-config.zena';
/** Captured command output and wall-clock duration for a step that executed. */
export class StepExecution {
command: String;
exitCode: i32;
stdout: String;
stderr: String;
durationMs: i64;
new(this.command, this.exitCode, this.stdout, this.stderr, this.durationMs);
combinedOutput(): String {
if (this.stdout == '') {
return this.stderr;
}
if (this.stderr == '') {
return this.stdout;
}
if (this.stdout.endsWith('\n')) {
return this.stdout + this.stderr;
}
return `${this.stdout}\n${this.stderr}`;
}
}
/** Parameters for running one package's native Zena `"tests"` step. */
export class TestStepRequest {
root: String;
packageDir: String;
patterns: Array<String>;
extraArgs: Array<String>;
env: Array<(String, String)>;
envRemove: Array<String>;
debug: boolean;
new(
this.root,
this.packageDir,
this.patterns,
this.extraArgs,
this.env,
this.envRemove,
this.debug,
);
}
/**
* In-process executor for native Zena `"tests"` steps, supplied by `zena`
* so test targets compile and run without spawning shell or CLI child processes.
*/
export interface TestStepRunner {
/**
* Identity of the compiler and standard library running tests, evaluated
* lazily after a step's dependencies settle so upstream compiler builds
* invalidate downstream test steps in the same run.
*/
fingerprint(): Future<String>;
/** Runs one package's Zena test step and returns its captured output and status. */
runTests(request: TestStepRequest): Future<StepExecution>;
}
/** A planned build: the step graph, initial leaf map, and captured step runs. */
export class Plan {
graph: Graph;
leaves: HashMap<String, Fingerprint>;
executions: HashMap<String, StepExecution>;
new(
this.graph,
this.leaves,
this.executions = new HashMap<String, StepExecution>(),
);
}
function appendSegment(sb: StringBuilder, s: String) {
sb.appendI32(s.length);
sb.append(':');
sb.append(s);
}
function appendEnvSegments(
sb: StringBuilder,
env: Array<(String, EnvBinding)>,
) {
for (let (name, binding) in env) {
match (binding) {
case EnvLiteral as lit: {
appendSegment(sb, 'env:' + name);
appendSegment(sb, lit.value);
}
case EnvExternal as ext: {
let effective = getEnv(name) ?? ext.defaultValue;
if (effective != null) {
appendSegment(sb, 'env-ext:' + name);
appendSegment(sb, effective);
} else {
appendSegment(sb, 'env-unset:' + name);
}
}
}
}
}
/**
* Remote-cache variables consumed by the outer Zebra process. Child scripts
* and services strip these from their inherited environment unless their own
* `env` config declares them, so nested builds and test suites do not read or
* write the outer run's remote cache.
*/
export let REMOTE_CACHE_ENV_VARS = [
'ZEBRA_CACHE',
'ZEBRA_CACHE_READ_ONLY',
'ZEBRA_GITHUB_CUSTODIAN_URL',
'WIREIT_CACHE_GITHUB_CUSTODIAN_PORT',
];
/** Concrete `(name, value)` pairs to pass in `SpawnOptions.env`. */
function resolveEnvOverrides(
env: Array<(String, EnvBinding)>,
): Array<(String, String)> {
let overrides = new GrowableArray<(String, String)>();
for (let (name, binding) in env) {
match (binding) {
case EnvLiteral as lit: {
overrides.push((name, lit.value));
}
case EnvExternal as ext: {
if (getEnv(name) == null) {
let def = ext.defaultValue;
if (def != null) {
overrides.push((name, def));
}
}
}
}
}
return overrides;
}
/** Remote-cache variables to omit from `SpawnOptions.envRemove`. */
function resolveEnvRemovals(
env: Array<(String, EnvBinding)>,
): Array<String> {
let removals = new GrowableArray<String>();
for (let candidate in REMOTE_CACHE_ENV_VARS) {
var declared = false;
for (let (name, _) in env) {
if (name == candidate) {
declared = true;
break;
}
}
if (!declared) {
removals.push(candidate);
}
}
return removals;
}
function configFingerprint(
config: ScriptConfig,
salt: String,
forceSalted: boolean = false,
hasInProcessRunner: boolean = false,
) {
let sb = new StringBuilder();
appendSegment(sb, 'zebra-config-v1');
let command = config.command;
appendSegment(sb, command ?? '');
appendSegment(sb, config.clean);
appendSegment(sb, if (config.isWireit) 'wireit' else 'bare');
let svc = config.service;
if (svc != null) {
let pattern = svc.readyWhenLineMatches;
appendSegment(
sb,
if (pattern != null) 'service:line:' + pattern else 'service:ready',
);
} else {
appendSegment(sb, 'service:none');
}
let ruleType = config.ruleType;
if (ruleType != null) {
appendSegment(sb, 'type:' + ruleType);
}
let tests = config.tests;
if (ruleType == ZENA_TEST_RULE_TYPE && tests != null) {
for (let pat in tests) {
appendSegment(sb, 'test-glob:' + pat);
}
appendSegment(sb, if (config.testDebug) 'test-debug:1' else 'test-debug:0');
appendSegment(
sb,
if (hasInProcessRunner) 'test-runner:in-process' else 'test-runner:shell',
);
}
for (let glob in config.files) {
appendSegment(sb, 'file:' + glob);
}
for (let glob in config.output) {
appendSegment(sb, 'out:' + glob);
}
for (let lock in config.packageLocks) {
appendSegment(sb, 'lock:' + lock);
}
appendEnvSegments(sb, config.env);
if (forceSalted || !config.fingerprintable) {
// Bare npm scripts, scripts with undeclared inputs/outputs, and
// foreground top-level services must never go fresh or hit cache.
appendSegment(sb, 'salt:' + salt);
}
return new Fingerprint('sha256:' + sha256HexOfString(sb.toString()));
}
async function cleanOutputs(
root: String,
config: ScriptConfig,
ctx: RunContext,
): Future<void> {
if (config.clean == 'false') {
return;
}
if (config.clean == 'if-file-deleted') {
let delta = ctx.delta;
if (delta == null || delta.removed.length == 0) {
return;
}
}
let previous = ctx.previous;
if (previous != null) {
for (let entry in previous.outputs) {
if (entry.fingerprint.value.startsWith('sha256:')) {
await deleteIfExists(withRoot(root, entry.name));
}
}
} else {
for (let path in await expandGlobs(root, config.ref.packageDir, config.output)) {
await deleteIfExists(withRoot(root, path));
}
}
}
/**
* The PATH prefix npm would give the script: the package's own
* node_modules/.bin and the workspace root's, relative to the package
* directory the command runs in.
*/
function binPathPrefix(packageDir: String) {
if (packageDir == '') {
return 'node_modules/.bin';
}
let up = new StringBuilder();
for (let segment in packageDir.split('/')) {
if (segment != '') {
up.append('../');
}
}
return `node_modules/.bin:${up.toString()}node_modules/.bin`;
}
function fallbackTestShellCommand(
patterns: Array<String>,
extraArgs: Array<String>,
debug: boolean,
) {
let base = if (debug) 'zena -g test' else 'zena test';
let allArgs = new GrowableArray<String>();
for (let p in patterns) {
allArgs.push(p);
}
for (let a in extraArgs) {
allArgs.push(a);
}
return appendShellArgs(base, allArgs);
}
/**
* Runs one script's command (or native Zena `"tests"` step) and collects its
* outputs. Shell commands run through `sh -c` in the package directory with
* node_modules/.bin on PATH, like wireit through npm, and await exit
* asynchronously so independent steps run in parallel.
*/
export async function runScript(
root: String,
config: ScriptConfig,
ctx: RunContext,
volatileMark: String?,
hasher: FileHasher?,
executions: HashMap<String, StepExecution>? = null,
testRunner: TestStepRunner? = null,
): Future<OutputManifest> {
let command = config.command;
let isZenaTest = config.ruleType == ZENA_TEST_RULE_TYPE;
let tests = if (isZenaTest) config.tests else null;
if (command == null && tests == null && config.output.length == 0) {
// Command-less group or fileset scripts forward their resolved
// inputs as virtual entries so upstream manifest changes (and
// $volatile markers) propagate to downstream dependents.
let forwarded = new GrowableArray<OutputEntry>();
for (let inp in ctx.inputs) {
forwarded.push(new OutputEntry(
'$in:' + inp.name,
new Fingerprint('ref:' + inp.fingerprint.value),
));
}
return new OutputManifest(forwarded);
}
if (tests != null) {
let key = refKey(config.ref);
let sigBefore = checkSignal();
if (sigBefore != 0) {
throw new Error(
`script "${key}" interrupted by signal ${i32ToString(sigBefore)}`,
);
}
let extra = config.testExtraArgs ?? new GrowableArray<String>();
let envOverrides = resolveEnvOverrides(config.env);
let envRemovals = resolveEnvRemovals(config.env);
var exec: StepExecution | null = null;
if (testRunner != null) {
exec = await testRunner.runTests(
new TestStepRequest(
root,
config.ref.packageDir,
tests,
extra,
envOverrides,
envRemovals,
config.testDebug,
),
);
} else {
let startNs = monotonic() as i64;
let cwd = withRoot(root, config.ref.packageDir);
let shellCmd = fallbackTestShellCommand(tests, extra, config.testDebug);
let wrapped =
`export PATH="${binPathPrefix(config.ref.packageDir)}:$PATH"; ${shellCmd}`;
let result = await runWith(['sh', '-c', wrapped], {
cwd: cwd,
env: envOverrides,
envRemove: envRemovals,
processGroup: true,
});
let durationMs = div((monotonic() as i64) - startNs, 1000000 as i64);
exec = new StepExecution(
shellCmd,
result.exitCode,
result.stdout,
result.stderr,
durationMs,
);
}
let sigAfter = checkSignal();
if (sigAfter != 0) {
throw new Error(
`script "${key}" interrupted by signal ${i32ToString(sigAfter)}`,
);
}
let finished = exec;
if (executions != null) {
executions[key] = finished;
}
if (finished.exitCode != 0) {
throw new Error(
`script "${key}" failed (exit ${i32ToString(finished.exitCode)}): ${finished.command}\n` +
finished.stdout + finished.stderr,
);
}
} else if (command != null) {
let key = refKey(config.ref);
let sigBefore = checkSignal();
if (sigBefore != 0) {
throw new Error(
`script "${key}" interrupted by signal ${i32ToString(sigBefore)}`,
);
}
if (hasher != null) {
hasher.markMutated();
}
await cleanOutputs(root, config, ctx);
let startNs = monotonic() as i64;
let cwd = withRoot(root, config.ref.packageDir);
let wrapped =
`export PATH="${binPathPrefix(config.ref.packageDir)}:$PATH"; ${command}`;
let result = await runWith(['sh', '-c', wrapped], {
cwd: cwd,
env: resolveEnvOverrides(config.env),
envRemove: resolveEnvRemovals(config.env),
processGroup: true,
});
let sigAfter = checkSignal();
if (sigAfter != 0) {
throw new Error(
`script "${key}" interrupted by signal ${i32ToString(sigAfter)}`,
);
}
if (executions != null) {
let durationMs = div((monotonic() as i64) - startNs, 1000000 as i64);
executions[key] = new StepExecution(
command,
result.exitCode,
result.stdout,
result.stderr,
durationMs,
);
}
if (!result.succeeded()) {
throw new Error(
`script "${key}" failed (exit ${i32ToString(result.exitCode)}): ${command}\n` +
result.stdout + result.stderr,
);
}
}
let outFiles = await expandGlobs(root, config.ref.packageDir, config.output);
let outputs = new GrowableArray<OutputEntry>();
for (let path in outFiles) {
let fp = if (hasher != null) await hasher.hash(path) else await hashFile(root, path);
outputs.push(new OutputEntry(path, fp));
}
if (volatileMark != null) {
// An unfingerprintable script's true outputs are unknown, so its
// manifest must never repeat: dependents' cache keys change every
// run, which keeps them from going fresh over outputs this script
// may have changed without declaring (wireit poisons dependents of
// untracked scripts the same way).
outputs.push(new OutputEntry(
'$volatile',
new Fingerprint('volatile:' + volatileMark),
));
}
return new OutputManifest(outputs);
}
class ZbServiceHandle implements ServiceHandle {
#key: String;
#command: String;
#proc: Unmanaged<Process>;
var #stopped = false;
new(key: String, command: String, proc: Unmanaged<Process>)
: #key = key,
#command = command,
#proc = proc;
isAlive(): boolean {
if (this.#stopped) {
return false;
}
return !this.#proc.hasExited();
}
async stop(): Future<void> {
if (this.#stopped) {
return;
}
this.#stopped = true;
let proc = adopt(this.#proc);
if (proc.hasExited()) {
let res = await proc.wait();
if (checkSignal() == 0) {
throw new Error(
`service "${this.#key}" exited unexpectedly (exit ${i32ToString(res.exitCode)}): ${this.#command}\n` +
res.stdout + res.stderr,
);
}
return;
}
await proc.stop(2000 as i64);
}
}
function compileReadyRegex(config: ScriptConfig): Regex? {
let svc = config.service;
if (svc == null) {
return null;
}
let pattern = svc.readyWhenLineMatches;
if (pattern == null) {
return null;
}
try {
return new Regex(pattern);
} catch (e) {
throw new ConfigError(
`script "${refKey(config.ref)}": invalid readyWhen.lineMatches regex "${pattern}": ${e.message}`,
);
}
}
async function startService(
root: String,
config: ScriptConfig,
readyRegex: Regex?,
hasher: FileHasher,
executions: HashMap<String, StepExecution>,
): Future<ServiceHandle> {
let key = refKey(config.ref);
let sigBefore = checkSignal();
if (sigBefore != 0) {
throw new Error(
`service "${key}" interrupted by signal ${i32ToString(sigBefore)}`,
);
}
hasher.markMutated();
let startNs = monotonic() as i64;
let command = config.command ?? '';
let cwd = withRoot(root, config.ref.packageDir);
let wrapped =
`export PATH="${binPathPrefix(config.ref.packageDir)}:$PATH"; ${command}`;
let proc = spawnWith(['sh', '-c', wrapped], {
cwd: cwd,
env: resolveEnvOverrides(config.env),
envRemove: resolveEnvRemovals(config.env),
processGroup: true,
});
if (readyRegex != null) {
var matched = false;
while (true) {
let line = await proc.readLine();
if (line == null) {
break;
}
if (readyRegex.test(line)) {
matched = true;
break;
}
}
let sigAfter = checkSignal();
if (sigAfter != 0) {
await proc.stop(2000 as i64);
throw new Error(
`service "${key}" interrupted by signal ${i32ToString(sigAfter)}`,
);
}
if (!matched) {
let res = await proc.stop(500 as i64);
let durationMs = div((monotonic() as i64) - startNs, 1000000 as i64);
executions[key] = new StepExecution(
command,
res.exitCode,
res.stdout,
res.stderr,
durationMs,
);
throw new Error(
`service "${key}" exited before readyWhen.lineMatches matched (exit ${i32ToString(res.exitCode)}): ${command}\n` +
res.stdout + res.stderr,
);
}
} else if (proc.hasExited()) {
let res = await proc.wait();
let durationMs = div((monotonic() as i64) - startNs, 1000000 as i64);
executions[key] = new StepExecution(
command,
res.exitCode,
res.stdout,
res.stderr,
durationMs,
);
throw new Error(
`service "${key}" exited immediately (exit ${i32ToString(res.exitCode)}): ${command}\n` +
res.stdout + res.stderr,
);
}
let durationMs = div((monotonic() as i64) - startNs, 1000000 as i64);
executions[key] = new StepExecution(command, 0, '', '', durationMs);
return new ZbServiceHandle(key, command, disown(proc));
}
async function runForegroundService(
root: String,
config: ScriptConfig,
readyRegex: Regex?,
salt: String,
hasher: FileHasher,
): Future<OutputManifest> {
let sigBefore = checkSignal();
if (sigBefore != 0) {
throw new Error(
`service "${refKey(config.ref)}" interrupted by signal ${i32ToString(sigBefore)}`,
);
}
hasher.markMutated();
let command = config.command ?? '';
let cwd = withRoot(root, config.ref.packageDir);
let wrapped =
`export PATH="${binPathPrefix(config.ref.packageDir)}:$PATH"; ${command}`;
let proc = spawnWith(['sh', '-c', wrapped], {
cwd: cwd,
env: resolveEnvOverrides(config.env),
envRemove: resolveEnvRemovals(config.env),
processGroup: true,
});
var matched = readyRegex == null;
while (true) {
let line = await proc.readLine();
if (line == null) {
break;
}
console.log(line);
if (!matched && readyRegex != null && readyRegex.test(line)) {
matched = true;
}
}
let sigAfterRead = checkSignal();
if (sigAfterRead != 0) {
await proc.stop(2000 as i64);
throw new Error(
`service "${refKey(config.ref)}" interrupted by signal ${i32ToString(sigAfterRead)}`,
);
}
let res = await proc.wait();
let sigAfterWait = checkSignal();
if (sigAfterWait != 0) {
throw new Error(
`service "${refKey(config.ref)}" interrupted by signal ${i32ToString(sigAfterWait)}`,
);
}
if (!matched) {
throw new Error(
`service "${refKey(config.ref)}" exited before readyWhen.lineMatches matched (exit ${i32ToString(res.exitCode)}): ${command}\n` +
res.stdout + res.stderr,
);
}
if (!res.succeeded()) {
throw new Error(
`service "${refKey(config.ref)}" failed (exit ${i32ToString(res.exitCode)}): ${command}\n` +
res.stdout + res.stderr,
);
}
return new OutputManifest([
new OutputEntry('$volatile', new Fingerprint('volatile:' + salt)),
]);
}
export function lockfileCandidates(
packageDir: String,
lockNames: Array<String>,
): Array<String> {
let candidates = new GrowableArray<String>();
let seen = new HashSet<String>();
var cur = packageDir;
while (true) {
for (let name in lockNames) {
let candidate = joinWorkspace(cur, name);
if (seen.add(candidate)) {
candidates.push(candidate);
}
}
if (cur == '') {
break;
}
var slash = -1;
for (var i = 0; i < cur.length; i += 1) {
if (cur.getByteAt(i) == 47) {
slash = i;
}
}
cur = if (slash < 0) '' else cur.sliceBytes(0, slash);
}
return candidates;
}
async function resolveTestRunnerLeaf(
runner: TestStepRunner,
): Future<Array<(String, Fingerprint)>> {
let entries = new GrowableArray<(String, Fingerprint)>();
entries.push(('__zena_test_runner__', new Fingerprint(await runner.fingerprint())));
return entries;
}
class CoalescedRunnerLeaf {
#runner: TestStepRunner;
var #pending: Future<Array<(String, Fingerprint)>>? = null;
new(runner: TestStepRunner) : #runner = runner;
resolve(): Future<Array<(String, Fingerprint)>> {
let existing = this.#pending;
if (existing != null) {
return existing;
}
let started = resolveTestRunnerLeaf(this.#runner);
this.#pending = started;
return started;
}
}
function buildStepInputs(
config: ScriptConfig,
hasher: FileHasher,
runnerLeaf: CoalescedRunnerLeaf? = null,
): Array<InputRef> {
validateGlobs(config.ref.packageDir, config.files);
validateGlobs(config.ref.packageDir, config.output);
let inputs = new GrowableArray<InputRef>();
let isZenaTest = config.ruleType == ZENA_TEST_RULE_TYPE && config.tests != null;
let lockCandidates = if (
config.isWireit &&
(config.command != null || isZenaTest) &&
config.fingerprintable &&
config.packageLocks.length > 0
)
lockfileCandidates(config.ref.packageDir, config.packageLocks)
else
new GrowableArray<String>();
if (config.files.length > 0 || lockCandidates.length > 0) {
let pkgDir = config.ref.packageDir;
let globs = config.files;
inputs.push(
new LeafSet('files', () => hasher.hashInputs(pkgDir, globs, lockCandidates)),
);
}
if (isZenaTest && config.fingerprintable && runnerLeaf != null) {
let sharedLeaf = runnerLeaf;
inputs.push(new LeafSet('test-runner', () => sharedLeaf.resolve()));
}
// Deduplicate dependency edges, letting `cascade: true` win if listed both ways.
let order = new GrowableArray<String>();
let cascadeByKey = new HashMap<String, boolean>();
let seen = new HashSet<String>();
for (let dep in config.dependencies) {
let key = refKey(dep.ref);
if (seen.add(key)) {
order.push(key);
cascadeByKey[key] = dep.cascade;
} else if (dep.cascade) {
cascadeByKey[key] = true;
}
}
for (let key in order) {
if (cascadeByKey.getOr(key, true)) {
inputs.push(new StepOutputs(new StepKey(key)));
} else {
inputs.push(new StepOrder(new StepKey(key)));
}
}
return inputs;
}
/**
* Turns collected script configs into a step graph with lazy per-step
* input glob expansion backed by `hasher`. `foregroundServiceKey` names
* a service invoked directly as the top-level target so it runs in the
* foreground instead of sitting idle with zero dependents.
*/
export function planScripts(
root: String,
configs: Array<ScriptConfig>,
salt: String,
hasher: FileHasher? = null,
foregroundServiceKey: String? = null,
testRunner: TestStepRunner? = null,
): Plan {
let fileHasher = hasher ?? new FileHasher(root);
let hasRunner = testRunner != null;
let runnerLeaf = if (testRunner != null) new CoalescedRunnerLeaf(testRunner) else null;
let leaves = new HashMap<String, Fingerprint>();
let executions = new HashMap<String, StepExecution>();
let steps = new GrowableArray<Step>();
for (let config in configs) {
let key = refKey(config.ref);
let inputs = buildStepInputs(config, fileHasher, runnerLeaf);
if (config.service != null) {
let readyRegex = compileReadyRegex(config);
if (foregroundServiceKey != null && key == foregroundServiceKey) {
steps.push(new TaskStep(
new StepKey(key),
configFingerprint(config, salt, true, hasRunner),
inputs,
(ctx) => runForegroundService(root, config, readyRegex, salt, fileHasher),
));
} else {
steps.push(new ServiceStep(
new StepKey(key),
configFingerprint(config, salt, false, hasRunner),
inputs,
(svcCtx) => startService(root, config, readyRegex, fileHasher, executions),
));
}
} else {
steps.push(new TaskStep(
new StepKey(key),
configFingerprint(config, salt, false, hasRunner),
inputs,
(ctx) => runScript(
root,
config,
ctx,
if (config.fingerprintable) null else salt,
fileHasher,
executions,
testRunner,
),
));
}
}
return new Plan(new Graph(steps), leaves, executions);
}
| Version | Published | Dependencies | Approved by |
|---|---|---|---|
| 0.0.1-preview.1 | 2026-11-19 |
|