zebra 0.0.1-preview.1
Zebra: an incremental build orchestrator for Zena and standalone workspaces
zebra#
An incremental script and service runner for Zena (zena-package.json), standalone Zebra (zebra.json), and Wireit (package.json) workspaces, built on zena-lang:workflow.
Zebra runs a dependency graph of scripts and long-running services with declared inputs, outputs, and dependencies, rerunning only what changed. For each script, Zebra content-hashes the declared inputs and asks the workflow engine what to do: nothing (when the previous run recorded the same fingerprints and declared outputs are intact), restore from the local content-addressed cache, or execute the command (sh -c in the package directory with node_modules/.bin on PATH).
Installation#
zebra is built into the zena CLI (zena build <target>). To use it as a library or standalone runner:
zena pkg add zena-lang:zebra
CLI Usage#
zena build [-j <jobs>] [-w|--watch] [-f|--force] [--no-cache] [-v|--verbose] [<target>] [-- <args>...]
zena run [-j <jobs>] [-w|--watch] [-f|--force] [--no-cache] [-v|--verbose] <target> [-- <args>...]
zena test [-j <jobs>] [-w|--watch] [--force] [--no-cache] [-v|--verbose] [-- <args>...]
zena build --migrate [--dry-run] [-f|--force] [<package-dir>...]
<target>can be a script name in the current package (zena build test,zena run check), a package directory (zena build ./packages/app, which runsbuild), or<package-dir>:<script>(zena build ./packages/app:test). With no<target>,zena buildruns the current package'sbuildscript andzena testruns itstestscript.-- <args>...appends shell-quoted arguments to the entry target's command only, running that entry step uncached without overwriting its saved freshness record.-j/--jobs <n>bounds parallel command execution (defaulting toZEBRA_PARALLEL,WIREIT_PARALLEL, orZENA_AVAILABLE_PARALLELISM).-w/--watchwatches input globs, lockfiles, and package manifests for changes, re-running affected steps incrementally while keeping unchanged services alive in aServicePoolacross rebuilds.-f/--forcebypasses freshness and cached outputs to rerun scripts unconditionally.--no-cachedisables restoring outputs from.zebra/blobs/when a step is not fresh.-v/--verboseprints every step (includingfreshandskipped) and any captured stdout/stderr on the rail.--migrateconverts Wireit blocks inpackage.jsonintozena-package.json(when present) orzebra.json.
Programmatic Usage#
import { buildTarget, reportBuild } from 'zena-lang:zebra/runner';
import { ScriptRef } from 'zena-lang:zebra/wireit-config';
let outcome = await buildTarget('.', new ScriptRef('packages/app', 'build'), {
parallelism: 4,
});
let exitCode = reportBuild(outcome);
Configuration (zena-package.json and zebra.json)#
Zena packages configure scripts and filesets directly in zena-package.json. Non-Zena projects can use a standalone zebra.json (with an optional "workspace": {"packages": [...]} at the workspace root). Both support // and /* ... */ comments:
{
"name": "app",
"dependencies": {
"lib": {"path": "../lib"},
},
"filesets": {
"assets": ["assets/**"],
},
"scripts": {
"build": {
"command": "zena build src/main.zena -o out/app.wasm",
"output": ["out/app.wasm"],
"dependencies": ["source", "assets"],
},
"test": {
"type": "zena:test",
"tests": "test/*_test.zena",
},
"fmt": "zena fmt --check .",
},
}
- Declarative
zena:teststeps and defaulttesttarget: Scripts inzena-package.jsonandzebra.jsoncan set"type": "zena:test"(with optional"tests": true,"tests": "test/*_test.zena", or"tests": ["test/*_test.zena", ...]— defaulting to["test/**/*_test.zena", "tests/**/*_test.zena"]when omitted — and optional"debug": truefor-g) to run Zena tests in-process throughTestStepRunner. Test pattern globs merge intofiles, andzena-package.json"zena:test"steps automatically depend onsource. When azena-package.jsonomits atestscript (and no fallbackpackage.jsondefinestest), Zebra synthesizes a"type": "zena:test"target overtest/**/*_test.zenaandtests/**/*_test.zena. - Automatic
sourcefileset: Every package withzena-package.jsonexposes an automaticsourcefileset covering its library.zenafiles (from"source", defaulting to"src"and excluding*_test.zena,test/**, andtests/**),zena-package.json, any"wit"dependencies, and thesourcefilesets of all"path"dependencies. Scripts opt into it via"dependencies": ["source"](or"../pkg:source"). - Named
filesets: Entries accept either a glob array shorthand ("wit": ["wit/**/*.wit"]) or{"files": [...], "dependencies": [...]}and compose transitively without spawning commands. - Caching and tracking: Task scripts cache automatically whenever
files,output, or a fileset dependency is declared, defaultingoutputto[]when omitted. Setting"cache": falseopts a step out of freshness and caching while keeping itsfilesglobs active in--watchmode. Omittingfiles,output, and fileset dependencies on a command step leaves it untracked so it runs every time. String entries ("fmt": "zena fmt --check .") are shorthands for always-run commands. - Locks:
locksdefaults to["zena-lock.json"]and may be set at the package root or overridden per script ("locks": []disables lockfile hashing). - Services: Scripts with
"service": trueor"service": {"readyWhen": {"lineMatches": "..."}}start on demand when a dependent executes, wait for readiness when configured, are shared across dependents, and are stopped as a Unix process group (SIGTERMthenSIGKILL) when dependents finish. Invoking a service directly as the top-level target runs its dependencies and streams the service in the foreground. - Wireit fallback: When a package directory has no
zebra.jsonand no"scripts"block inzena-package.json, Zebra falls back topackage.jsonWireit scripts while still serving filesets fromzena-package.json. Dependencies can cross freely betweenzena-package.json,zebra.json, andpackage.jsonpackages.
Cache Layout (.zebra/)#
Each package's <pkg>/.zebra/ directory stores:
state.json— freshness records for that package's scripts (the last cache key and output manifest).cache/<key>.json— cache key to output manifest.blobs/— content-addressed output bytes (sha256).
The workspace root's .zebra/ directory stores:
hashes.json— the(mtime, size)tosha256file hash memo shared across packages (with a 20ms racy-timestamp guard).plans/<hash>.plan—stat-verified cachedScriptConfigclosures keyed by target entry.
A dependency contributes the digest of its outputs to dependents' cache keys rather than its input fingerprint, giving early cutoff when a dependency reruns and produces identical outputs.
GitHub Actions Remote Cache#
Setting ZEBRA_CACHE=github layers GitHubActionsCache behind the local .zebra/ DiskCache (TieredCache). On a local cache miss, Zebra queries the GitHub Actions v2 CacheService Twirp JSON API, verifies downloaded blob sha256 digests, populates <pkg>/.zebra/, and materializes outputs into the workspace. On a build step execution, Zebra writes <pkg>/.zebra/ first and uploads the step's entry to GitHub Actions Cache (ZEBRA_CACHE_READ_ONLY=1 disables uploads; ZEBRA_CACHE=none disables both local and remote caching).
In GitHub Actions workflows, run .github/actions/setup-zebra-cache before build steps so ACTIONS_RESULTS_URL and ACTIONS_RUNTIME_TOKEN are served to run: steps over a 127.0.0.1 token custodian (ZEBRA_GITHUB_CUSTODIAN_URL).
Libraries#
zebra(zebra/main) — CLI entry point (main).zebra/runner—buildTarget,watchTarget,BuildOptions,WatchOptions,reportBuild,reportMigration,formatBuildReport, andBuildOutcome.zebra/zebra-config—zena-package.jsonandzebra.jsonparsing (ZebraWorkspace), synthesizedsourcefilesets, and Wireit migration (migrateWorkspaceToZebra,convertPackageJsonToZebraJson,mergeWireitIntoZenaPackageJson).zebra/wireit-config—package.jsonWireit parsing (WireitWorkspace) and shared script configuration types (ScriptConfig,ScriptRef,DependencySpec,ServiceConfig,EnvBinding,ConfigError,refKey,resolveDependency).zebra/plan-cache—.zebra/plans/snapshot serialization andstat-verified warm plan loading (loadOrCollectConfigs,PlanLoadStats,ConfigProbe).zebra/builder— TranslatesScriptConfigclosures intoworkflowsteps and execution callbacks (planScripts,StepExecution,lockfileCandidates).zebra/disk-cache,zebra/github-cache, andzebra/state-json—.zebra/blob/manifest storage (DiskCache,TieredCache,stateDirOf), GitHub Actions remote caching (GitHubActionsCache,GitHubCacheConfig,CacheMode,resolveCacheMode,resolveGitHubCacheConfig), and JSON serialization for state, manifests, and hash memos.zebra/fs-walkandzebra/fs-util— Input glob discovery (FileHasher,expandGlobs,hashFile,validateGlobs,splitBase) and workspace path utilities.zebra/shell— POSIX shell argument quoting (shellQuote,appendShellArgs). Zebra's terminal output useszena-lang:terminal(packages/terminal): colors, the Clack-style rail layout and the spinner.
License#
MIT
import {Future} from 'zena:async';
import {getEnv} from 'zena:cli';
import {HashSet} from 'zena:collections';
import {StringBuilder} from 'zena:core';
import {isFile, readFile, writeFile} from 'zena:fs';
import {div} from 'zena:math';
import {checkSignal, listenSignals, waitSignalFor} from 'zena:process';
import {monotonic} from 'zena:time';
import {NullCache, StepCache} from 'workflow/cache';
import {
Engine,
Failed,
RunOptions,
RunResult,
ServicePool,
Skipped,
StepOutcome,
StepSource,
Succeeded,
} from 'workflow/engine';
import {Fingerprint} from 'workflow/fingerprint';
import {OutputManifest} from 'workflow/manifest';
import {EngineState, StepRecord} from 'workflow/state';
import {StepKey} from 'workflow/step';
import {Spinner} from 'terminal/animation';
import {
StepExecution,
TestStepRequest,
TestStepRunner,
lockfileCandidates,
planScripts,
} from './builder.zena';
import {Colors} from 'terminal/colors';
import {DiskCache, TieredCache, stateDirOf} from './disk-cache.zena';
import {ensureParentDir, withRoot} from './fs-util.zena';
import {FileHasher} from './fs-walk.zena';
import {
CacheMode,
GitHubActionsCache,
resolveCacheMode,
resolveGitHubCacheConfig,
} from './github-cache.zena';
import {RailLayout, StepKind, formatDuration} from 'terminal/layout';
import {PlanLoadStats, loadOrCollectConfigs} from './plan-cache.zena';
import {sha256HexOfString} from 'zena:crypto';
import {appendShellArgs} from './shell.zena';
import {stateFromJson, stateToJson} from './state-json.zena';
import {ConfigError, ScriptConfig, ScriptRef, refKey} from './wireit-config.zena';
import {MigrationResult} from './zebra-config.zena';
export {TestStepRequest, TestStepRunner} from './builder.zena';
// State is stored per package ('<pkg>/.zebra/state.json'), like wireit's
// .wireit/, so its location does not depend on the invocation
// directory.
function packageStatePath(root: String, stateDir: String) {
return withRoot(root, stateDir + '/state.json');
}
async function loadState(root: String, stateDirs: Array<String>) {
let state = new EngineState();
for (let stateDir in stateDirs) {
let path = packageStatePath(root, stateDir);
if (!(await isFile(path))) {
continue;
}
stateFromJson(await readFile(path)).forEach((key, record) => {
state.setRecord(new StepKey(key), record);
});
}
return state;
}
/**
* Writes each package's records back, merged over what its state file
* already holds — a run of one target must not drop a package's
* records for scripts outside the run's closure.
*/
async function saveState(root: String, result: RunResult, skipKey: String? = null) {
let entries = new GrowableArray<(String, StepRecord)>();
let dirs = new GrowableArray<String>();
let seenDirs = new HashSet<String>();
result.state.forEach((key, record) => {
if (skipKey != null && key == skipKey) {
return;
}
entries.push((key, record));
let stateDir = stateDirOf(key);
if (seenDirs.add(stateDir)) {
dirs.push(stateDir);
}
});
let byDir = new HashMap<String, EngineState>();
for (let stateDir in dirs) {
let path = packageStatePath(root, stateDir);
byDir[stateDir] = if (await isFile(path))
stateFromJson(await readFile(path))
else
EngineState.empty();
}
for (let (key, record) in entries) {
let stateDir = stateDirOf(key);
if (let (true, packageState) = byDir.get(stateDir)) {
packageState.setRecord(new StepKey(key), record);
}
}
for (let stateDir in dirs) {
if (let (true, packageState) = byDir.get(stateDir)) {
let path = packageStatePath(root, stateDir);
await ensureParentDir(path);
await writeFile(path, stateToJson(packageState));
}
}
}
function joinComma(parts: GrowableArray<String>) {
let sb = new StringBuilder();
for (var i = 0; i < parts.length; i += 1) {
if (i > 0) {
sb.append(', ');
}
sb.append(parts[i]);
}
return sb.toString();
}
function formatActiveSpinnerMessage(
inFlight: GrowableArray<String>,
completed: i32,
total: i32,
) {
if (inFlight.length == 0) {
return `Building (${i32ToString(completed)}/${i32ToString(total)})...`;
}
let first = inFlight[0];
let more = if (inFlight.length > 1)
` (+${i32ToString(inFlight.length - 1)} more)`
else
'';
return `Running ${first}${more} (${i32ToString(completed)}/${i32ToString(total)})...`;
}
/**
* Formats a finished build as a Clack-style rail report (`┌`, `│`, `└`)
* with step status symbols (`◇`, `◆`, `○`, `▲`, `■`), rail-indented
* command output blocks, and a duration/count summary in the footer.
*
* By default, `ran`, `restored`, and `failed` steps are listed on the
* rail while `fresh` and `skipped` steps are summarized in the footer;
* passing `verbose = true` lists every step on the rail and prints
* captured stdout/stderr for passing steps too.
*/
export function formatBuildReport(
outcome: BuildOutcome,
verbose: boolean = false,
colors: Colors = new Colors(),
): String {
let layout = new RailLayout(colors);
let lines = new GrowableArray<String>();
let title = if (outcome.target != '')
`${layout.badge('zebra')} ${colors.bold(outcome.target)}`
else
layout.badge('zebra');
lines.push(layout.intro(title));
let result = outcome.result;
var ranCount = 0;
var restoredCount = 0;
var freshCount = 0;
var failedCount = 0;
var skippedCount = 0;
for (let key in outcome.stepKeys) {
let stepOutcome = result.outcomeOf(new StepKey(key));
if (stepOutcome is Succeeded) {
if (stepOutcome.source == StepSource.Fresh) {
freshCount += 1;
if (verbose) {
lines.push(layout.bar());
lines.push(layout.step(StepKind.Fresh, `${key}: fresh`));
}
} else if (stepOutcome.source == StepSource.Cache) {
restoredCount += 1;
lines.push(layout.bar());
lines.push(layout.step(StepKind.Active, `${key}: restored`));
} else {
ranCount += 1;
lines.push(layout.bar());
var detail = '';
var output = '';
if (let (true, exec) = outcome.executions.get(key)) {
detail = `(${formatDuration(exec.durationMs)})`;
output = exec.combinedOutput();
}
lines.push(layout.step(StepKind.Success, `${key}: ran`, detail));
if (verbose && output != '') {
for (let logLine in layout.logBlock(output, false)) {
lines.push(logLine);
}
}
}
} else if (stepOutcome is Failed) {
failedCount += 1;
lines.push(layout.bar());
if (let (true, exec) = outcome.executions.get(key)) {
let detail = `(${formatDuration(exec.durationMs)})`;
lines.push(
layout.step(
StepKind.Error,
`${key}: failed (exit ${i32ToString(exec.exitCode)})`,
detail,
),
);
let combined = exec.combinedOutput();
let blockText = if (combined != '')
`$ ${exec.command}\n${combined}`
else
`$ ${exec.command}`;
for (let logLine in layout.logBlock(blockText, true)) {
lines.push(logLine);
}
} else {
lines.push(layout.step(StepKind.Error, `${key}: failed`));
for (let logLine in layout.logBlock(stepOutcome.error.message, true)) {
lines.push(logLine);
}
}
} else if (stepOutcome is Skipped) {
skippedCount += 1;
if (verbose) {
lines.push(layout.bar());
lines.push(
layout.step(StepKind.Warning, `${key}: skipped (${stepOutcome.reason})`),
);
}
}
}
lines.push(layout.bar());
let counts = new GrowableArray<String>();
if (failedCount > 0) {
counts.push(`${i32ToString(failedCount)} failed`);
}
if (skippedCount > 0) {
counts.push(`${i32ToString(skippedCount)} skipped`);
}
if (ranCount > 0) {
counts.push(`${i32ToString(ranCount)} ran`);
}
if (restoredCount > 0) {
counts.push(`${i32ToString(restoredCount)} restored`);
}
if (freshCount > 0) {
counts.push(`${i32ToString(freshCount)} fresh`);
}
let status = if (result.allSucceeded)
colors.green('build succeeded')
else
colors.red('build failed');
let elapsed = colors.dim(`in ${formatDuration(outcome.durationMs)}`);
let summary = if (counts.length > 0)
`${status} ${elapsed} ${colors.dim(`(${joinComma(counts)})`)}`
else
`${status} ${elapsed}`;
lines.push(layout.outro(summary));
let sb = new StringBuilder();
for (var i = 0; i < lines.length; i += 1) {
if (i > 0) {
sb.append('\n');
}
sb.append(lines[i]);
}
return sb.toString();
}
/**
* Prints a Clack-style build report and returns the exit status: 0 when
* every step succeeded, 1 otherwise (or `128 + sig` if interrupted).
* `zebra` and `zena build <target>` both report this way.
*/
export function reportBuild(
outcome: BuildOutcome,
verbose: boolean = false,
colors: Colors = new Colors(),
): i32 {
let sig = checkSignal();
if (sig != 0) {
return 128 + sig;
}
console.log(formatBuildReport(outcome, verbose, colors));
if (!outcome.result.allSucceeded) {
return 1;
}
return 0;
}
/**
* A finished build: the engine result plus the step keys in dependency
* order, target label, wall-clock duration, and captured step outputs.
*/
export class BuildOutcome {
result: RunResult;
stepKeys: Array<String>;
target: String;
durationMs: i64;
executions: HashMap<String, StepExecution>;
new(
this.result,
this.stepKeys,
this.target = '',
this.durationMs = 0 as i64,
this.executions = new HashMap<String, StepExecution>(),
);
}
/**
* Default max concurrent script executions: ZEBRA_PARALLEL, WIREIT_PARALLEL,
* ZENA_AVAILABLE_PARALLELISM, or 8.
*/
export function defaultParallelism(
lookupEnv: (name: String) => String? = getEnv,
): i32 {
for (let varName in [
'ZEBRA_PARALLEL',
'WIREIT_PARALLEL',
'ZENA_AVAILABLE_PARALLELISM',
]) {
let raw = lookupEnv(varName);
if (raw != null) {
if (let (true, n) = parseI32(raw)) {
if (n >= 1) {
return n;
}
}
}
}
return 8;
}
async function verifyManifestOutputs(
hasher: FileHasher,
manifest: OutputManifest,
): Future<boolean> {
for (let entry in manifest.outputs) {
if (!entry.fingerprint.value.startsWith('sha256:')) {
continue;
}
let current = await hasher.hashIfExists(entry.name);
if (current == null || current.value != entry.fingerprint.value) {
return false;
}
}
return true;
}
/** Options for `buildTarget`. Every field is optional. */
export type BuildOptions = {
/** Maximum concurrent script executions (`<= 0` uses `defaultParallelism()`). */
parallelism?: i32,
/** Bypass freshness and cached outputs to rerun every reachable script. */
force?: boolean,
/** Read and write outputs through the local `.zebra/` disk cache (`true` if absent). */
useCache?: boolean,
/** Optional second-level cache layered behind the local `.zebra/` disk cache. */
remoteCache?: StepCache?,
/** Optional environment lookup override (`getEnv` if absent). */
lookupEnv?: ((name: String) => String?)?,
/** Optional sink for non-fatal remote cache warnings (`console.error` if absent). */
onCacheWarning?: ((message: String) => void)?,
/** Optional caller-supplied `FileHasher` for inspecting hash and stat counters. */
hasherOut?: FileHasher?,
/** Optional sink populated with warm/cold plan snapshot load statistics. */
planStatsOut?: PlanLoadStats?,
/** Extra CLI arguments after `--` appended to the entry target's command or `"tests"` step. */
extraArgs?: Array<String>?,
/** Optional in-process executor for native Zena `"tests"` steps. */
testRunner?: TestStepRunner?,
};
/** Options for `watchTarget`. Every field is optional. */
export type WatchOptions = {
/** Maximum concurrent script executions (`<= 0` uses `defaultParallelism()`). */
parallelism?: i32,
/** Bypass freshness and cached outputs on each rebuild. */
force?: boolean,
/** Read and write outputs through the local `.zebra/` disk cache (`true` if absent). */
useCache?: boolean,
/** Optional second-level cache layered behind the local `.zebra/` disk cache. */
remoteCache?: StepCache?,
/** Optional environment lookup override (`getEnv` if absent). */
lookupEnv?: ((name: String) => String?)?,
/** Optional sink for non-fatal remote cache warnings (`console.error` if absent). */
onCacheWarning?: ((message: String) => void)?,
/** Polling interval between file-snapshot checks in milliseconds (`150` if absent). */
pollIntervalMs?: i64,
/**
* Callback invoked after each iteration's post-build snapshot is captured.
* Returning `false` stops watching and tears down pinned services.
* When omitted, each iteration prints `reportBuild(outcome, verbose)`.
*/
onIteration?: ((iteration: i32, outcome: BuildOutcome) => Future<boolean>)?,
/** Print `fresh` and `skipped` steps and passing step output on each watch report. */
verbose?: boolean,
/** Extra CLI arguments after `--` appended to the entry target's command or `"tests"` step. */
extraArgs?: Array<String>?,
/** Optional in-process executor for native Zena `"tests"` steps. */
testRunner?: TestStepRunner?,
};
type RunTargetOptions = {
parallelism: i32,
force: boolean,
useCache: boolean,
remoteCache: StepCache?,
lookupEnv: ((name: String) => String?)?,
onCacheWarning: ((message: String) => void)?,
hasherOut: FileHasher?,
servicePool: ServicePool?,
allowForegroundService: boolean,
configPathsOut: GrowableArray<String>?,
configsOut: GrowableArray<ScriptConfig>?,
planStatsOut: PlanLoadStats?,
extraArgs: Array<String>?,
testRunner: TestStepRunner?,
};
async function runTargetOnce(
root: String,
entry: ScriptRef,
options: RunTargetOptions,
) {
let {
parallelism,
force,
useCache,
remoteCache,
lookupEnv,
onCacheWarning,
hasherOut,
servicePool,
allowForegroundService,
configPathsOut,
configsOut,
planStatsOut,
extraArgs,
testRunner,
} = options;
listenSignals();
let startNs = monotonic() as i64;
let envLookup = lookupEnv ?? getEnv;
let jobs = if (parallelism > 0) parallelism else defaultParallelism(envLookup);
let entryKey = refKey(entry);
let loadedConfigs = await loadOrCollectConfigs(
root,
entry,
configPathsOut,
planStatsOut,
);
let hasExtraArgs = extraArgs != null && extraArgs.length > 0;
var configs = loadedConfigs;
if (hasExtraArgs) {
let cliExtra = extraArgs as Array<String>;
let rewritten = new GrowableArray<ScriptConfig>();
for (let config in loadedConfigs) {
if (refKey(config.ref) == entryKey) {
let baseCommand = config.command;
let baseTests = config.tests;
if (config.isFileset || (baseCommand == null && baseTests == null)) {
throw new ConfigError(
`script "${entryKey}" has no command and cannot accept extra arguments after --`,
);
}
rewritten.push(
new ScriptConfig(
config.ref,
config.isWireit,
if (baseCommand != null) appendShellArgs(baseCommand, cliExtra) else null,
config.files,
config.output,
config.filesDeclared,
config.outputDeclared,
config.dependencies,
config.clean,
config.service,
config.env,
config.packageLocks,
false,
config.isFileset,
baseTests,
config.testDebug,
if (baseTests != null) cliExtra else null,
config.ruleType,
),
);
} else {
rewritten.push(config);
}
}
configs = rewritten;
}
if (configsOut != null) {
for (let config in configs) {
configsOut.push(config);
}
}
let foregroundServiceKey = if (
allowForegroundService &&
configs.length > 0 &&
configs[0].service != null
)
entryKey
else
null;
let hasher = hasherOut ?? new FileHasher(root);
await hasher.load();
let plan = planScripts(
root,
configs,
i64ToString(monotonic() as i64),
hasher,
foregroundServiceKey,
testRunner,
);
let stateDirs = new GrowableArray<String>();
let seenDirs = new HashSet<String>();
let uncachedSteps = new HashSet<String>();
for (let config in configs) {
let k = refKey(config.ref);
let stateDir = stateDirOf(k);
if (seenDirs.add(stateDir)) {
stateDirs.push(stateDir);
}
if (!config.fingerprintable || config.isFileset) {
uncachedSteps.add(k);
}
}
if (foregroundServiceKey != null) {
uncachedSteps.add(foregroundServiceKey);
}
if (hasExtraArgs) {
uncachedSteps.add(entryKey);
}
var cache: StepCache = new NullCache();
if (useCache) {
let mode = if (remoteCache != null)
CacheMode.Local
else
resolveCacheMode(envLookup);
if (mode != CacheMode.None) {
let diskCache = new DiskCache(
root,
() => {
hasher.markMutated();
},
(path, fp) => hasher.recordRestored(path, fp),
uncachedSteps,
);
if (remoteCache != null) {
cache = new TieredCache(diskCache, remoteCache, uncachedSteps);
} else if (mode == CacheMode.GitHub) {
let ghConfig = await resolveGitHubCacheConfig(envLookup);
let warn = onCacheWarning ?? ((msg: String) => {
console.error(`zebra: ${msg}`);
});
let ghCache = new GitHubActionsCache(root, diskCache, ghConfig, warn);
cache = new TieredCache(diskCache, ghCache, uncachedSteps);
} else {
cache = diskCache;
}
}
}
let engine = new Engine(
(data: String) => new Fingerprint('sha256:' + sha256HexOfString(data)),
cache,
);
let spinner = new Spinner();
let inFlight = new GrowableArray<String>();
var completed = 0;
let filesetKeys = new HashSet<String>();
for (let config in configs) {
let k = refKey(config.ref);
if (config.isFileset && k != entryKey) {
filesetKeys.add(k);
}
}
let totalSteps = configs.length - filesetKeys.size;
var result: RunResult | null = null;
try {
result = await engine.run(
plan.graph,
plan.leaves,
await loadState(root, stateDirs),
new RunOptions(
jobs,
force,
(step, manifest) => verifyManifestOutputs(hasher, manifest),
servicePool,
(step: StepKey) => {
if (filesetKeys.has(step.value)) {
return;
}
if (foregroundServiceKey != null && step.value == foregroundServiceKey) {
spinner.stop();
return;
}
inFlight.push(step.value);
spinner.update(formatActiveSpinnerMessage(inFlight, completed, totalSteps));
},
(step: StepKey, stepOutcome: StepOutcome) => {
if (filesetKeys.has(step.value)) {
return;
}
completed += 1;
let remaining = new GrowableArray<String>();
for (let k in inFlight) {
if (k != step.value) {
remaining.push(k);
}
}
inFlight.clear();
for (let k in remaining) {
inFlight.push(k);
}
if (inFlight.length > 0) {
spinner.update(formatActiveSpinnerMessage(inFlight, completed, totalSteps));
} else {
spinner.stop();
}
},
),
);
} finally {
spinner.stop();
}
let runResult = result as RunResult;
let skipStateKey = if (hasExtraArgs) entryKey else foregroundServiceKey;
await saveState(root, runResult, skipStateKey);
await hasher.save();
let stepKeys = new GrowableArray<String>();
for (let step in plan.graph.steps) {
let k = step.key.value;
if (!filesetKeys.has(k)) {
stepKeys.push(k);
}
}
let durationMs = div((monotonic() as i64) - startNs, 1000000 as i64);
return new BuildOutcome(
runResult,
stepKeys,
entryKey,
durationMs,
plan.executions,
);
}
/**
* One full build of a script and its transitive dependencies: load
* config and prior state, snapshot inputs, run the engine with the
* local disk cache, persist the new state and hash memo.
*/
export async function buildTarget(
root: String,
entry: ScriptRef,
options: BuildOptions = {},
): Future<BuildOutcome> {
let {parallelism = 0, force = false, useCache = true} = options;
var remoteCache: StepCache? = null;
if (let {remoteCache as r} = options) {
remoteCache = r;
}
var lookupEnv: ((name: String) => String?)? = null;
if (let {lookupEnv as l} = options) {
lookupEnv = l;
}
var onCacheWarning: ((message: String) => void)? = null;
if (let {onCacheWarning as w} = options) {
onCacheWarning = w;
}
var hasherOut: FileHasher? = null;
if (let {hasherOut as h} = options) {
hasherOut = h;
}
var planStatsOut: PlanLoadStats? = null;
if (let {planStatsOut as s} = options) {
planStatsOut = s;
}
var extraArgs: Array<String>? = null;
if (let {extraArgs as a} = options) {
extraArgs = a;
}
var testRunner: TestStepRunner? = null;
if (let {testRunner as tr} = options) {
testRunner = tr;
}
return await runTargetOnce(root, entry, {
parallelism,
force,
useCache,
remoteCache,
lookupEnv,
onCacheWarning,
hasherOut,
servicePool: null,
allowForegroundService: true,
configPathsOut: null,
configsOut: null,
planStatsOut,
extraArgs,
testRunner,
});
}
/** Formats a Wireit-to-Zebra migration report on a Clack rail. */
export function formatMigrationReport(
migration: MigrationResult,
dryRun: boolean = false,
colors: Colors = new Colors(),
): String {
let layout = new RailLayout(colors);
let lines = new GrowableArray<String>();
let title = if (dryRun) 'migrate (dry run)' else 'migrate';
lines.push(layout.intro(`${layout.badge('zebra')} ${colors.dim(title)}`));
if (migration.written.length > 0 || migration.skippedExisting.length > 0) {
lines.push(layout.bar());
for (let item in migration.written) {
let action = if (dryRun) colors.cyan('preview') else colors.green('wrote');
lines.push(layout.step(StepKind.Success, `${colors.bold(item.path)}: ${action}`));
if (dryRun) {
for (let logLine in layout.logBlock(item.content)) {
lines.push(logLine);
}
}
}
for (let skippedPath in migration.skippedExisting) {
lines.push(layout.step(
StepKind.Warning,
`${colors.bold(skippedPath)}: ${colors.yellow('skipped (already exists; pass -f to overwrite)')}`,
));
}
}
lines.push(layout.bar());
let count = migration.written.length;
let footer = if (count > 0) {
let verb = if (dryRun) 'would write' else 'migrated';
let noun = if (count == 1) 'package' else 'packages';
colors.green(`${verb} ${i32ToString(count)} ${noun}`)
} else if (migration.skippedExisting.length > 0) {
colors.yellow(`0 packages written (${i32ToString(migration.skippedExisting.length)} existing skipped)`)
} else {
colors.dim('no wireit packages found to migrate')
};
lines.push(layout.outro(footer));
let sb = new StringBuilder();
for (var i = 0; i < lines.length; i += 1) {
if (i > 0) {
sb.append('\n');
}
sb.append(lines[i]);
}
return sb.toString();
}
/** Prints a migration report and returns exit code 0. */
export function reportMigration(
migration: MigrationResult,
dryRun: boolean = false,
colors: Colors = new Colors(),
): i32 {
console.log(formatMigrationReport(migration, dryRun, colors));
return 0;
}
async function snapshotWatchFiles(
root: String,
configPaths: Array<String>,
configs: Array<ScriptConfig>,
): Future<HashMap<String, String>> {
let hasher = new FileHasher(root);
await hasher.load();
let snapshot = new HashMap<String, String>();
for (let path in configPaths) {
let fp = await hasher.hashIfExists(path);
snapshot[path] = fp?.value ?? '$missing';
}
for (let config in configs) {
let lockCandidates = if (
config.isWireit &&
(config.command != null || config.tests != null) &&
config.fingerprintable &&
config.packageLocks.length > 0
)
lockfileCandidates(config.ref.packageDir, config.packageLocks)
else
new GrowableArray<String>();
if (config.files.length > 0 || lockCandidates.length > 0) {
for (let (path, fp) in await hasher.hashInputs(
config.ref.packageDir,
config.files,
lockCandidates,
)) {
snapshot[path] = fp.value;
}
}
}
return snapshot;
}
function snapshotsEqual(
a: HashMap<String, String>,
b: HashMap<String, String>,
) {
if (a.size != b.size) {
return false;
}
var same = true;
a.forEach((key, valA) => {
if (let (true, valB) = b.get(key)) {
if (valA != valB) {
same = false;
}
} else {
same = false;
}
});
return same;
}
/**
* Runs `entry` and watches its input files, lockfiles, and `package.json`
* manifests for changes, re-running incrementally and keeping services
* alive in a `ServicePool` across rebuilds.
*
* When `onIteration` is null (CLI watch mode), each completed iteration
* is reported with `reportBuild(outcome)` and watching continues until
* the process is terminated. When `onIteration` is supplied, it is
* called after each iteration's post-build file snapshot is captured;
* returning `false` stops watching and tears down all pinned services.
*/
export async function watchTarget(
root: String,
entry: ScriptRef,
options: WatchOptions = {},
): Future<BuildOutcome> {
let {
parallelism = 0,
force = false,
useCache = true,
pollIntervalMs = 150 as i64,
verbose = false,
} = options;
var onIteration: ((iteration: i32, outcome: BuildOutcome) => Future<boolean>)? = null;
if (let {onIteration as cb} = options) {
onIteration = cb;
}
var remoteCache: StepCache? = null;
if (let {remoteCache as r} = options) {
remoteCache = r;
}
var lookupEnv: ((name: String) => String?)? = null;
if (let {lookupEnv as l} = options) {
lookupEnv = l;
}
var onCacheWarning: ((message: String) => void)? = null;
if (let {onCacheWarning as w} = options) {
onCacheWarning = w;
}
var extraArgs: Array<String>? = null;
if (let {extraArgs as a} = options) {
extraArgs = a;
}
var testRunner: TestStepRunner? = null;
if (let {testRunner as tr} = options) {
testRunner = tr;
}
listenSignals();
let servicePool = new ServicePool();
let interval = if (pollIntervalMs > (0 as i64)) pollIntervalMs else 50 as i64;
var iteration = 0;
try {
while (true) {
let configPaths = new GrowableArray<String>();
let configs = new GrowableArray<ScriptConfig>();
let outcome = await runTargetOnce(root, entry, {
parallelism,
force,
useCache,
remoteCache,
lookupEnv,
onCacheWarning,
hasherOut: null,
servicePool,
allowForegroundService: false,
configPathsOut: configPaths,
configsOut: configs,
planStatsOut: null,
extraArgs,
testRunner,
});
if (checkSignal() != 0) {
return outcome;
}
iteration += 1;
let baseline = await snapshotWatchFiles(root, configPaths, configs);
if (onIteration != null) {
if (!(await onIteration(iteration, outcome))) {
return outcome;
}
} else {
reportBuild(outcome, verbose);
}
while (true) {
if (checkSignal() != 0) {
return outcome;
}
let current = await snapshotWatchFiles(root, configPaths, configs);
if (!snapshotsEqual(baseline, current)) {
break;
}
if ((await waitSignalFor(interval)) != 0) {
return outcome;
}
}
}
} finally {
await servicePool.stopAll();
}
}
| Version | Published | Dependencies | Approved by |
|---|---|---|---|
| 0.0.1-preview.1 | 2026-11-19 |
|