zebra 0.0.1-preview.1

Zebra: an incremental build orchestrator for Zena and standalone workspaces

Readme Files Dependencies Versions

zebra#

An incremental script and service runner for Zena (zena-package.json), standalone Zebra (zebra.json), and Wireit (package.json) workspaces, built on zena-lang:workflow.

Zebra runs a dependency graph of scripts and long-running services with declared inputs, outputs, and dependencies, rerunning only what changed. For each script, Zebra content-hashes the declared inputs and asks the workflow engine what to do: nothing (when the previous run recorded the same fingerprints and declared outputs are intact), restore from the local content-addressed cache, or execute the command (sh -c in the package directory with node_modules/.bin on PATH).

Installation#

zebra is built into the zena CLI (zena build <target>). To use it as a library or standalone runner:

zena pkg add zena-lang:zebra

CLI Usage#

zena build [-j <jobs>] [-w|--watch] [-f|--force] [--no-cache] [-v|--verbose] [<target>] [-- <args>...]
zena run [-j <jobs>] [-w|--watch] [-f|--force] [--no-cache] [-v|--verbose] <target> [-- <args>...]
zena test [-j <jobs>] [-w|--watch] [--force] [--no-cache] [-v|--verbose] [-- <args>...]
zena build --migrate [--dry-run] [-f|--force] [<package-dir>...]
  • <target> can be a script name in the current package (zena build test, zena run check), a package directory (zena build ./packages/app, which runs build), or <package-dir>:<script> (zena build ./packages/app:test). With no <target>, zena build runs the current package's build script and zena test runs its test script.
  • -- <args>... appends shell-quoted arguments to the entry target's command only, running that entry step uncached without overwriting its saved freshness record.
  • -j / --jobs <n> bounds parallel command execution (defaulting to ZEBRA_PARALLEL, WIREIT_PARALLEL, or ZENA_AVAILABLE_PARALLELISM).
  • -w / --watch watches input globs, lockfiles, and package manifests for changes, re-running affected steps incrementally while keeping unchanged services alive in a ServicePool across rebuilds.
  • -f / --force bypasses freshness and cached outputs to rerun scripts unconditionally.
  • --no-cache disables restoring outputs from .zebra/blobs/ when a step is not fresh.
  • -v / --verbose prints every step (including fresh and skipped) and any captured stdout/stderr on the rail.
  • --migrate converts Wireit blocks in package.json into zena-package.json (when present) or zebra.json.

Programmatic Usage#

import { buildTarget, reportBuild } from 'zena-lang:zebra/runner';
import { ScriptRef } from 'zena-lang:zebra/wireit-config';

let outcome = await buildTarget('.', new ScriptRef('packages/app', 'build'), {
  parallelism: 4,
});
let exitCode = reportBuild(outcome);

Configuration (zena-package.json and zebra.json)#

Zena packages configure scripts and filesets directly in zena-package.json. Non-Zena projects can use a standalone zebra.json (with an optional "workspace": {"packages": [...]} at the workspace root). Both support // and /* ... */ comments:

{
  "name": "app",
  "dependencies": {
    "lib": {"path": "../lib"},
  },
  "filesets": {
    "assets": ["assets/**"],
  },
  "scripts": {
    "build": {
      "command": "zena build src/main.zena -o out/app.wasm",
      "output": ["out/app.wasm"],
      "dependencies": ["source", "assets"],
    },
    "test": {
      "type": "zena:test",
      "tests": "test/*_test.zena",
    },
    "fmt": "zena fmt --check .",
  },
}
  • Declarative zena:test steps and default test target: Scripts in zena-package.json and zebra.json can set "type": "zena:test" (with optional "tests": true, "tests": "test/*_test.zena", or "tests": ["test/*_test.zena", ...] — defaulting to ["test/**/*_test.zena", "tests/**/*_test.zena"] when omitted — and optional "debug": true for -g) to run Zena tests in-process through TestStepRunner. Test pattern globs merge into files, and zena-package.json "zena:test" steps automatically depend on source. When a zena-package.json omits a test script (and no fallback package.json defines test), Zebra synthesizes a "type": "zena:test" target over test/**/*_test.zena and tests/**/*_test.zena.
  • Automatic source fileset: Every package with zena-package.json exposes an automatic source fileset covering its library .zena files (from "source", defaulting to "src" and excluding *_test.zena, test/**, and tests/**), zena-package.json, any "wit" dependencies, and the source filesets of all "path" dependencies. Scripts opt into it via "dependencies": ["source"] (or "../pkg:source").
  • Named filesets: Entries accept either a glob array shorthand ("wit": ["wit/**/*.wit"]) or {"files": [...], "dependencies": [...]} and compose transitively without spawning commands.
  • Caching and tracking: Task scripts cache automatically whenever files, output, or a fileset dependency is declared, defaulting output to [] when omitted. Setting "cache": false opts a step out of freshness and caching while keeping its files globs active in --watch mode. Omitting files, output, and fileset dependencies on a command step leaves it untracked so it runs every time. String entries ("fmt": "zena fmt --check .") are shorthands for always-run commands.
  • Locks: locks defaults to ["zena-lock.json"] and may be set at the package root or overridden per script ("locks": [] disables lockfile hashing).
  • Services: Scripts with "service": true or "service": {"readyWhen": {"lineMatches": "..."}} start on demand when a dependent executes, wait for readiness when configured, are shared across dependents, and are stopped as a Unix process group (SIGTERM then SIGKILL) when dependents finish. Invoking a service directly as the top-level target runs its dependencies and streams the service in the foreground.
  • Wireit fallback: When a package directory has no zebra.json and no "scripts" block in zena-package.json, Zebra falls back to package.json Wireit scripts while still serving filesets from zena-package.json. Dependencies can cross freely between zena-package.json, zebra.json, and package.json packages.

Cache Layout (.zebra/)#

Each package's <pkg>/.zebra/ directory stores:

  • state.json — freshness records for that package's scripts (the last cache key and output manifest).
  • cache/<key>.json — cache key to output manifest.
  • blobs/ — content-addressed output bytes (sha256).

The workspace root's .zebra/ directory stores:

  • hashes.json — the (mtime, size) to sha256 file hash memo shared across packages (with a 20ms racy-timestamp guard).
  • plans/<hash>.plan — stat-verified cached ScriptConfig closures keyed by target entry.

A dependency contributes the digest of its outputs to dependents' cache keys rather than its input fingerprint, giving early cutoff when a dependency reruns and produces identical outputs.

GitHub Actions Remote Cache#

Setting ZEBRA_CACHE=github layers GitHubActionsCache behind the local .zebra/ DiskCache (TieredCache). On a local cache miss, Zebra queries the GitHub Actions v2 CacheService Twirp JSON API, verifies downloaded blob sha256 digests, populates <pkg>/.zebra/, and materializes outputs into the workspace. On a build step execution, Zebra writes <pkg>/.zebra/ first and uploads the step's entry to GitHub Actions Cache (ZEBRA_CACHE_READ_ONLY=1 disables uploads; ZEBRA_CACHE=none disables both local and remote caching).

In GitHub Actions workflows, run .github/actions/setup-zebra-cache before build steps so ACTIONS_RESULTS_URL and ACTIONS_RUNTIME_TOKEN are served to run: steps over a 127.0.0.1 token custodian (ZEBRA_GITHUB_CUSTODIAN_URL).

Libraries#

  • zebra (zebra/main) — CLI entry point (main).
  • zebra/runner — buildTarget, watchTarget, BuildOptions, WatchOptions, reportBuild, reportMigration, formatBuildReport, and BuildOutcome.
  • zebra/zebra-config — zena-package.json and zebra.json parsing (ZebraWorkspace), synthesized source filesets, and Wireit migration (migrateWorkspaceToZebra, convertPackageJsonToZebraJson, mergeWireitIntoZenaPackageJson).
  • zebra/wireit-config — package.json Wireit parsing (WireitWorkspace) and shared script configuration types (ScriptConfig, ScriptRef, DependencySpec, ServiceConfig, EnvBinding, ConfigError, refKey, resolveDependency).
  • zebra/plan-cache — .zebra/plans/ snapshot serialization and stat-verified warm plan loading (loadOrCollectConfigs, PlanLoadStats, ConfigProbe).
  • zebra/builder — Translates ScriptConfig closures into workflow steps and execution callbacks (planScripts, StepExecution, lockfileCandidates).
  • zebra/disk-cache, zebra/github-cache, and zebra/state-json — .zebra/ blob/manifest storage (DiskCache, TieredCache, stateDirOf), GitHub Actions remote caching (GitHubActionsCache, GitHubCacheConfig, CacheMode, resolveCacheMode, resolveGitHubCacheConfig), and JSON serialization for state, manifests, and hash memos.
  • zebra/fs-walk and zebra/fs-util — Input glob discovery (FileHasher, expandGlobs, hashFile, validateGlobs, splitBase) and workspace path utilities.
  • zebra/shell — POSIX shell argument quoting (shellQuote, appendShellArgs). Zebra's terminal output uses zena-lang:terminal (packages/terminal): colors, the Clack-style rail layout and the spinner.

License#

MIT

27.5 KiB Raw
import {Future} from 'zena:async';
import {getEnv} from 'zena:cli';
import {HashSet} from 'zena:collections';
import {StringBuilder} from 'zena:core';
import {isFile, readFile, writeFile} from 'zena:fs';
import {div} from 'zena:math';
import {checkSignal, listenSignals, waitSignalFor} from 'zena:process';
import {monotonic} from 'zena:time';
import {NullCache, StepCache} from 'workflow/cache';
import {
  Engine,
  Failed,
  RunOptions,
  RunResult,
  ServicePool,
  Skipped,
  StepOutcome,
  StepSource,
  Succeeded,
} from 'workflow/engine';
import {Fingerprint} from 'workflow/fingerprint';
import {OutputManifest} from 'workflow/manifest';
import {EngineState, StepRecord} from 'workflow/state';
import {StepKey} from 'workflow/step';
import {Spinner} from 'terminal/animation';
import {
  StepExecution,
  TestStepRequest,
  TestStepRunner,
  lockfileCandidates,
  planScripts,
} from './builder.zena';
import {Colors} from 'terminal/colors';
import {DiskCache, TieredCache, stateDirOf} from './disk-cache.zena';
import {ensureParentDir, withRoot} from './fs-util.zena';
import {FileHasher} from './fs-walk.zena';
import {
  CacheMode,
  GitHubActionsCache,
  resolveCacheMode,
  resolveGitHubCacheConfig,
} from './github-cache.zena';
import {RailLayout, StepKind, formatDuration} from 'terminal/layout';
import {PlanLoadStats, loadOrCollectConfigs} from './plan-cache.zena';
import {sha256HexOfString} from 'zena:crypto';
import {appendShellArgs} from './shell.zena';
import {stateFromJson, stateToJson} from './state-json.zena';
import {ConfigError, ScriptConfig, ScriptRef, refKey} from './wireit-config.zena';
import {MigrationResult} from './zebra-config.zena';

export {TestStepRequest, TestStepRunner} from './builder.zena';

// State is stored per package ('<pkg>/.zebra/state.json'), like wireit's
// .wireit/, so its location does not depend on the invocation
// directory.

function packageStatePath(root: String, stateDir: String) {
  return withRoot(root, stateDir + '/state.json');
}

async function loadState(root: String, stateDirs: Array<String>) {
  let state = new EngineState();
  for (let stateDir in stateDirs) {
    let path = packageStatePath(root, stateDir);
    if (!(await isFile(path))) {
      continue;
    }
    stateFromJson(await readFile(path)).forEach((key, record) => {
      state.setRecord(new StepKey(key), record);
    });
  }
  return state;
}

/**
 * Writes each package's records back, merged over what its state file
 * already holds — a run of one target must not drop a package's
 * records for scripts outside the run's closure.
 */
async function saveState(root: String, result: RunResult, skipKey: String? = null) {
  let entries = new GrowableArray<(String, StepRecord)>();
  let dirs = new GrowableArray<String>();
  let seenDirs = new HashSet<String>();
  result.state.forEach((key, record) => {
    if (skipKey != null && key == skipKey) {
      return;
    }
    entries.push((key, record));
    let stateDir = stateDirOf(key);
    if (seenDirs.add(stateDir)) {
      dirs.push(stateDir);
    }
  });
  let byDir = new HashMap<String, EngineState>();
  for (let stateDir in dirs) {
    let path = packageStatePath(root, stateDir);
    byDir[stateDir] = if (await isFile(path))
      stateFromJson(await readFile(path))
    else
      EngineState.empty();
  }
  for (let (key, record) in entries) {
    let stateDir = stateDirOf(key);
    if (let (true, packageState) = byDir.get(stateDir)) {
      packageState.setRecord(new StepKey(key), record);
    }
  }
  for (let stateDir in dirs) {
    if (let (true, packageState) = byDir.get(stateDir)) {
      let path = packageStatePath(root, stateDir);
      await ensureParentDir(path);
      await writeFile(path, stateToJson(packageState));
    }
  }
}

function joinComma(parts: GrowableArray<String>) {
  let sb = new StringBuilder();
  for (var i = 0; i < parts.length; i += 1) {
    if (i > 0) {
      sb.append(', ');
    }
    sb.append(parts[i]);
  }
  return sb.toString();
}

function formatActiveSpinnerMessage(
  inFlight: GrowableArray<String>,
  completed: i32,
  total: i32,
) {
  if (inFlight.length == 0) {
    return `Building (${i32ToString(completed)}/${i32ToString(total)})...`;
  }
  let first = inFlight[0];
  let more = if (inFlight.length > 1)
    ` (+${i32ToString(inFlight.length - 1)} more)`
  else
    '';
  return `Running ${first}${more} (${i32ToString(completed)}/${i32ToString(total)})...`;
}

/**
 * Formats a finished build as a Clack-style rail report (`┌`, `│`, `└`)
 * with step status symbols (`◇`, `◆`, `○`, `▲`, `■`), rail-indented
 * command output blocks, and a duration/count summary in the footer.
 *
 * By default, `ran`, `restored`, and `failed` steps are listed on the
 * rail while `fresh` and `skipped` steps are summarized in the footer;
 * passing `verbose = true` lists every step on the rail and prints
 * captured stdout/stderr for passing steps too.
 */
export function formatBuildReport(
  outcome: BuildOutcome,
  verbose: boolean = false,
  colors: Colors = new Colors(),
): String {
  let layout = new RailLayout(colors);
  let lines = new GrowableArray<String>();
  let title = if (outcome.target != '')
    `${layout.badge('zebra')} ${colors.bold(outcome.target)}`
  else
    layout.badge('zebra');
  lines.push(layout.intro(title));

  let result = outcome.result;
  var ranCount = 0;
  var restoredCount = 0;
  var freshCount = 0;
  var failedCount = 0;
  var skippedCount = 0;

  for (let key in outcome.stepKeys) {
    let stepOutcome = result.outcomeOf(new StepKey(key));
    if (stepOutcome is Succeeded) {
      if (stepOutcome.source == StepSource.Fresh) {
        freshCount += 1;
        if (verbose) {
          lines.push(layout.bar());
          lines.push(layout.step(StepKind.Fresh, `${key}: fresh`));
        }
      } else if (stepOutcome.source == StepSource.Cache) {
        restoredCount += 1;
        lines.push(layout.bar());
        lines.push(layout.step(StepKind.Active, `${key}: restored`));
      } else {
        ranCount += 1;
        lines.push(layout.bar());
        var detail = '';
        var output = '';
        if (let (true, exec) = outcome.executions.get(key)) {
          detail = `(${formatDuration(exec.durationMs)})`;
          output = exec.combinedOutput();
        }
        lines.push(layout.step(StepKind.Success, `${key}: ran`, detail));
        if (verbose && output != '') {
          for (let logLine in layout.logBlock(output, false)) {
            lines.push(logLine);
          }
        }
      }
    } else if (stepOutcome is Failed) {
      failedCount += 1;
      lines.push(layout.bar());
      if (let (true, exec) = outcome.executions.get(key)) {
        let detail = `(${formatDuration(exec.durationMs)})`;
        lines.push(
          layout.step(
            StepKind.Error,
            `${key}: failed (exit ${i32ToString(exec.exitCode)})`,
            detail,
          ),
        );
        let combined = exec.combinedOutput();
        let blockText = if (combined != '')
          `$ ${exec.command}\n${combined}`
        else
          `$ ${exec.command}`;
        for (let logLine in layout.logBlock(blockText, true)) {
          lines.push(logLine);
        }
      } else {
        lines.push(layout.step(StepKind.Error, `${key}: failed`));
        for (let logLine in layout.logBlock(stepOutcome.error.message, true)) {
          lines.push(logLine);
        }
      }
    } else if (stepOutcome is Skipped) {
      skippedCount += 1;
      if (verbose) {
        lines.push(layout.bar());
        lines.push(
          layout.step(StepKind.Warning, `${key}: skipped (${stepOutcome.reason})`),
        );
      }
    }
  }

  lines.push(layout.bar());
  let counts = new GrowableArray<String>();
  if (failedCount > 0) {
    counts.push(`${i32ToString(failedCount)} failed`);
  }
  if (skippedCount > 0) {
    counts.push(`${i32ToString(skippedCount)} skipped`);
  }
  if (ranCount > 0) {
    counts.push(`${i32ToString(ranCount)} ran`);
  }
  if (restoredCount > 0) {
    counts.push(`${i32ToString(restoredCount)} restored`);
  }
  if (freshCount > 0) {
    counts.push(`${i32ToString(freshCount)} fresh`);
  }
  let status = if (result.allSucceeded)
    colors.green('build succeeded')
  else
    colors.red('build failed');
  let elapsed = colors.dim(`in ${formatDuration(outcome.durationMs)}`);
  let summary = if (counts.length > 0)
    `${status} ${elapsed} ${colors.dim(`(${joinComma(counts)})`)}`
  else
    `${status} ${elapsed}`;
  lines.push(layout.outro(summary));

  let sb = new StringBuilder();
  for (var i = 0; i < lines.length; i += 1) {
    if (i > 0) {
      sb.append('\n');
    }
    sb.append(lines[i]);
  }
  return sb.toString();
}

/**
 * Prints a Clack-style build report and returns the exit status: 0 when
 * every step succeeded, 1 otherwise (or `128 + sig` if interrupted).
 * `zebra` and `zena build <target>` both report this way.
 */
export function reportBuild(
  outcome: BuildOutcome,
  verbose: boolean = false,
  colors: Colors = new Colors(),
): i32 {
  let sig = checkSignal();
  if (sig != 0) {
    return 128 + sig;
  }
  console.log(formatBuildReport(outcome, verbose, colors));
  if (!outcome.result.allSucceeded) {
    return 1;
  }
  return 0;
}

/**
 * A finished build: the engine result plus the step keys in dependency
 * order, target label, wall-clock duration, and captured step outputs.
 */
export class BuildOutcome {
  result: RunResult;
  stepKeys: Array<String>;
  target: String;
  durationMs: i64;
  executions: HashMap<String, StepExecution>;

  new(
    this.result,
    this.stepKeys,
    this.target = '',
    this.durationMs = 0 as i64,
    this.executions = new HashMap<String, StepExecution>(),
  );
}

/**
 * Default max concurrent script executions: ZEBRA_PARALLEL, WIREIT_PARALLEL,
 * ZENA_AVAILABLE_PARALLELISM, or 8.
 */
export function defaultParallelism(
  lookupEnv: (name: String) => String? = getEnv,
): i32 {
  for (let varName in [
    'ZEBRA_PARALLEL',
    'WIREIT_PARALLEL',
    'ZENA_AVAILABLE_PARALLELISM',
  ]) {
    let raw = lookupEnv(varName);
    if (raw != null) {
      if (let (true, n) = parseI32(raw)) {
        if (n >= 1) {
          return n;
        }
      }
    }
  }
  return 8;
}

async function verifyManifestOutputs(
  hasher: FileHasher,
  manifest: OutputManifest,
): Future<boolean> {
  for (let entry in manifest.outputs) {
    if (!entry.fingerprint.value.startsWith('sha256:')) {
      continue;
    }
    let current = await hasher.hashIfExists(entry.name);
    if (current == null || current.value != entry.fingerprint.value) {
      return false;
    }
  }
  return true;
}

/** Options for `buildTarget`. Every field is optional. */
export type BuildOptions = {
  /** Maximum concurrent script executions (`<= 0` uses `defaultParallelism()`). */
  parallelism?: i32,
  /** Bypass freshness and cached outputs to rerun every reachable script. */
  force?: boolean,
  /** Read and write outputs through the local `.zebra/` disk cache (`true` if absent). */
  useCache?: boolean,
  /** Optional second-level cache layered behind the local `.zebra/` disk cache. */
  remoteCache?: StepCache?,
  /** Optional environment lookup override (`getEnv` if absent). */
  lookupEnv?: ((name: String) => String?)?,
  /** Optional sink for non-fatal remote cache warnings (`console.error` if absent). */
  onCacheWarning?: ((message: String) => void)?,
  /** Optional caller-supplied `FileHasher` for inspecting hash and stat counters. */
  hasherOut?: FileHasher?,
  /** Optional sink populated with warm/cold plan snapshot load statistics. */
  planStatsOut?: PlanLoadStats?,
  /** Extra CLI arguments after `--` appended to the entry target's command or `"tests"` step. */
  extraArgs?: Array<String>?,
  /** Optional in-process executor for native Zena `"tests"` steps. */
  testRunner?: TestStepRunner?,
};

/** Options for `watchTarget`. Every field is optional. */
export type WatchOptions = {
  /** Maximum concurrent script executions (`<= 0` uses `defaultParallelism()`). */
  parallelism?: i32,
  /** Bypass freshness and cached outputs on each rebuild. */
  force?: boolean,
  /** Read and write outputs through the local `.zebra/` disk cache (`true` if absent). */
  useCache?: boolean,
  /** Optional second-level cache layered behind the local `.zebra/` disk cache. */
  remoteCache?: StepCache?,
  /** Optional environment lookup override (`getEnv` if absent). */
  lookupEnv?: ((name: String) => String?)?,
  /** Optional sink for non-fatal remote cache warnings (`console.error` if absent). */
  onCacheWarning?: ((message: String) => void)?,
  /** Polling interval between file-snapshot checks in milliseconds (`150` if absent). */
  pollIntervalMs?: i64,
  /**
   * Callback invoked after each iteration's post-build snapshot is captured.
   * Returning `false` stops watching and tears down pinned services.
   * When omitted, each iteration prints `reportBuild(outcome, verbose)`.
   */
  onIteration?: ((iteration: i32, outcome: BuildOutcome) => Future<boolean>)?,
  /** Print `fresh` and `skipped` steps and passing step output on each watch report. */
  verbose?: boolean,
  /** Extra CLI arguments after `--` appended to the entry target's command or `"tests"` step. */
  extraArgs?: Array<String>?,
  /** Optional in-process executor for native Zena `"tests"` steps. */
  testRunner?: TestStepRunner?,
};

type RunTargetOptions = {
  parallelism: i32,
  force: boolean,
  useCache: boolean,
  remoteCache: StepCache?,
  lookupEnv: ((name: String) => String?)?,
  onCacheWarning: ((message: String) => void)?,
  hasherOut: FileHasher?,
  servicePool: ServicePool?,
  allowForegroundService: boolean,
  configPathsOut: GrowableArray<String>?,
  configsOut: GrowableArray<ScriptConfig>?,
  planStatsOut: PlanLoadStats?,
  extraArgs: Array<String>?,
  testRunner: TestStepRunner?,
};

async function runTargetOnce(
  root: String,
  entry: ScriptRef,
  options: RunTargetOptions,
) {
  let {
    parallelism,
    force,
    useCache,
    remoteCache,
    lookupEnv,
    onCacheWarning,
    hasherOut,
    servicePool,
    allowForegroundService,
    configPathsOut,
    configsOut,
    planStatsOut,
    extraArgs,
    testRunner,
  } = options;
  listenSignals();
  let startNs = monotonic() as i64;
  let envLookup = lookupEnv ?? getEnv;
  let jobs = if (parallelism > 0) parallelism else defaultParallelism(envLookup);
  let entryKey = refKey(entry);
  let loadedConfigs = await loadOrCollectConfigs(
    root,
    entry,
    configPathsOut,
    planStatsOut,
  );
  let hasExtraArgs = extraArgs != null && extraArgs.length > 0;
  var configs = loadedConfigs;
  if (hasExtraArgs) {
    let cliExtra = extraArgs as Array<String>;
    let rewritten = new GrowableArray<ScriptConfig>();
    for (let config in loadedConfigs) {
      if (refKey(config.ref) == entryKey) {
        let baseCommand = config.command;
        let baseTests = config.tests;
        if (config.isFileset || (baseCommand == null && baseTests == null)) {
          throw new ConfigError(
            `script "${entryKey}" has no command and cannot accept extra arguments after --`,
          );
        }
        rewritten.push(
          new ScriptConfig(
            config.ref,
            config.isWireit,
            if (baseCommand != null) appendShellArgs(baseCommand, cliExtra) else null,
            config.files,
            config.output,
            config.filesDeclared,
            config.outputDeclared,
            config.dependencies,
            config.clean,
            config.service,
            config.env,
            config.packageLocks,
            false,
            config.isFileset,
            baseTests,
            config.testDebug,
            if (baseTests != null) cliExtra else null,
            config.ruleType,
          ),
        );
      } else {
        rewritten.push(config);
      }
    }
    configs = rewritten;
  }
  if (configsOut != null) {
    for (let config in configs) {
      configsOut.push(config);
    }
  }
  let foregroundServiceKey = if (
    allowForegroundService &&
    configs.length > 0 &&
    configs[0].service != null
  )
    entryKey
  else
    null;
  let hasher = hasherOut ?? new FileHasher(root);
  await hasher.load();
  let plan = planScripts(
    root,
    configs,
    i64ToString(monotonic() as i64),
    hasher,
    foregroundServiceKey,
    testRunner,
  );
  let stateDirs = new GrowableArray<String>();
  let seenDirs = new HashSet<String>();
  let uncachedSteps = new HashSet<String>();
  for (let config in configs) {
    let k = refKey(config.ref);
    let stateDir = stateDirOf(k);
    if (seenDirs.add(stateDir)) {
      stateDirs.push(stateDir);
    }
    if (!config.fingerprintable || config.isFileset) {
      uncachedSteps.add(k);
    }
  }
  if (foregroundServiceKey != null) {
    uncachedSteps.add(foregroundServiceKey);
  }
  if (hasExtraArgs) {
    uncachedSteps.add(entryKey);
  }
  var cache: StepCache = new NullCache();
  if (useCache) {
    let mode = if (remoteCache != null)
      CacheMode.Local
    else
      resolveCacheMode(envLookup);
    if (mode != CacheMode.None) {
      let diskCache = new DiskCache(
        root,
        () => {
          hasher.markMutated();
        },
        (path, fp) => hasher.recordRestored(path, fp),
        uncachedSteps,
      );
      if (remoteCache != null) {
        cache = new TieredCache(diskCache, remoteCache, uncachedSteps);
      } else if (mode == CacheMode.GitHub) {
        let ghConfig = await resolveGitHubCacheConfig(envLookup);
        let warn = onCacheWarning ?? ((msg: String) => {
          console.error(`zebra: ${msg}`);
        });
        let ghCache = new GitHubActionsCache(root, diskCache, ghConfig, warn);
        cache = new TieredCache(diskCache, ghCache, uncachedSteps);
      } else {
        cache = diskCache;
      }
    }
  }
  let engine = new Engine(
    (data: String) => new Fingerprint('sha256:' + sha256HexOfString(data)),
    cache,
  );
  let spinner = new Spinner();
  let inFlight = new GrowableArray<String>();
  var completed = 0;
  let filesetKeys = new HashSet<String>();
  for (let config in configs) {
    let k = refKey(config.ref);
    if (config.isFileset && k != entryKey) {
      filesetKeys.add(k);
    }
  }
  let totalSteps = configs.length - filesetKeys.size;
  var result: RunResult | null = null;
  try {
    result = await engine.run(
      plan.graph,
      plan.leaves,
      await loadState(root, stateDirs),
      new RunOptions(
        jobs,
        force,
        (step, manifest) => verifyManifestOutputs(hasher, manifest),
        servicePool,
        (step: StepKey) => {
          if (filesetKeys.has(step.value)) {
            return;
          }
          if (foregroundServiceKey != null && step.value == foregroundServiceKey) {
            spinner.stop();
            return;
          }
          inFlight.push(step.value);
          spinner.update(formatActiveSpinnerMessage(inFlight, completed, totalSteps));
        },
        (step: StepKey, stepOutcome: StepOutcome) => {
          if (filesetKeys.has(step.value)) {
            return;
          }
          completed += 1;
          let remaining = new GrowableArray<String>();
          for (let k in inFlight) {
            if (k != step.value) {
              remaining.push(k);
            }
          }
          inFlight.clear();
          for (let k in remaining) {
            inFlight.push(k);
          }
          if (inFlight.length > 0) {
            spinner.update(formatActiveSpinnerMessage(inFlight, completed, totalSteps));
          } else {
            spinner.stop();
          }
        },
      ),
    );
  } finally {
    spinner.stop();
  }
  let runResult = result as RunResult;
  let skipStateKey = if (hasExtraArgs) entryKey else foregroundServiceKey;
  await saveState(root, runResult, skipStateKey);
  await hasher.save();
  let stepKeys = new GrowableArray<String>();
  for (let step in plan.graph.steps) {
    let k = step.key.value;
    if (!filesetKeys.has(k)) {
      stepKeys.push(k);
    }
  }
  let durationMs = div((monotonic() as i64) - startNs, 1000000 as i64);
  return new BuildOutcome(
    runResult,
    stepKeys,
    entryKey,
    durationMs,
    plan.executions,
  );
}

/**
 * One full build of a script and its transitive dependencies: load
 * config and prior state, snapshot inputs, run the engine with the
 * local disk cache, persist the new state and hash memo.
 */
export async function buildTarget(
  root: String,
  entry: ScriptRef,
  options: BuildOptions = {},
): Future<BuildOutcome> {
  let {parallelism = 0, force = false, useCache = true} = options;
  var remoteCache: StepCache? = null;
  if (let {remoteCache as r} = options) {
    remoteCache = r;
  }
  var lookupEnv: ((name: String) => String?)? = null;
  if (let {lookupEnv as l} = options) {
    lookupEnv = l;
  }
  var onCacheWarning: ((message: String) => void)? = null;
  if (let {onCacheWarning as w} = options) {
    onCacheWarning = w;
  }
  var hasherOut: FileHasher? = null;
  if (let {hasherOut as h} = options) {
    hasherOut = h;
  }
  var planStatsOut: PlanLoadStats? = null;
  if (let {planStatsOut as s} = options) {
    planStatsOut = s;
  }
  var extraArgs: Array<String>? = null;
  if (let {extraArgs as a} = options) {
    extraArgs = a;
  }
  var testRunner: TestStepRunner? = null;
  if (let {testRunner as tr} = options) {
    testRunner = tr;
  }
  return await runTargetOnce(root, entry, {
    parallelism,
    force,
    useCache,
    remoteCache,
    lookupEnv,
    onCacheWarning,
    hasherOut,
    servicePool: null,
    allowForegroundService: true,
    configPathsOut: null,
    configsOut: null,
    planStatsOut,
    extraArgs,
    testRunner,
  });
}

/** Formats a Wireit-to-Zebra migration report on a Clack rail. */
export function formatMigrationReport(
  migration: MigrationResult,
  dryRun: boolean = false,
  colors: Colors = new Colors(),
): String {
  let layout = new RailLayout(colors);
  let lines = new GrowableArray<String>();
  let title = if (dryRun) 'migrate (dry run)' else 'migrate';
  lines.push(layout.intro(`${layout.badge('zebra')}  ${colors.dim(title)}`));

  if (migration.written.length > 0 || migration.skippedExisting.length > 0) {
    lines.push(layout.bar());
    for (let item in migration.written) {
      let action = if (dryRun) colors.cyan('preview') else colors.green('wrote');
      lines.push(layout.step(StepKind.Success, `${colors.bold(item.path)}: ${action}`));
      if (dryRun) {
        for (let logLine in layout.logBlock(item.content)) {
          lines.push(logLine);
        }
      }
    }
    for (let skippedPath in migration.skippedExisting) {
      lines.push(layout.step(
        StepKind.Warning,
        `${colors.bold(skippedPath)}: ${colors.yellow('skipped (already exists; pass -f to overwrite)')}`,
      ));
    }
  }

  lines.push(layout.bar());
  let count = migration.written.length;
  let footer = if (count > 0) {
    let verb = if (dryRun) 'would write' else 'migrated';
    let noun = if (count == 1) 'package' else 'packages';
    colors.green(`${verb} ${i32ToString(count)} ${noun}`)
  } else if (migration.skippedExisting.length > 0) {
    colors.yellow(`0 packages written (${i32ToString(migration.skippedExisting.length)} existing skipped)`)
  } else {
    colors.dim('no wireit packages found to migrate')
  };
  lines.push(layout.outro(footer));

  let sb = new StringBuilder();
  for (var i = 0; i < lines.length; i += 1) {
    if (i > 0) {
      sb.append('\n');
    }
    sb.append(lines[i]);
  }
  return sb.toString();
}

/** Prints a migration report and returns exit code 0. */
export function reportMigration(
  migration: MigrationResult,
  dryRun: boolean = false,
  colors: Colors = new Colors(),
): i32 {
  console.log(formatMigrationReport(migration, dryRun, colors));
  return 0;
}

async function snapshotWatchFiles(
  root: String,
  configPaths: Array<String>,
  configs: Array<ScriptConfig>,
): Future<HashMap<String, String>> {
  let hasher = new FileHasher(root);
  await hasher.load();
  let snapshot = new HashMap<String, String>();
  for (let path in configPaths) {
    let fp = await hasher.hashIfExists(path);
    snapshot[path] = fp?.value ?? '$missing';
  }
  for (let config in configs) {
    let lockCandidates = if (
      config.isWireit &&
      (config.command != null || config.tests != null) &&
      config.fingerprintable &&
      config.packageLocks.length > 0
    )
      lockfileCandidates(config.ref.packageDir, config.packageLocks)
    else
      new GrowableArray<String>();
    if (config.files.length > 0 || lockCandidates.length > 0) {
      for (let (path, fp) in await hasher.hashInputs(
        config.ref.packageDir,
        config.files,
        lockCandidates,
      )) {
        snapshot[path] = fp.value;
      }
    }
  }
  return snapshot;
}

function snapshotsEqual(
  a: HashMap<String, String>,
  b: HashMap<String, String>,
) {
  if (a.size != b.size) {
    return false;
  }
  var same = true;
  a.forEach((key, valA) => {
    if (let (true, valB) = b.get(key)) {
      if (valA != valB) {
        same = false;
      }
    } else {
      same = false;
    }
  });
  return same;
}

/**
 * Runs `entry` and watches its input files, lockfiles, and `package.json`
 * manifests for changes, re-running incrementally and keeping services
 * alive in a `ServicePool` across rebuilds.
 *
 * When `onIteration` is null (CLI watch mode), each completed iteration
 * is reported with `reportBuild(outcome)` and watching continues until
 * the process is terminated. When `onIteration` is supplied, it is
 * called after each iteration's post-build file snapshot is captured;
 * returning `false` stops watching and tears down all pinned services.
 */
export async function watchTarget(
  root: String,
  entry: ScriptRef,
  options: WatchOptions = {},
): Future<BuildOutcome> {
  let {
    parallelism = 0,
    force = false,
    useCache = true,
    pollIntervalMs = 150 as i64,
    verbose = false,
  } = options;
  var onIteration: ((iteration: i32, outcome: BuildOutcome) => Future<boolean>)? = null;
  if (let {onIteration as cb} = options) {
    onIteration = cb;
  }
  var remoteCache: StepCache? = null;
  if (let {remoteCache as r} = options) {
    remoteCache = r;
  }
  var lookupEnv: ((name: String) => String?)? = null;
  if (let {lookupEnv as l} = options) {
    lookupEnv = l;
  }
  var onCacheWarning: ((message: String) => void)? = null;
  if (let {onCacheWarning as w} = options) {
    onCacheWarning = w;
  }
  var extraArgs: Array<String>? = null;
  if (let {extraArgs as a} = options) {
    extraArgs = a;
  }
  var testRunner: TestStepRunner? = null;
  if (let {testRunner as tr} = options) {
    testRunner = tr;
  }
  listenSignals();
  let servicePool = new ServicePool();
  let interval = if (pollIntervalMs > (0 as i64)) pollIntervalMs else 50 as i64;
  var iteration = 0;
  try {
    while (true) {
      let configPaths = new GrowableArray<String>();
      let configs = new GrowableArray<ScriptConfig>();
      let outcome = await runTargetOnce(root, entry, {
        parallelism,
        force,
        useCache,
        remoteCache,
        lookupEnv,
        onCacheWarning,
        hasherOut: null,
        servicePool,
        allowForegroundService: false,
        configPathsOut: configPaths,
        configsOut: configs,
        planStatsOut: null,
        extraArgs,
        testRunner,
      });
      if (checkSignal() != 0) {
        return outcome;
      }
      iteration += 1;
      let baseline = await snapshotWatchFiles(root, configPaths, configs);
      if (onIteration != null) {
        if (!(await onIteration(iteration, outcome))) {
          return outcome;
        }
      } else {
        reportBuild(outcome, verbose);
      }
      while (true) {
        if (checkSignal() != 0) {
          return outcome;
        }
        let current = await snapshotWatchFiles(root, configPaths, configs);
        if (!snapshotsEqual(baseline, current)) {
          break;
        }
        if ((await waitSignalFor(interval)) != 0) {
          return outcome;
        }
      }
    }
  } finally {
    await servicePool.stopAll();
  }
}
  • terminal terminal.zena-lang.pkg.zena-lang.dev ~0.0.1-preview.1
  • workflow workflow.zena-lang.pkg.zena-lang.dev ~0.0.1-preview.1
VersionPublishedDependenciesApproved by
0.0.1-preview.1 2026-11-19
  • terminal terminal.zena-lang.pkg.zena-lang.dev ~0.0.1-preview.1
  • workflow workflow.zena-lang.pkg.zena-lang.dev ~0.0.1-preview.1