zebra 0.0.1-preview.1

Zebra: an incremental build orchestrator for Zena and standalone workspaces

Readme Files Dependencies Versions

zebra#

An incremental script and service runner for Zena (zena-package.json), standalone Zebra (zebra.json), and Wireit (package.json) workspaces, built on zena-lang:workflow.

Zebra runs a dependency graph of scripts and long-running services with declared inputs, outputs, and dependencies, rerunning only what changed. For each script, Zebra content-hashes the declared inputs and asks the workflow engine what to do: nothing (when the previous run recorded the same fingerprints and declared outputs are intact), restore from the local content-addressed cache, or execute the command (sh -c in the package directory with node_modules/.bin on PATH).

Installation#

zebra is built into the zena CLI (zena build <target>). To use it as a library or standalone runner:

zena pkg add zena-lang:zebra

CLI Usage#

zena build [-j <jobs>] [-w|--watch] [-f|--force] [--no-cache] [-v|--verbose] [<target>] [-- <args>...]
zena run [-j <jobs>] [-w|--watch] [-f|--force] [--no-cache] [-v|--verbose] <target> [-- <args>...]
zena test [-j <jobs>] [-w|--watch] [--force] [--no-cache] [-v|--verbose] [-- <args>...]
zena build --migrate [--dry-run] [-f|--force] [<package-dir>...]
  • <target> can be a script name in the current package (zena build test, zena run check), a package directory (zena build ./packages/app, which runs build), or <package-dir>:<script> (zena build ./packages/app:test). With no <target>, zena build runs the current package's build script and zena test runs its test script.
  • -- <args>... appends shell-quoted arguments to the entry target's command only, running that entry step uncached without overwriting its saved freshness record.
  • -j / --jobs <n> bounds parallel command execution (defaulting to ZEBRA_PARALLEL, WIREIT_PARALLEL, or ZENA_AVAILABLE_PARALLELISM).
  • -w / --watch watches input globs, lockfiles, and package manifests for changes, re-running affected steps incrementally while keeping unchanged services alive in a ServicePool across rebuilds.
  • -f / --force bypasses freshness and cached outputs to rerun scripts unconditionally.
  • --no-cache disables restoring outputs from .zebra/blobs/ when a step is not fresh.
  • -v / --verbose prints every step (including fresh and skipped) and any captured stdout/stderr on the rail.
  • --migrate converts Wireit blocks in package.json into zena-package.json (when present) or zebra.json.

Programmatic Usage#

import { buildTarget, reportBuild } from 'zena-lang:zebra/runner';
import { ScriptRef } from 'zena-lang:zebra/wireit-config';

let outcome = await buildTarget('.', new ScriptRef('packages/app', 'build'), {
  parallelism: 4,
});
let exitCode = reportBuild(outcome);

Configuration (zena-package.json and zebra.json)#

Zena packages configure scripts and filesets directly in zena-package.json. Non-Zena projects can use a standalone zebra.json (with an optional "workspace": {"packages": [...]} at the workspace root). Both support // and /* ... */ comments:

{
  "name": "app",
  "dependencies": {
    "lib": {"path": "../lib"},
  },
  "filesets": {
    "assets": ["assets/**"],
  },
  "scripts": {
    "build": {
      "command": "zena build src/main.zena -o out/app.wasm",
      "output": ["out/app.wasm"],
      "dependencies": ["source", "assets"],
    },
    "test": {
      "type": "zena:test",
      "tests": "test/*_test.zena",
    },
    "fmt": "zena fmt --check .",
  },
}
  • Declarative zena:test steps and default test target: Scripts in zena-package.json and zebra.json can set "type": "zena:test" (with optional "tests": true, "tests": "test/*_test.zena", or "tests": ["test/*_test.zena", ...] — defaulting to ["test/**/*_test.zena", "tests/**/*_test.zena"] when omitted — and optional "debug": true for -g) to run Zena tests in-process through TestStepRunner. Test pattern globs merge into files, and zena-package.json "zena:test" steps automatically depend on source. When a zena-package.json omits a test script (and no fallback package.json defines test), Zebra synthesizes a "type": "zena:test" target over test/**/*_test.zena and tests/**/*_test.zena.
  • Automatic source fileset: Every package with zena-package.json exposes an automatic source fileset covering its library .zena files (from "source", defaulting to "src" and excluding *_test.zena, test/**, and tests/**), zena-package.json, any "wit" dependencies, and the source filesets of all "path" dependencies. Scripts opt into it via "dependencies": ["source"] (or "../pkg:source").
  • Named filesets: Entries accept either a glob array shorthand ("wit": ["wit/**/*.wit"]) or {"files": [...], "dependencies": [...]} and compose transitively without spawning commands.
  • Caching and tracking: Task scripts cache automatically whenever files, output, or a fileset dependency is declared, defaulting output to [] when omitted. Setting "cache": false opts a step out of freshness and caching while keeping its files globs active in --watch mode. Omitting files, output, and fileset dependencies on a command step leaves it untracked so it runs every time. String entries ("fmt": "zena fmt --check .") are shorthands for always-run commands.
  • Locks: locks defaults to ["zena-lock.json"] and may be set at the package root or overridden per script ("locks": [] disables lockfile hashing).
  • Services: Scripts with "service": true or "service": {"readyWhen": {"lineMatches": "..."}} start on demand when a dependent executes, wait for readiness when configured, are shared across dependents, and are stopped as a Unix process group (SIGTERM then SIGKILL) when dependents finish. Invoking a service directly as the top-level target runs its dependencies and streams the service in the foreground.
  • Wireit fallback: When a package directory has no zebra.json and no "scripts" block in zena-package.json, Zebra falls back to package.json Wireit scripts while still serving filesets from zena-package.json. Dependencies can cross freely between zena-package.json, zebra.json, and package.json packages.

Cache Layout (.zebra/)#

Each package's <pkg>/.zebra/ directory stores:

  • state.json — freshness records for that package's scripts (the last cache key and output manifest).
  • cache/<key>.json — cache key to output manifest.
  • blobs/ — content-addressed output bytes (sha256).

The workspace root's .zebra/ directory stores:

  • hashes.json — the (mtime, size) to sha256 file hash memo shared across packages (with a 20ms racy-timestamp guard).
  • plans/<hash>.plan — stat-verified cached ScriptConfig closures keyed by target entry.

A dependency contributes the digest of its outputs to dependents' cache keys rather than its input fingerprint, giving early cutoff when a dependency reruns and produces identical outputs.

GitHub Actions Remote Cache#

Setting ZEBRA_CACHE=github layers GitHubActionsCache behind the local .zebra/ DiskCache (TieredCache). On a local cache miss, Zebra queries the GitHub Actions v2 CacheService Twirp JSON API, verifies downloaded blob sha256 digests, populates <pkg>/.zebra/, and materializes outputs into the workspace. On a build step execution, Zebra writes <pkg>/.zebra/ first and uploads the step's entry to GitHub Actions Cache (ZEBRA_CACHE_READ_ONLY=1 disables uploads; ZEBRA_CACHE=none disables both local and remote caching).

In GitHub Actions workflows, run .github/actions/setup-zebra-cache before build steps so ACTIONS_RESULTS_URL and ACTIONS_RUNTIME_TOKEN are served to run: steps over a 127.0.0.1 token custodian (ZEBRA_GITHUB_CUSTODIAN_URL).

Libraries#

  • zebra (zebra/main) — CLI entry point (main).
  • zebra/runner — buildTarget, watchTarget, BuildOptions, WatchOptions, reportBuild, reportMigration, formatBuildReport, and BuildOutcome.
  • zebra/zebra-config — zena-package.json and zebra.json parsing (ZebraWorkspace), synthesized source filesets, and Wireit migration (migrateWorkspaceToZebra, convertPackageJsonToZebraJson, mergeWireitIntoZenaPackageJson).
  • zebra/wireit-config — package.json Wireit parsing (WireitWorkspace) and shared script configuration types (ScriptConfig, ScriptRef, DependencySpec, ServiceConfig, EnvBinding, ConfigError, refKey, resolveDependency).
  • zebra/plan-cache — .zebra/plans/ snapshot serialization and stat-verified warm plan loading (loadOrCollectConfigs, PlanLoadStats, ConfigProbe).
  • zebra/builder — Translates ScriptConfig closures into workflow steps and execution callbacks (planScripts, StepExecution, lockfileCandidates).
  • zebra/disk-cache, zebra/github-cache, and zebra/state-json — .zebra/ blob/manifest storage (DiskCache, TieredCache, stateDirOf), GitHub Actions remote caching (GitHubActionsCache, GitHubCacheConfig, CacheMode, resolveCacheMode, resolveGitHubCacheConfig), and JSON serialization for state, manifests, and hash memos.
  • zebra/fs-walk and zebra/fs-util — Input glob discovery (FileHasher, expandGlobs, hashFile, validateGlobs, splitBase) and workspace path utilities.
  • zebra/shell — POSIX shell argument quoting (shellQuote, appendShellArgs). Zebra's terminal output uses zena-lang:terminal (packages/terminal): colors, the Clack-style rail layout and the spinner.

License#

MIT

23.9 KiB Raw
import {Future} from 'zena:async';
import {getEnv} from 'zena:cli';
import { HashSet } from 'zena:collections';
import {div} from 'zena:math';
import {Process, checkSignal, runWith, spawnWith} from 'zena:process';
import {Regex} from 'zena:regex';
import {monotonic} from 'zena:time';
import {StringBuilder, Unmanaged, adopt, disown} from 'zena:core';
import {Fingerprint} from 'workflow/fingerprint';
import {Graph} from 'workflow/graph';
import {OutputEntry, OutputManifest} from 'workflow/manifest';
import {
  InputRef,
  LeafSet,
  RunContext,
  ServiceHandle,
  ServiceStep,
  Step,
  StepKey,
  StepOrder,
  StepOutputs,
  TaskStep,
} from 'workflow/step';
import {deleteIfExists, joinWorkspace, withRoot} from './fs-util.zena';
import {FileHasher, expandGlobs, hashFile, validateGlobs} from './fs-walk.zena';
import {sha256HexOfString} from 'zena:crypto';
import {appendShellArgs} from './shell.zena';
import {
  ConfigError,
  DependencySpec,
  EnvBinding,
  EnvExternal,
  EnvLiteral,
  ScriptConfig,
  ZENA_TEST_RULE_TYPE,
  refKey,
} from './wireit-config.zena';

/** Captured command output and wall-clock duration for a step that executed. */
export class StepExecution {
  command: String;
  exitCode: i32;
  stdout: String;
  stderr: String;
  durationMs: i64;

  new(this.command, this.exitCode, this.stdout, this.stderr, this.durationMs);

  combinedOutput(): String {
    if (this.stdout == '') {
      return this.stderr;
    }
    if (this.stderr == '') {
      return this.stdout;
    }
    if (this.stdout.endsWith('\n')) {
      return this.stdout + this.stderr;
    }
    return `${this.stdout}\n${this.stderr}`;
  }
}

/** Parameters for running one package's native Zena `"tests"` step. */
export class TestStepRequest {
  root: String;
  packageDir: String;
  patterns: Array<String>;
  extraArgs: Array<String>;
  env: Array<(String, String)>;
  envRemove: Array<String>;
  debug: boolean;

  new(
    this.root,
    this.packageDir,
    this.patterns,
    this.extraArgs,
    this.env,
    this.envRemove,
    this.debug,
  );
}

/**
 * In-process executor for native Zena `"tests"` steps, supplied by `zena`
 * so test targets compile and run without spawning shell or CLI child processes.
 */
export interface TestStepRunner {
  /**
   * Identity of the compiler and standard library running tests, evaluated
   * lazily after a step's dependencies settle so upstream compiler builds
   * invalidate downstream test steps in the same run.
   */
  fingerprint(): Future<String>;
  /** Runs one package's Zena test step and returns its captured output and status. */
  runTests(request: TestStepRequest): Future<StepExecution>;
}

/** A planned build: the step graph, initial leaf map, and captured step runs. */
export class Plan {
  graph: Graph;
  leaves: HashMap<String, Fingerprint>;
  executions: HashMap<String, StepExecution>;

  new(
    this.graph,
    this.leaves,
    this.executions = new HashMap<String, StepExecution>(),
  );
}

function appendSegment(sb: StringBuilder, s: String) {
  sb.appendI32(s.length);
  sb.append(':');
  sb.append(s);
}

function appendEnvSegments(
  sb: StringBuilder,
  env: Array<(String, EnvBinding)>,
) {
  for (let (name, binding) in env) {
    match (binding) {
      case EnvLiteral as lit: {
        appendSegment(sb, 'env:' + name);
        appendSegment(sb, lit.value);
      }
      case EnvExternal as ext: {
        let effective = getEnv(name) ?? ext.defaultValue;
        if (effective != null) {
          appendSegment(sb, 'env-ext:' + name);
          appendSegment(sb, effective);
        } else {
          appendSegment(sb, 'env-unset:' + name);
        }
      }
    }
  }
}

/**
 * Remote-cache variables consumed by the outer Zebra process. Child scripts
 * and services strip these from their inherited environment unless their own
 * `env` config declares them, so nested builds and test suites do not read or
 * write the outer run's remote cache.
 */
export let REMOTE_CACHE_ENV_VARS = [
  'ZEBRA_CACHE',
  'ZEBRA_CACHE_READ_ONLY',
  'ZEBRA_GITHUB_CUSTODIAN_URL',
  'WIREIT_CACHE_GITHUB_CUSTODIAN_PORT',
];

/** Concrete `(name, value)` pairs to pass in `SpawnOptions.env`. */
function resolveEnvOverrides(
  env: Array<(String, EnvBinding)>,
): Array<(String, String)> {
  let overrides = new GrowableArray<(String, String)>();
  for (let (name, binding) in env) {
    match (binding) {
      case EnvLiteral as lit: {
        overrides.push((name, lit.value));
      }
      case EnvExternal as ext: {
        if (getEnv(name) == null) {
          let def = ext.defaultValue;
          if (def != null) {
            overrides.push((name, def));
          }
        }
      }
    }
  }
  return overrides;
}

/** Remote-cache variables to omit from `SpawnOptions.envRemove`. */
function resolveEnvRemovals(
  env: Array<(String, EnvBinding)>,
): Array<String> {
  let removals = new GrowableArray<String>();
  for (let candidate in REMOTE_CACHE_ENV_VARS) {
    var declared = false;
    for (let (name, _) in env) {
      if (name == candidate) {
        declared = true;
        break;
      }
    }
    if (!declared) {
      removals.push(candidate);
    }
  }
  return removals;
}

function configFingerprint(
  config: ScriptConfig,
  salt: String,
  forceSalted: boolean = false,
  hasInProcessRunner: boolean = false,
) {
  let sb = new StringBuilder();
  appendSegment(sb, 'zebra-config-v1');
  let command = config.command;
  appendSegment(sb, command ?? '');
  appendSegment(sb, config.clean);
  appendSegment(sb, if (config.isWireit) 'wireit' else 'bare');
  let svc = config.service;
  if (svc != null) {
    let pattern = svc.readyWhenLineMatches;
    appendSegment(
      sb,
      if (pattern != null) 'service:line:' + pattern else 'service:ready',
    );
  } else {
    appendSegment(sb, 'service:none');
  }
  let ruleType = config.ruleType;
  if (ruleType != null) {
    appendSegment(sb, 'type:' + ruleType);
  }
  let tests = config.tests;
  if (ruleType == ZENA_TEST_RULE_TYPE && tests != null) {
    for (let pat in tests) {
      appendSegment(sb, 'test-glob:' + pat);
    }
    appendSegment(sb, if (config.testDebug) 'test-debug:1' else 'test-debug:0');
    appendSegment(
      sb,
      if (hasInProcessRunner) 'test-runner:in-process' else 'test-runner:shell',
    );
  }
  for (let glob in config.files) {
    appendSegment(sb, 'file:' + glob);
  }
  for (let glob in config.output) {
    appendSegment(sb, 'out:' + glob);
  }
  for (let lock in config.packageLocks) {
    appendSegment(sb, 'lock:' + lock);
  }
  appendEnvSegments(sb, config.env);
  if (forceSalted || !config.fingerprintable) {
    // Bare npm scripts, scripts with undeclared inputs/outputs, and
    // foreground top-level services must never go fresh or hit cache.
    appendSegment(sb, 'salt:' + salt);
  }
  return new Fingerprint('sha256:' + sha256HexOfString(sb.toString()));
}

async function cleanOutputs(
  root: String,
  config: ScriptConfig,
  ctx: RunContext,
): Future<void> {
  if (config.clean == 'false') {
    return;
  }
  if (config.clean == 'if-file-deleted') {
    let delta = ctx.delta;
    if (delta == null || delta.removed.length == 0) {
      return;
    }
  }
  let previous = ctx.previous;
  if (previous != null) {
    for (let entry in previous.outputs) {
      if (entry.fingerprint.value.startsWith('sha256:')) {
        await deleteIfExists(withRoot(root, entry.name));
      }
    }
  } else {
    for (let path in await expandGlobs(root, config.ref.packageDir, config.output)) {
      await deleteIfExists(withRoot(root, path));
    }
  }
}

/**
 * The PATH prefix npm would give the script: the package's own
 * node_modules/.bin and the workspace root's, relative to the package
 * directory the command runs in.
 */
function binPathPrefix(packageDir: String) {
  if (packageDir == '') {
    return 'node_modules/.bin';
  }
  let up = new StringBuilder();
  for (let segment in packageDir.split('/')) {
    if (segment != '') {
      up.append('../');
    }
  }
  return `node_modules/.bin:${up.toString()}node_modules/.bin`;
}

function fallbackTestShellCommand(
  patterns: Array<String>,
  extraArgs: Array<String>,
  debug: boolean,
) {
  let base = if (debug) 'zena -g test' else 'zena test';
  let allArgs = new GrowableArray<String>();
  for (let p in patterns) {
    allArgs.push(p);
  }
  for (let a in extraArgs) {
    allArgs.push(a);
  }
  return appendShellArgs(base, allArgs);
}

/**
 * Runs one script's command (or native Zena `"tests"` step) and collects its
 * outputs. Shell commands run through `sh -c` in the package directory with
 * node_modules/.bin on PATH, like wireit through npm, and await exit
 * asynchronously so independent steps run in parallel.
 */
export async function runScript(
  root: String,
  config: ScriptConfig,
  ctx: RunContext,
  volatileMark: String?,
  hasher: FileHasher?,
  executions: HashMap<String, StepExecution>? = null,
  testRunner: TestStepRunner? = null,
): Future<OutputManifest> {
  let command = config.command;
  let isZenaTest = config.ruleType == ZENA_TEST_RULE_TYPE;
  let tests = if (isZenaTest) config.tests else null;
  if (command == null && tests == null && config.output.length == 0) {
    // Command-less group or fileset scripts forward their resolved
    // inputs as virtual entries so upstream manifest changes (and
    // $volatile markers) propagate to downstream dependents.
    let forwarded = new GrowableArray<OutputEntry>();
    for (let inp in ctx.inputs) {
      forwarded.push(new OutputEntry(
        '$in:' + inp.name,
        new Fingerprint('ref:' + inp.fingerprint.value),
      ));
    }
    return new OutputManifest(forwarded);
  }
  if (tests != null) {
    let key = refKey(config.ref);
    let sigBefore = checkSignal();
    if (sigBefore != 0) {
      throw new Error(
        `script "${key}" interrupted by signal ${i32ToString(sigBefore)}`,
      );
    }
    let extra = config.testExtraArgs ?? new GrowableArray<String>();
    let envOverrides = resolveEnvOverrides(config.env);
    let envRemovals = resolveEnvRemovals(config.env);
    var exec: StepExecution | null = null;
    if (testRunner != null) {
      exec = await testRunner.runTests(
        new TestStepRequest(
          root,
          config.ref.packageDir,
          tests,
          extra,
          envOverrides,
          envRemovals,
          config.testDebug,
        ),
      );
    } else {
      let startNs = monotonic() as i64;
      let cwd = withRoot(root, config.ref.packageDir);
      let shellCmd = fallbackTestShellCommand(tests, extra, config.testDebug);
      let wrapped =
        `export PATH="${binPathPrefix(config.ref.packageDir)}:$PATH"; ${shellCmd}`;
      let result = await runWith(['sh', '-c', wrapped], {
        cwd: cwd,
        env: envOverrides,
        envRemove: envRemovals,
        processGroup: true,
      });
      let durationMs = div((monotonic() as i64) - startNs, 1000000 as i64);
      exec = new StepExecution(
        shellCmd,
        result.exitCode,
        result.stdout,
        result.stderr,
        durationMs,
      );
    }
    let sigAfter = checkSignal();
    if (sigAfter != 0) {
      throw new Error(
        `script "${key}" interrupted by signal ${i32ToString(sigAfter)}`,
      );
    }
    let finished = exec;
    if (executions != null) {
      executions[key] = finished;
    }
    if (finished.exitCode != 0) {
      throw new Error(
        `script "${key}" failed (exit ${i32ToString(finished.exitCode)}): ${finished.command}\n` +
        finished.stdout + finished.stderr,
      );
    }
  } else if (command != null) {
    let key = refKey(config.ref);
    let sigBefore = checkSignal();
    if (sigBefore != 0) {
      throw new Error(
        `script "${key}" interrupted by signal ${i32ToString(sigBefore)}`,
      );
    }
    if (hasher != null) {
      hasher.markMutated();
    }
    await cleanOutputs(root, config, ctx);
    let startNs = monotonic() as i64;
    let cwd = withRoot(root, config.ref.packageDir);
    let wrapped =
      `export PATH="${binPathPrefix(config.ref.packageDir)}:$PATH"; ${command}`;
    let result = await runWith(['sh', '-c', wrapped], {
      cwd: cwd,
      env: resolveEnvOverrides(config.env),
      envRemove: resolveEnvRemovals(config.env),
      processGroup: true,
    });
    let sigAfter = checkSignal();
    if (sigAfter != 0) {
      throw new Error(
        `script "${key}" interrupted by signal ${i32ToString(sigAfter)}`,
      );
    }
    if (executions != null) {
      let durationMs = div((monotonic() as i64) - startNs, 1000000 as i64);
      executions[key] = new StepExecution(
        command,
        result.exitCode,
        result.stdout,
        result.stderr,
        durationMs,
      );
    }
    if (!result.succeeded()) {
      throw new Error(
        `script "${key}" failed (exit ${i32ToString(result.exitCode)}): ${command}\n` +
        result.stdout + result.stderr,
      );
    }
  }
  let outFiles = await expandGlobs(root, config.ref.packageDir, config.output);
  let outputs = new GrowableArray<OutputEntry>();
  for (let path in outFiles) {
    let fp = if (hasher != null) await hasher.hash(path) else await hashFile(root, path);
    outputs.push(new OutputEntry(path, fp));
  }
  if (volatileMark != null) {
    // An unfingerprintable script's true outputs are unknown, so its
    // manifest must never repeat: dependents' cache keys change every
    // run, which keeps them from going fresh over outputs this script
    // may have changed without declaring (wireit poisons dependents of
    // untracked scripts the same way).
    outputs.push(new OutputEntry(
      '$volatile',
      new Fingerprint('volatile:' + volatileMark),
    ));
  }
  return new OutputManifest(outputs);
}

class ZbServiceHandle implements ServiceHandle {
  #key: String;
  #command: String;
  #proc: Unmanaged<Process>;
  var #stopped = false;

  new(key: String, command: String, proc: Unmanaged<Process>)
      : #key = key,
        #command = command,
        #proc = proc;

  isAlive(): boolean {
    if (this.#stopped) {
      return false;
    }
    return !this.#proc.hasExited();
  }

  async stop(): Future<void> {
    if (this.#stopped) {
      return;
    }
    this.#stopped = true;
    let proc = adopt(this.#proc);
    if (proc.hasExited()) {
      let res = await proc.wait();
      if (checkSignal() == 0) {
        throw new Error(
          `service "${this.#key}" exited unexpectedly (exit ${i32ToString(res.exitCode)}): ${this.#command}\n` +
          res.stdout + res.stderr,
        );
      }
      return;
    }
    await proc.stop(2000 as i64);
  }
}

function compileReadyRegex(config: ScriptConfig): Regex? {
  let svc = config.service;
  if (svc == null) {
    return null;
  }
  let pattern = svc.readyWhenLineMatches;
  if (pattern == null) {
    return null;
  }
  try {
    return new Regex(pattern);
  } catch (e) {
    throw new ConfigError(
      `script "${refKey(config.ref)}": invalid readyWhen.lineMatches regex "${pattern}": ${e.message}`,
    );
  }
}

async function startService(
  root: String,
  config: ScriptConfig,
  readyRegex: Regex?,
  hasher: FileHasher,
  executions: HashMap<String, StepExecution>,
): Future<ServiceHandle> {
  let key = refKey(config.ref);
  let sigBefore = checkSignal();
  if (sigBefore != 0) {
    throw new Error(
      `service "${key}" interrupted by signal ${i32ToString(sigBefore)}`,
    );
  }
  hasher.markMutated();
  let startNs = monotonic() as i64;
  let command = config.command ?? '';
  let cwd = withRoot(root, config.ref.packageDir);
  let wrapped =
    `export PATH="${binPathPrefix(config.ref.packageDir)}:$PATH"; ${command}`;
  let proc = spawnWith(['sh', '-c', wrapped], {
    cwd: cwd,
    env: resolveEnvOverrides(config.env),
    envRemove: resolveEnvRemovals(config.env),
    processGroup: true,
  });
  if (readyRegex != null) {
    var matched = false;
    while (true) {
      let line = await proc.readLine();
      if (line == null) {
        break;
      }
      if (readyRegex.test(line)) {
        matched = true;
        break;
      }
    }
    let sigAfter = checkSignal();
    if (sigAfter != 0) {
      await proc.stop(2000 as i64);
      throw new Error(
        `service "${key}" interrupted by signal ${i32ToString(sigAfter)}`,
      );
    }
    if (!matched) {
      let res = await proc.stop(500 as i64);
      let durationMs = div((monotonic() as i64) - startNs, 1000000 as i64);
      executions[key] = new StepExecution(
        command,
        res.exitCode,
        res.stdout,
        res.stderr,
        durationMs,
      );
      throw new Error(
        `service "${key}" exited before readyWhen.lineMatches matched (exit ${i32ToString(res.exitCode)}): ${command}\n` +
        res.stdout + res.stderr,
      );
    }
  } else if (proc.hasExited()) {
    let res = await proc.wait();
    let durationMs = div((monotonic() as i64) - startNs, 1000000 as i64);
    executions[key] = new StepExecution(
      command,
      res.exitCode,
      res.stdout,
      res.stderr,
      durationMs,
    );
    throw new Error(
      `service "${key}" exited immediately (exit ${i32ToString(res.exitCode)}): ${command}\n` +
      res.stdout + res.stderr,
    );
  }
  let durationMs = div((monotonic() as i64) - startNs, 1000000 as i64);
  executions[key] = new StepExecution(command, 0, '', '', durationMs);
  return new ZbServiceHandle(key, command, disown(proc));
}

async function runForegroundService(
  root: String,
  config: ScriptConfig,
  readyRegex: Regex?,
  salt: String,
  hasher: FileHasher,
): Future<OutputManifest> {
  let sigBefore = checkSignal();
  if (sigBefore != 0) {
    throw new Error(
      `service "${refKey(config.ref)}" interrupted by signal ${i32ToString(sigBefore)}`,
    );
  }
  hasher.markMutated();
  let command = config.command ?? '';
  let cwd = withRoot(root, config.ref.packageDir);
  let wrapped =
    `export PATH="${binPathPrefix(config.ref.packageDir)}:$PATH"; ${command}`;
  let proc = spawnWith(['sh', '-c', wrapped], {
    cwd: cwd,
    env: resolveEnvOverrides(config.env),
    envRemove: resolveEnvRemovals(config.env),
    processGroup: true,
  });
  var matched = readyRegex == null;
  while (true) {
    let line = await proc.readLine();
    if (line == null) {
      break;
    }
    console.log(line);
    if (!matched && readyRegex != null && readyRegex.test(line)) {
      matched = true;
    }
  }
  let sigAfterRead = checkSignal();
  if (sigAfterRead != 0) {
    await proc.stop(2000 as i64);
    throw new Error(
      `service "${refKey(config.ref)}" interrupted by signal ${i32ToString(sigAfterRead)}`,
    );
  }
  let res = await proc.wait();
  let sigAfterWait = checkSignal();
  if (sigAfterWait != 0) {
    throw new Error(
      `service "${refKey(config.ref)}" interrupted by signal ${i32ToString(sigAfterWait)}`,
    );
  }
  if (!matched) {
    throw new Error(
      `service "${refKey(config.ref)}" exited before readyWhen.lineMatches matched (exit ${i32ToString(res.exitCode)}): ${command}\n` +
      res.stdout + res.stderr,
    );
  }
  if (!res.succeeded()) {
    throw new Error(
      `service "${refKey(config.ref)}" failed (exit ${i32ToString(res.exitCode)}): ${command}\n` +
      res.stdout + res.stderr,
    );
  }
  return new OutputManifest([
    new OutputEntry('$volatile', new Fingerprint('volatile:' + salt)),
  ]);
}

export function lockfileCandidates(
  packageDir: String,
  lockNames: Array<String>,
): Array<String> {
  let candidates = new GrowableArray<String>();
  let seen = new HashSet<String>();
  var cur = packageDir;
  while (true) {
    for (let name in lockNames) {
      let candidate = joinWorkspace(cur, name);
      if (seen.add(candidate)) {
        candidates.push(candidate);
      }
    }
    if (cur == '') {
      break;
    }
    var slash = -1;
    for (var i = 0; i < cur.length; i += 1) {
      if (cur.getByteAt(i) == 47) {
        slash = i;
      }
    }
    cur = if (slash < 0) '' else cur.sliceBytes(0, slash);
  }
  return candidates;
}

async function resolveTestRunnerLeaf(
  runner: TestStepRunner,
): Future<Array<(String, Fingerprint)>> {
  let entries = new GrowableArray<(String, Fingerprint)>();
  entries.push(('__zena_test_runner__', new Fingerprint(await runner.fingerprint())));
  return entries;
}

class CoalescedRunnerLeaf {
  #runner: TestStepRunner;
  var #pending: Future<Array<(String, Fingerprint)>>? = null;

  new(runner: TestStepRunner) : #runner = runner;

  resolve(): Future<Array<(String, Fingerprint)>> {
    let existing = this.#pending;
    if (existing != null) {
      return existing;
    }
    let started = resolveTestRunnerLeaf(this.#runner);
    this.#pending = started;
    return started;
  }
}

function buildStepInputs(
  config: ScriptConfig,
  hasher: FileHasher,
  runnerLeaf: CoalescedRunnerLeaf? = null,
): Array<InputRef> {
  validateGlobs(config.ref.packageDir, config.files);
  validateGlobs(config.ref.packageDir, config.output);
  let inputs = new GrowableArray<InputRef>();
  let isZenaTest = config.ruleType == ZENA_TEST_RULE_TYPE && config.tests != null;
  let lockCandidates = if (
    config.isWireit &&
    (config.command != null || isZenaTest) &&
    config.fingerprintable &&
    config.packageLocks.length > 0
  )
    lockfileCandidates(config.ref.packageDir, config.packageLocks)
  else
    new GrowableArray<String>();
  if (config.files.length > 0 || lockCandidates.length > 0) {
    let pkgDir = config.ref.packageDir;
    let globs = config.files;
    inputs.push(
      new LeafSet('files', () => hasher.hashInputs(pkgDir, globs, lockCandidates)),
    );
  }
  if (isZenaTest && config.fingerprintable && runnerLeaf != null) {
    let sharedLeaf = runnerLeaf;
    inputs.push(new LeafSet('test-runner', () => sharedLeaf.resolve()));
  }
  // Deduplicate dependency edges, letting `cascade: true` win if listed both ways.
  let order = new GrowableArray<String>();
  let cascadeByKey = new HashMap<String, boolean>();
  let seen = new HashSet<String>();
  for (let dep in config.dependencies) {
    let key = refKey(dep.ref);
    if (seen.add(key)) {
      order.push(key);
      cascadeByKey[key] = dep.cascade;
    } else if (dep.cascade) {
      cascadeByKey[key] = true;
    }
  }
  for (let key in order) {
    if (cascadeByKey.getOr(key, true)) {
      inputs.push(new StepOutputs(new StepKey(key)));
    } else {
      inputs.push(new StepOrder(new StepKey(key)));
    }
  }
  return inputs;
}

/**
 * Turns collected script configs into a step graph with lazy per-step
 * input glob expansion backed by `hasher`. `foregroundServiceKey` names
 * a service invoked directly as the top-level target so it runs in the
 * foreground instead of sitting idle with zero dependents.
 */
export function planScripts(
  root: String,
  configs: Array<ScriptConfig>,
  salt: String,
  hasher: FileHasher? = null,
  foregroundServiceKey: String? = null,
  testRunner: TestStepRunner? = null,
): Plan {
  let fileHasher = hasher ?? new FileHasher(root);
  let hasRunner = testRunner != null;
  let runnerLeaf = if (testRunner != null) new CoalescedRunnerLeaf(testRunner) else null;
  let leaves = new HashMap<String, Fingerprint>();
  let executions = new HashMap<String, StepExecution>();
  let steps = new GrowableArray<Step>();
  for (let config in configs) {
    let key = refKey(config.ref);
    let inputs = buildStepInputs(config, fileHasher, runnerLeaf);
    if (config.service != null) {
      let readyRegex = compileReadyRegex(config);
      if (foregroundServiceKey != null && key == foregroundServiceKey) {
        steps.push(new TaskStep(
          new StepKey(key),
          configFingerprint(config, salt, true, hasRunner),
          inputs,
          (ctx) => runForegroundService(root, config, readyRegex, salt, fileHasher),
        ));
      } else {
        steps.push(new ServiceStep(
          new StepKey(key),
          configFingerprint(config, salt, false, hasRunner),
          inputs,
          (svcCtx) => startService(root, config, readyRegex, fileHasher, executions),
        ));
      }
    } else {
      steps.push(new TaskStep(
        new StepKey(key),
        configFingerprint(config, salt, false, hasRunner),
        inputs,
        (ctx) => runScript(
          root,
          config,
          ctx,
          if (config.fingerprintable) null else salt,
          fileHasher,
          executions,
          testRunner,
        ),
      ));
    }
  }
  return new Plan(new Graph(steps), leaves, executions);
}
  • terminal terminal.zena-lang.pkg.zena-lang.dev ~0.0.1-preview.1
  • workflow workflow.zena-lang.pkg.zena-lang.dev ~0.0.1-preview.1
VersionPublishedDependenciesApproved by
0.0.1-preview.1 2026-11-19
  • terminal terminal.zena-lang.pkg.zena-lang.dev ~0.0.1-preview.1
  • workflow workflow.zena-lang.pkg.zena-lang.dev ~0.0.1-preview.1