zebra 0.0.1-preview.1

Zebra: an incremental build orchestrator for Zena and standalone workspaces

Readme Files Dependencies Versions

zebra#

An incremental script and service runner for Zena (zena-package.json), standalone Zebra (zebra.json), and Wireit (package.json) workspaces, built on zena-lang:workflow.

Zebra runs a dependency graph of scripts and long-running services with declared inputs, outputs, and dependencies, rerunning only what changed. For each script, Zebra content-hashes the declared inputs and asks the workflow engine what to do: nothing (when the previous run recorded the same fingerprints and declared outputs are intact), restore from the local content-addressed cache, or execute the command (sh -c in the package directory with node_modules/.bin on PATH).

Installation#

zebra is built into the zena CLI (zena build <target>). To use it as a library or standalone runner:

zena pkg add zena-lang:zebra

CLI Usage#

zena build [-j <jobs>] [-w|--watch] [-f|--force] [--no-cache] [-v|--verbose] [<target>] [-- <args>...]
zena run [-j <jobs>] [-w|--watch] [-f|--force] [--no-cache] [-v|--verbose] <target> [-- <args>...]
zena test [-j <jobs>] [-w|--watch] [--force] [--no-cache] [-v|--verbose] [-- <args>...]
zena build --migrate [--dry-run] [-f|--force] [<package-dir>...]
  • <target> can be a script name in the current package (zena build test, zena run check), a package directory (zena build ./packages/app, which runs build), or <package-dir>:<script> (zena build ./packages/app:test). With no <target>, zena build runs the current package's build script and zena test runs its test script.
  • -- <args>... appends shell-quoted arguments to the entry target's command only, running that entry step uncached without overwriting its saved freshness record.
  • -j / --jobs <n> bounds parallel command execution (defaulting to ZEBRA_PARALLEL, WIREIT_PARALLEL, or ZENA_AVAILABLE_PARALLELISM).
  • -w / --watch watches input globs, lockfiles, and package manifests for changes, re-running affected steps incrementally while keeping unchanged services alive in a ServicePool across rebuilds.
  • -f / --force bypasses freshness and cached outputs to rerun scripts unconditionally.
  • --no-cache disables restoring outputs from .zebra/blobs/ when a step is not fresh.
  • -v / --verbose prints every step (including fresh and skipped) and any captured stdout/stderr on the rail.
  • --migrate converts Wireit blocks in package.json into zena-package.json (when present) or zebra.json.

Programmatic Usage#

import { buildTarget, reportBuild } from 'zena-lang:zebra/runner';
import { ScriptRef } from 'zena-lang:zebra/wireit-config';

let outcome = await buildTarget('.', new ScriptRef('packages/app', 'build'), {
  parallelism: 4,
});
let exitCode = reportBuild(outcome);

Configuration (zena-package.json and zebra.json)#

Zena packages configure scripts and filesets directly in zena-package.json. Non-Zena projects can use a standalone zebra.json (with an optional "workspace": {"packages": [...]} at the workspace root). Both support // and /* ... */ comments:

{
  "name": "app",
  "dependencies": {
    "lib": {"path": "../lib"},
  },
  "filesets": {
    "assets": ["assets/**"],
  },
  "scripts": {
    "build": {
      "command": "zena build src/main.zena -o out/app.wasm",
      "output": ["out/app.wasm"],
      "dependencies": ["source", "assets"],
    },
    "test": {
      "type": "zena:test",
      "tests": "test/*_test.zena",
    },
    "fmt": "zena fmt --check .",
  },
}
  • Declarative zena:test steps and default test target: Scripts in zena-package.json and zebra.json can set "type": "zena:test" (with optional "tests": true, "tests": "test/*_test.zena", or "tests": ["test/*_test.zena", ...] — defaulting to ["test/**/*_test.zena", "tests/**/*_test.zena"] when omitted — and optional "debug": true for -g) to run Zena tests in-process through TestStepRunner. Test pattern globs merge into files, and zena-package.json "zena:test" steps automatically depend on source. When a zena-package.json omits a test script (and no fallback package.json defines test), Zebra synthesizes a "type": "zena:test" target over test/**/*_test.zena and tests/**/*_test.zena.
  • Automatic source fileset: Every package with zena-package.json exposes an automatic source fileset covering its library .zena files (from "source", defaulting to "src" and excluding *_test.zena, test/**, and tests/**), zena-package.json, any "wit" dependencies, and the source filesets of all "path" dependencies. Scripts opt into it via "dependencies": ["source"] (or "../pkg:source").
  • Named filesets: Entries accept either a glob array shorthand ("wit": ["wit/**/*.wit"]) or {"files": [...], "dependencies": [...]} and compose transitively without spawning commands.
  • Caching and tracking: Task scripts cache automatically whenever files, output, or a fileset dependency is declared, defaulting output to [] when omitted. Setting "cache": false opts a step out of freshness and caching while keeping its files globs active in --watch mode. Omitting files, output, and fileset dependencies on a command step leaves it untracked so it runs every time. String entries ("fmt": "zena fmt --check .") are shorthands for always-run commands.
  • Locks: locks defaults to ["zena-lock.json"] and may be set at the package root or overridden per script ("locks": [] disables lockfile hashing).
  • Services: Scripts with "service": true or "service": {"readyWhen": {"lineMatches": "..."}} start on demand when a dependent executes, wait for readiness when configured, are shared across dependents, and are stopped as a Unix process group (SIGTERM then SIGKILL) when dependents finish. Invoking a service directly as the top-level target runs its dependencies and streams the service in the foreground.
  • Wireit fallback: When a package directory has no zebra.json and no "scripts" block in zena-package.json, Zebra falls back to package.json Wireit scripts while still serving filesets from zena-package.json. Dependencies can cross freely between zena-package.json, zebra.json, and package.json packages.

Cache Layout (.zebra/)#

Each package's <pkg>/.zebra/ directory stores:

  • state.json — freshness records for that package's scripts (the last cache key and output manifest).
  • cache/<key>.json — cache key to output manifest.
  • blobs/ — content-addressed output bytes (sha256).

The workspace root's .zebra/ directory stores:

  • hashes.json — the (mtime, size) to sha256 file hash memo shared across packages (with a 20ms racy-timestamp guard).
  • plans/<hash>.plan — stat-verified cached ScriptConfig closures keyed by target entry.

A dependency contributes the digest of its outputs to dependents' cache keys rather than its input fingerprint, giving early cutoff when a dependency reruns and produces identical outputs.

GitHub Actions Remote Cache#

Setting ZEBRA_CACHE=github layers GitHubActionsCache behind the local .zebra/ DiskCache (TieredCache). On a local cache miss, Zebra queries the GitHub Actions v2 CacheService Twirp JSON API, verifies downloaded blob sha256 digests, populates <pkg>/.zebra/, and materializes outputs into the workspace. On a build step execution, Zebra writes <pkg>/.zebra/ first and uploads the step's entry to GitHub Actions Cache (ZEBRA_CACHE_READ_ONLY=1 disables uploads; ZEBRA_CACHE=none disables both local and remote caching).

In GitHub Actions workflows, run .github/actions/setup-zebra-cache before build steps so ACTIONS_RESULTS_URL and ACTIONS_RUNTIME_TOKEN are served to run: steps over a 127.0.0.1 token custodian (ZEBRA_GITHUB_CUSTODIAN_URL).

Libraries#

  • zebra (zebra/main) — CLI entry point (main).
  • zebra/runner — buildTarget, watchTarget, BuildOptions, WatchOptions, reportBuild, reportMigration, formatBuildReport, and BuildOutcome.
  • zebra/zebra-config — zena-package.json and zebra.json parsing (ZebraWorkspace), synthesized source filesets, and Wireit migration (migrateWorkspaceToZebra, convertPackageJsonToZebraJson, mergeWireitIntoZenaPackageJson).
  • zebra/wireit-config — package.json Wireit parsing (WireitWorkspace) and shared script configuration types (ScriptConfig, ScriptRef, DependencySpec, ServiceConfig, EnvBinding, ConfigError, refKey, resolveDependency).
  • zebra/plan-cache — .zebra/plans/ snapshot serialization and stat-verified warm plan loading (loadOrCollectConfigs, PlanLoadStats, ConfigProbe).
  • zebra/builder — Translates ScriptConfig closures into workflow steps and execution callbacks (planScripts, StepExecution, lockfileCandidates).
  • zebra/disk-cache, zebra/github-cache, and zebra/state-json — .zebra/ blob/manifest storage (DiskCache, TieredCache, stateDirOf), GitHub Actions remote caching (GitHubActionsCache, GitHubCacheConfig, CacheMode, resolveCacheMode, resolveGitHubCacheConfig), and JSON serialization for state, manifests, and hash memos.
  • zebra/fs-walk and zebra/fs-util — Input glob discovery (FileHasher, expandGlobs, hashFile, validateGlobs, splitBase) and workspace path utilities.
  • zebra/shell — POSIX shell argument quoting (shellQuote, appendShellArgs). Zebra's terminal output uses zena-lang:terminal (packages/terminal): colors, the Clack-style rail layout and the spinner.

License#

MIT

6.1 KiB Raw
import {Box, i64ToString, parseI64} from 'zena:core';
import {
  JsonArray,
  JsonBuilder,
  JsonObject,
  JsonValue,
  parseJson,
} from 'zena:json';
import {ResolvedInput} from 'workflow/canonical';
import {CacheKey, Fingerprint} from 'workflow/fingerprint';
import {OutputEntry, OutputManifest} from 'workflow/manifest';
import {EngineState, StepRecord} from 'workflow/state';
import {StepKey} from 'workflow/step';

/**
 * JSON serialization for the engine's between-run state and for cached
 * output manifests. The workflow library treats both as plain data;
 * this codec is zebra's on-disk format for them (state under .zebra/, one
 * manifest per cache entry).
 */

function appendManifest(b: JsonBuilder, manifest: OutputManifest) {
  b.beginArray();
  for (let entry in manifest.outputs) {
    b.beginObject();
    b.key('name');
    b.string(entry.name);
    b.key('fp');
    b.string(entry.fingerprint.value);
    let deps = entry.dependsOn;
    if (deps != null) {
      b.key('dependsOn');
      b.beginArray();
      for (let dep in deps) {
        b.string(dep);
      }
      b.endArray();
    }
    b.endObject();
  }
  b.endArray();
}

export function manifestToJson(manifest: OutputManifest): String {
  let b = new JsonBuilder();
  appendManifest(b, manifest);
  return b.toString();
}

export function stateToJson(state: EngineState): String {
  let b = new JsonBuilder();
  b.beginObject();
  b.key('version');
  b.i32(1);
  b.key('steps');
  b.beginObject();
  state.forEach((key, record) => {
    b.key(key);
    b.beginObject();
    b.key('config');
    b.string(record.config.value);
    b.key('cacheKey');
    b.string(record.cacheKey.value);
    b.key('inputs');
    b.beginArray();
    for (let input in record.inputs) {
      b.beginArray();
      b.string(input.name);
      b.string(input.fingerprint.value);
      b.endArray();
    }
    b.endArray();
    b.key('outputs');
    appendManifest(b, record.manifest);
    b.endObject();
  });
  b.endObject();
  b.endObject();
  return b.toString();
}

function asString(value: JsonValue, what: String): String {
  if (value is String) {
    return value;
  }
  throw new Error('state json: expected a string for ' + what);
}

function asArray(value: JsonValue, what: String): JsonArray {
  if (value is JsonArray) {
    return value;
  }
  throw new Error('state json: expected an array for ' + what);
}

function asObject(value: JsonValue, what: String): JsonObject {
  if (value is JsonObject) {
    return value;
  }
  throw new Error('state json: expected an object for ' + what);
}

export function manifestFromJsonValue(value: JsonValue): OutputManifest {
  let entries = asArray(value, 'outputs');
  let outputs = new GrowableArray<OutputEntry>();
  for (var i = 0; i < entries.length; i += 1) {
    let obj = asObject(entries[i], 'output entry');
    var dependsOn: Array<String>? = null;
    let rawDeps = obj['dependsOn'];
    if (rawDeps != null) {
      let depsArr = asArray(rawDeps, 'dependsOn');
      let deps = new GrowableArray<String>();
      for (var j = 0; j < depsArr.length; j += 1) {
        deps.push(asString(depsArr[j], 'dependsOn entry'));
      }
      dependsOn = deps;
    }
    outputs.push(new OutputEntry(
      asString(obj['name'], 'output name'),
      new Fingerprint(asString(obj['fp'], 'output fp')),
      dependsOn,
    ));
  }
  return new OutputManifest(outputs);
}

export function manifestFromJson(text: String): OutputManifest {
  return manifestFromJsonValue(parseJson(text));
}

export function stateFromJson(text: String): EngineState {
  let root = asObject(parseJson(text), 'state root');
  let version = root['version'];
  if (!(version is Box<f64>)) {
    throw new Error('state json: missing version');
  }
  if (version.value != 1.0) {
    throw new Error('state json: unsupported version ' + f64ToString(version.value));
  }
  let state = new EngineState();
  let steps = asObject(root['steps'], 'steps');
  steps.forEach((key, value) => {
    let obj = asObject(value, 'step ' + key);
    let inputsArr = asArray(obj['inputs'], 'inputs');
    let inputs = new GrowableArray<ResolvedInput>();
    for (var i = 0; i < inputsArr.length; i += 1) {
      let pair = asArray(inputsArr[i], 'input pair');
      inputs.push(new ResolvedInput(
        asString(pair[0], 'input name'),
        new Fingerprint(asString(pair[1], 'input fp')),
      ));
    }
    state.setRecord(
      new StepKey(key),
      new StepRecord(
        new Fingerprint(asString(obj['config'], 'config')),
        new CacheKey(asString(obj['cacheKey'], 'cacheKey')),
        manifestFromJsonValue(obj['outputs']),
        inputs,
      ),
    );
  });
  return state;
}

/** One entry in the workspace-level `(modified, size) -> fingerprint` memo. */
export class HashMemoEntry(
  modified: i64,
  size: i64,
  fingerprint: Fingerprint,
  var verifiedThisRun: boolean = false,
)

export function hashesToJson(entries: HashMap<String, HashMemoEntry>): String {
  let b = new JsonBuilder();
  b.beginObject();
  b.key('version');
  b.i32(1);
  b.key('files');
  b.beginObject();
  entries.forEach((path, entry) => {
    if (entry.modified > (0 as i64)) {
      b.key(path);
      b.beginArray();
      b.string(i64ToString(entry.modified));
      b.string(i64ToString(entry.size));
      b.string(entry.fingerprint.value);
      b.endArray();
    }
  });
  b.endObject();
  b.endObject();
  return b.toString();
}

export function hashesFromJson(text: String): HashMap<String, HashMemoEntry> {
  let out = new HashMap<String, HashMemoEntry>();
  let root = asObject(parseJson(text), 'hashes root');
  let version = root['version'];
  if (!(version is Box<f64>) || version.value != 1.0) {
    return out;
  }
  let files = asObject(root['files'], 'files');
  files.forEach((path, value) => {
    let arr = asArray(value, 'hash entry');
    if (arr.length != 3) {
      return;
    }
    let modStr = asString(arr[0], 'modified');
    let sizeStr = asString(arr[1], 'size');
    let fpStr = asString(arr[2], 'fingerprint');
    if (let (true, modified) = parseI64(modStr)) {
      if (let (true, size) = parseI64(sizeStr)) {
        if (modified > (0 as i64) && size >= (0 as i64)) {
          out[path] = new HashMemoEntry(modified, size, new Fingerprint(fpStr), false);
        }
      }
    }
  });
  return out;
}
  • terminal terminal.zena-lang.pkg.zena-lang.dev ~0.0.1-preview.1
  • workflow workflow.zena-lang.pkg.zena-lang.dev ~0.0.1-preview.1
VersionPublishedDependenciesApproved by
0.0.1-preview.1 2026-11-19
  • terminal terminal.zena-lang.pkg.zena-lang.dev ~0.0.1-preview.1
  • workflow workflow.zena-lang.pkg.zena-lang.dev ~0.0.1-preview.1