zebra 0.0.1-preview.1
Zebra: an incremental build orchestrator for Zena and standalone workspaces
zebra#
An incremental script and service runner for Zena (zena-package.json), standalone Zebra (zebra.json), and Wireit (package.json) workspaces, built on zena-lang:workflow.
Zebra runs a dependency graph of scripts and long-running services with declared inputs, outputs, and dependencies, rerunning only what changed. For each script, Zebra content-hashes the declared inputs and asks the workflow engine what to do: nothing (when the previous run recorded the same fingerprints and declared outputs are intact), restore from the local content-addressed cache, or execute the command (sh -c in the package directory with node_modules/.bin on PATH).
Installation#
zebra is built into the zena CLI (zena build <target>). To use it as a library or standalone runner:
zena pkg add zena-lang:zebra
CLI Usage#
zena build [-j <jobs>] [-w|--watch] [-f|--force] [--no-cache] [-v|--verbose] [<target>] [-- <args>...]
zena run [-j <jobs>] [-w|--watch] [-f|--force] [--no-cache] [-v|--verbose] <target> [-- <args>...]
zena test [-j <jobs>] [-w|--watch] [--force] [--no-cache] [-v|--verbose] [-- <args>...]
zena build --migrate [--dry-run] [-f|--force] [<package-dir>...]
<target>can be a script name in the current package (zena build test,zena run check), a package directory (zena build ./packages/app, which runsbuild), or<package-dir>:<script>(zena build ./packages/app:test). With no<target>,zena buildruns the current package'sbuildscript andzena testruns itstestscript.-- <args>...appends shell-quoted arguments to the entry target's command only, running that entry step uncached without overwriting its saved freshness record.-j/--jobs <n>bounds parallel command execution (defaulting toZEBRA_PARALLEL,WIREIT_PARALLEL, orZENA_AVAILABLE_PARALLELISM).-w/--watchwatches input globs, lockfiles, and package manifests for changes, re-running affected steps incrementally while keeping unchanged services alive in aServicePoolacross rebuilds.-f/--forcebypasses freshness and cached outputs to rerun scripts unconditionally.--no-cachedisables restoring outputs from.zebra/blobs/when a step is not fresh.-v/--verboseprints every step (includingfreshandskipped) and any captured stdout/stderr on the rail.--migrateconverts Wireit blocks inpackage.jsonintozena-package.json(when present) orzebra.json.
Programmatic Usage#
import { buildTarget, reportBuild } from 'zena-lang:zebra/runner';
import { ScriptRef } from 'zena-lang:zebra/wireit-config';
let outcome = await buildTarget('.', new ScriptRef('packages/app', 'build'), {
parallelism: 4,
});
let exitCode = reportBuild(outcome);
Configuration (zena-package.json and zebra.json)#
Zena packages configure scripts and filesets directly in zena-package.json. Non-Zena projects can use a standalone zebra.json (with an optional "workspace": {"packages": [...]} at the workspace root). Both support // and /* ... */ comments:
{
"name": "app",
"dependencies": {
"lib": {"path": "../lib"},
},
"filesets": {
"assets": ["assets/**"],
},
"scripts": {
"build": {
"command": "zena build src/main.zena -o out/app.wasm",
"output": ["out/app.wasm"],
"dependencies": ["source", "assets"],
},
"test": {
"type": "zena:test",
"tests": "test/*_test.zena",
},
"fmt": "zena fmt --check .",
},
}
- Declarative
zena:teststeps and defaulttesttarget: Scripts inzena-package.jsonandzebra.jsoncan set"type": "zena:test"(with optional"tests": true,"tests": "test/*_test.zena", or"tests": ["test/*_test.zena", ...]— defaulting to["test/**/*_test.zena", "tests/**/*_test.zena"]when omitted — and optional"debug": truefor-g) to run Zena tests in-process throughTestStepRunner. Test pattern globs merge intofiles, andzena-package.json"zena:test"steps automatically depend onsource. When azena-package.jsonomits atestscript (and no fallbackpackage.jsondefinestest), Zebra synthesizes a"type": "zena:test"target overtest/**/*_test.zenaandtests/**/*_test.zena. - Automatic
sourcefileset: Every package withzena-package.jsonexposes an automaticsourcefileset covering its library.zenafiles (from"source", defaulting to"src"and excluding*_test.zena,test/**, andtests/**),zena-package.json, any"wit"dependencies, and thesourcefilesets of all"path"dependencies. Scripts opt into it via"dependencies": ["source"](or"../pkg:source"). - Named
filesets: Entries accept either a glob array shorthand ("wit": ["wit/**/*.wit"]) or{"files": [...], "dependencies": [...]}and compose transitively without spawning commands. - Caching and tracking: Task scripts cache automatically whenever
files,output, or a fileset dependency is declared, defaultingoutputto[]when omitted. Setting"cache": falseopts a step out of freshness and caching while keeping itsfilesglobs active in--watchmode. Omittingfiles,output, and fileset dependencies on a command step leaves it untracked so it runs every time. String entries ("fmt": "zena fmt --check .") are shorthands for always-run commands. - Locks:
locksdefaults to["zena-lock.json"]and may be set at the package root or overridden per script ("locks": []disables lockfile hashing). - Services: Scripts with
"service": trueor"service": {"readyWhen": {"lineMatches": "..."}}start on demand when a dependent executes, wait for readiness when configured, are shared across dependents, and are stopped as a Unix process group (SIGTERMthenSIGKILL) when dependents finish. Invoking a service directly as the top-level target runs its dependencies and streams the service in the foreground. - Wireit fallback: When a package directory has no
zebra.jsonand no"scripts"block inzena-package.json, Zebra falls back topackage.jsonWireit scripts while still serving filesets fromzena-package.json. Dependencies can cross freely betweenzena-package.json,zebra.json, andpackage.jsonpackages.
Cache Layout (.zebra/)#
Each package's <pkg>/.zebra/ directory stores:
state.json— freshness records for that package's scripts (the last cache key and output manifest).cache/<key>.json— cache key to output manifest.blobs/— content-addressed output bytes (sha256).
The workspace root's .zebra/ directory stores:
hashes.json— the(mtime, size)tosha256file hash memo shared across packages (with a 20ms racy-timestamp guard).plans/<hash>.plan—stat-verified cachedScriptConfigclosures keyed by target entry.
A dependency contributes the digest of its outputs to dependents' cache keys rather than its input fingerprint, giving early cutoff when a dependency reruns and produces identical outputs.
GitHub Actions Remote Cache#
Setting ZEBRA_CACHE=github layers GitHubActionsCache behind the local .zebra/ DiskCache (TieredCache). On a local cache miss, Zebra queries the GitHub Actions v2 CacheService Twirp JSON API, verifies downloaded blob sha256 digests, populates <pkg>/.zebra/, and materializes outputs into the workspace. On a build step execution, Zebra writes <pkg>/.zebra/ first and uploads the step's entry to GitHub Actions Cache (ZEBRA_CACHE_READ_ONLY=1 disables uploads; ZEBRA_CACHE=none disables both local and remote caching).
In GitHub Actions workflows, run .github/actions/setup-zebra-cache before build steps so ACTIONS_RESULTS_URL and ACTIONS_RUNTIME_TOKEN are served to run: steps over a 127.0.0.1 token custodian (ZEBRA_GITHUB_CUSTODIAN_URL).
Libraries#
zebra(zebra/main) — CLI entry point (main).zebra/runner—buildTarget,watchTarget,BuildOptions,WatchOptions,reportBuild,reportMigration,formatBuildReport, andBuildOutcome.zebra/zebra-config—zena-package.jsonandzebra.jsonparsing (ZebraWorkspace), synthesizedsourcefilesets, and Wireit migration (migrateWorkspaceToZebra,convertPackageJsonToZebraJson,mergeWireitIntoZenaPackageJson).zebra/wireit-config—package.jsonWireit parsing (WireitWorkspace) and shared script configuration types (ScriptConfig,ScriptRef,DependencySpec,ServiceConfig,EnvBinding,ConfigError,refKey,resolveDependency).zebra/plan-cache—.zebra/plans/snapshot serialization andstat-verified warm plan loading (loadOrCollectConfigs,PlanLoadStats,ConfigProbe).zebra/builder— TranslatesScriptConfigclosures intoworkflowsteps and execution callbacks (planScripts,StepExecution,lockfileCandidates).zebra/disk-cache,zebra/github-cache, andzebra/state-json—.zebra/blob/manifest storage (DiskCache,TieredCache,stateDirOf), GitHub Actions remote caching (GitHubActionsCache,GitHubCacheConfig,CacheMode,resolveCacheMode,resolveGitHubCacheConfig), and JSON serialization for state, manifests, and hash memos.zebra/fs-walkandzebra/fs-util— Input glob discovery (FileHasher,expandGlobs,hashFile,validateGlobs,splitBase) and workspace path utilities.zebra/shell— POSIX shell argument quoting (shellQuote,appendShellArgs). Zebra's terminal output useszena-lang:terminal(packages/terminal): colors, the Clack-style rail layout and the spinner.
License#
MIT
import {
exists,
isFile,
readFile,
readFileBytes,
rename,
writeFile,
writeFileBytes,
} from 'zena:fs';
import {HashSet} from 'zena:collections';
import {StepCache} from 'workflow/cache';
import {CacheKey, Fingerprint} from 'workflow/fingerprint';
import {OutputManifest} from 'workflow/manifest';
import {StepKey} from 'workflow/step';
import {ensureDir, ensureParentDir, withRoot} from './fs-util.zena';
import {manifestFromJson, manifestToJson} from './state-json.zena';
/**
* The package directory of a step key ('pkg/dir:script' — package
* directories never contain ':', script names may).
*/
export function packageDirOf(stepKey: String): String {
for (var i = 0; i < stepKey.length; i += 1) {
if (stepKey.getByteAt(i) == 58) {
return stepKey.sliceBytes(0, i);
}
}
throw new Error('malformed step key: ' + stepKey);
}
/** A package's build-state directory, '<pkg>/.zebra'. */
export function stateDirOf(stepKey: String): String {
let pkg = packageDirOf(stepKey);
return if (pkg == '') '.zebra' else pkg + '/.zebra';
}
/**
* Entries with content fingerprints are files in the workspace;
* anything else (e.g. the '$volatile' marker of an untracked script)
* is virtual and has no bytes to store or restore.
*/
export function isFileEntry(fp: Fingerprint): boolean {
return fp.value.startsWith('sha256:');
}
export function hasVolatileEntry(manifest: OutputManifest): boolean {
for (let entry in manifest.outputs) {
let v = entry.fingerprint.value;
if (v.startsWith('volatile:') || v.startsWith('ref:volatile:')) {
return true;
}
}
return false;
}
export function blobName(fp: Fingerprint): String {
// 'sha256:<hex>' -> 'sha256-<hex>', a safe file name.
let colonSplit = fp.value.split(':');
if (colonSplit.length != 2) {
throw new Error('unexpected fingerprint format: ' + fp.value);
}
return `${colonSplit[0]}-${colonSplit[1]}`;
}
/**
* The local cache, sharded per package like wireit's .wireit/: each
* package's '<pkg>/.zebra/' holds an action cache mapping cache keys to
* manifests (one JSON file per key) and a content-addressed blob store
* for output bytes, so the cache location does not depend on where zebra
* was invoked. A hit materializes the outputs into the workspace
* before returning, so dependents can read them; missing blobs make
* the entry a miss.
*/
export class DiskCache implements StepCache {
#root: String;
#onRestore: (() => void)?;
#onFileRestore: ((path: String, fp: Fingerprint) => Future<void>)?;
#uncachedSteps: HashSet<String>?;
var #nextTempId = 0;
new(
root: String,
onRestore: (() => void)? = null,
onFileRestore: ((path: String, fp: Fingerprint) => Future<void>)? = null,
uncachedSteps: HashSet<String>? = null,
) : #root = root,
#onRestore = onRestore,
#onFileRestore = onFileRestore,
#uncachedSteps = uncachedSteps;
#tempSuffix(): String {
let id = this.#nextTempId;
this.#nextTempId = id + 1;
return `.tmp-${i32ToString(id)}`;
}
#manifestPath(step: StepKey, key: CacheKey): String {
return withRoot(this.#root, `${stateDirOf(step.value)}/cache/${key.value}.json`);
}
#blobPath(step: StepKey, fp: Fingerprint): String {
return withRoot(this.#root, `${stateDirOf(step.value)}/blobs/${blobName(fp)}`);
}
async get(step: StepKey, key: CacheKey): Future<OutputManifest?> {
let uncached = this.#uncachedSteps;
if (uncached != null && uncached.has(step.value)) {
return null;
}
let manifestPath = this.#manifestPath(step, key);
if (!(await isFile(manifestPath))) {
return null;
}
let manifest = manifestFromJson(await readFile(manifestPath));
for (let entry in manifest.outputs) {
if (isFileEntry(entry.fingerprint) && !(await isFile(this.#blobPath(step, entry.fingerprint)))) {
return null;
}
}
let onFileRestore = this.#onFileRestore;
for (let entry in manifest.outputs) {
if (!isFileEntry(entry.fingerprint)) {
continue;
}
let target = withRoot(this.#root, entry.name);
await ensureParentDir(target);
// Restore via sibling temp file + rename so an existing inode (for
// example an mmap'd .cwasm or running binary) is never truncated in place.
let tmp = target + this.#tempSuffix();
await writeFileBytes(tmp, await readFileBytes(this.#blobPath(step, entry.fingerprint)));
await rename(tmp, target);
if (onFileRestore != null) {
await onFileRestore(entry.name, entry.fingerprint);
}
}
let onRestore = this.#onRestore;
if (onRestore != null) {
onRestore();
}
return manifest;
}
async put(step: StepKey, key: CacheKey, manifest: OutputManifest): Future<void> {
let uncached = this.#uncachedSteps;
if ((uncached != null && uncached.has(step.value)) || hasVolatileEntry(manifest)) {
return;
}
let stateDir = withRoot(this.#root, stateDirOf(step.value));
await ensureDir(stateDir + '/blobs');
await ensureDir(stateDir + '/cache');
for (let entry in manifest.outputs) {
if (!isFileEntry(entry.fingerprint)) {
continue;
}
let blob = this.#blobPath(step, entry.fingerprint);
if (!(await exists(blob))) {
let tmp = blob + this.#tempSuffix();
await writeFileBytes(tmp, await readFileBytes(withRoot(this.#root, entry.name)));
await rename(tmp, blob);
}
}
let manifestPath = this.#manifestPath(step, key);
let tmpManifest = manifestPath + this.#tempSuffix();
await writeFile(tmpManifest, manifestToJson(manifest));
await rename(tmpManifest, manifestPath);
}
}
/**
* Two-level step cache (`local` in front of `remote`).
*
* `get` checks `local` first and queries `remote` only when `local` misses.
* `put` writes `local` first so `.zebra/` holds the manifest and blobs before
* `remote` uploads them.
*/
export class TieredCache implements StepCache {
#local: StepCache;
#remote: StepCache;
#uncachedSteps: HashSet<String>?;
new(
local: StepCache,
remote: StepCache,
uncachedSteps: HashSet<String>? = null,
) : #local = local,
#remote = remote,
#uncachedSteps = uncachedSteps;
async get(step: StepKey, key: CacheKey): Future<OutputManifest?> {
let uncached = this.#uncachedSteps;
if (uncached != null && uncached.has(step.value)) {
return null;
}
let localHit = await this.#local.get(step, key);
return localHit ?? await this.#remote.get(step, key);
}
async put(step: StepKey, key: CacheKey, manifest: OutputManifest): Future<void> {
let uncached = this.#uncachedSteps;
if ((uncached != null && uncached.has(step.value)) || hasVolatileEntry(manifest)) {
return;
}
await this.#local.put(step, key, manifest);
await this.#remote.put(step, key, manifest);
}
}
| Version | Published | Dependencies | Approved by |
|---|---|---|---|
| 0.0.1-preview.1 | 2026-11-19 |
|