compiler 0.0.2-preview.1
The Zena self-hosted compiler
compiler#
The Zena compiler, written in Zena. It parses, type-checks, and compiles Zena source code to WebAssembly (Wasm GC) and WebAssembly Text (WAT).
Usage#
Parsing a Single File#
Use compiler/parser to parse Zena source text into an AST (Module):
import { parse } from 'compiler/parser';
let ast = parse('export let answer = 42;', 'example.zena');
Loading and Checking a Program#
Use compiler (createCompiler and checkCompilation) with a CompilerHost to resolve imports across a module graph and type-check a program:
import {
createCompiler, checkCompilation, buildStdlibConfig
} from 'compiler';
import { FsCompilerHost } from 'compiler/fs-host';
let stdlibRoot = '/stdlib';
let host = new FsCompilerHost(stdlibRoot);
let packages = buildStdlibConfig(stdlibRoot, 'host');
let compiler = createCompiler(host, {
stdlibRoot,
packages,
});
let result = await compiler.compile({
entry: '/workspace/main.zena',
target: 'host',
});
let checked = checkCompilation(result);
Compiling a File to WebAssembly#
Use compiler/compile-file to compile an entry file on disk to a .wasm or .wat output file:
import { compileFile } from 'compiler/compile-file';
let ok = await compileFile({
entry: 'src/main.zena',
outPath: 'out/main.wasm',
stdlibRoot: '/stdlib',
});
Libraries#
compiler/compiler/compiler— compiler entry point (Compiler,createCompiler,checkCompilation,ProgramCheckResult)compiler/compile-file— end-to-end file compilation (compileFile,CompileFileOptions)compiler/tokenizer— lexer and token definitions (tokenize,Token,TokenType)compiler/parser— recursive-descent parser (parse,Parser)compiler/ast— AST node definitions (Node,Module,Statement,Expression,TypeAnnotation)compiler/visitor— AST traversal helpers (walk,Visitor)compiler/scope— lexical scope and symbol resolution (ScopeBuilder,ScopeResult,Symbol)compiler/checker— static type checker (checkModule,CheckResult)compiler/semantic-model— semantic queries over a checked module (SemanticModel)compiler/types— compiler type representations (Type,ClassType,FunctionType,typeToString)compiler/diagnostics— compiler diagnostics and source spans (Diagnostic,DiagnosticCode,DiagnosticSeverity,SourceSpan)compiler/package-manifest—zena-package.jsonmanifest and workspace loader (loadWorkspace,loadPackageMap)compiler/module-resolver— package and relative module specifier resolution (ModuleResolver,PackageConfig)compiler/library-loader— module graph loader (LibraryLoader,CompilerHost,SourceFile)compiler/codegen/binary-generator— WebAssembly GC binary emitter (BinaryGenerator)compiler/codegen/wat-generator— WebAssembly Text (WAT) emitter (WatGenerator)
License#
MIT
64.8 KiB
Raw
// Component emission.
//
// Wraps an emitted core module in a WebAssembly component: the component
// contributes types, instantiation, and the canonical `lift`/`lower`
// adapters that convert between component-level values and core-level
// ones, while every instruction still lives in the core module nested
// inside it.
//
// This is the whole of the `component` target's output side. It is
// deliberately a *wrapper* rather than a second backend: nothing here
// touches ZIR, lowering, or the type system, and the core module it
// embeds is byte-for-byte the one the other targets emit.
//
// The types that cross are the flat scalars — `u8`-`u64`, `s8`-`s64`,
// `f32`, `f64`, `bool` — plus `string` on exports. A string means the
// canonical memory options, which name the runtime memory module's
// `memory` and `realloc`; that module is embedded and instantiated
// ahead of the program module, so a memory-carrying lowering has a
// memory that exists first (docs/design/component-emission.md, 1.3).
// Imported interfaces' types come from the WIT type encoder
// (`wit-imports.zena` derives a document from the shape); a string on
// an *import* still waits on the memory-carrying `canon lower`.
//
// The binary format is the Component Model's `Binary.md`. Section ids
// differ from core wasm's and are listed in `ComponentSectionId` below.
import { ByteBuffer } from 'zena:core';
/**
* One section of the component head produced by the WIT type encoder,
* and the counts the emitter continues its index spaces from. These
* mirror `wit-parser`'s `EncodedWorld` rather than importing it: the
* encoder is *injected* by drivers (`wit-imports.zena` builds the
* closure), so that codegen — and everything that links it, like the
* language service — does not carry the WIT parser in its module graph.
*/
export final class EncodedImportPiece {
id: i32;
payload: ByteBuffer;
new(this.id, this.payload);
}
/** One type an exported interface declares itself, for its instance
* to export: the name, and the component-level type index. */
export final class ExportedInstanceType {
name: String;
index: i32;
new(this.name, this.index);
}
/** A type a nested component's instantiation takes from this one. */
export final class NestedInstanceArg {
name: String;
index: i32;
new(this.name, this.index);
}
/**
* An exported interface realized as a nested component rather than a
* bag of exports: what the encoder built (the whole nested component's
* bytes) and what its instantiation supplies — the types by name and
* index, and the lifted functions by name, one per export of the
* instance in `shape.exports` order. An interface with a resource of
* its own needs this: a `[method]r.f` export is validated against the
* resource names a component's own exports establish.
*/
export final class NestedInstanceExport {
bytes: ByteBuffer;
typeArgs = new Array<NestedInstanceArg>();
funcImports = new Array<String>();
new(this.bytes);
}
/** A resource the program provides, whose destructor is the shim's
* trampoline at this position (`DtorSlot` in the encoder). */
export final class ResourceDtorSlot {
qualified: String;
name: String;
new(this.qualified, this.name);
}
/**
* The program's destructor for one of its resources: the resource's
* WIT name, and the core export the wrapper's `<resource>__dtor`
* function is exported under, which the shim's table is filled with.
*/
export final class ComponentDtorExport {
resourceName: String;
coreName: String;
new(this.resourceName, this.coreName);
}
export final class EncodedImports {
pieces = new Array<EncodedImportPiece>();
var typeCount = 0;
var instanceCount = 0;
/** The program's resources, in destructor-slot order. */
dtorSlots = new Array<ResourceDtorSlot>();
/** The exported interfaces realized as nested components, by the
* interface's qualified name. */
nestedExports = new Map<String, NestedInstanceExport>();
/** The types each exported interface declares itself, by the
* interface's qualified name, encoded by the encoder at component
* level; its instance exports each under its name. */
exportedTypes = new Map<String, GrowableArray<ExportedInstanceType>>();
/** Each imported interface's component instance index, by name. */
instanceIndices = new Map<String, i32>();
/**
* Component type indices for the canon builtins' nominal type
* immediates, keyed by the builtin's `canonType` spelling
* (`resource:<interface>#<name>`). Aliased out of the imported
* instances by the encoder — a resource's type is the instance's
* own, and only an alias can name it.
*/
canonTypeIndices = new Map<String, i32>();
/**
* Component type indices for the exports whose lifted type the
* encoder wrote from the WIT rather than the emitter from a core
* signature — the declared world's async exports — by export name.
*/
exportTypeIndices = new Map<String, i32>();
}
/**
* The WIT import encoder, as codegen consumes it. Implemented by
* `wit-imports.zena` and injected by the drivers
* (`BinaryGenerator.importEncoder`) — this indirection is what keeps
* the WIT parser out of codegen's module graph, and with it out of
* hosts like the language service that never emit a component.
*
* `witProvided` answers during shape building: is this interface's WIT
* the encoder's business rather than something to derive from the Zena
* declaration? True for the interfaces the compiler carries WIT for,
* and for *every* interface when a world was declared with `--wit` —
* the declaration is then core-shaped and checked against the WIT in
* `encode`, which also validates a declared world's whole surface
* before encoding it.
*/
export interface ImportTypeEncoder {
witProvided(interfaceName: String): boolean;
encode(shape: ComponentShape): EncodedImports;
}
/**
* Component section ids. A component's sections are a different space
* from a core module's: `1` is a nested core module, not a type section.
*/
export enum ComponentSectionId {
CoreModule = 1,
CoreInstance = 2,
CoreType = 3,
Component = 4,
Instance = 5,
Alias = 6,
Type = 7,
Canon = 8,
Start = 9,
Import = 10,
Export = 11,
Value = 12
}
/**
* Component-level primitive value types. These are the `primvaltype`
* encodings, and they are what a flat-scalar import or export is made
* of. `void` is not among them: a function with no result encodes the
* absence rather than a type.
*/
export enum ComponentValType {
Bool = 0x7f,
S8 = 0x7e,
U8 = 0x7d,
S16 = 0x7c,
U16 = 0x7b,
S32 = 0x7a,
U32 = 0x79,
S64 = 0x78,
U64 = 0x77,
F32 = 0x76,
F64 = 0x75,
/**
* The one non-scalar type so far. It is a single byte like the rest,
* but it is not like the rest: a `string` is bytes in linear memory,
* so every signature mentioning one drags in the canonical options —
* a memory to put them in and a `realloc` to place them with.
*/
String = 0x73
}
/** Whether `t` needs the canonical ABI's memory options to cross. */
export function componentTypeNeedsMemory(t: i32): boolean {
return t == (ComponentValType.String as i32);
}
/** `NO_RESULT` in a signature's `result` field: the function returns nothing. */
export let noComponentResult = -1;
/** A function signature at the component level. */
export final class ComponentFuncType {
/** Parameter types, as `ComponentValType` opcodes. */
paramTypes: GrowableArray<i32>;
/** Parameter names, already kebab-case. Same length as `paramTypes`. */
paramNames: GrowableArray<String>;
/** The result type, or `noComponentResult`. */
result: i32;
/**
* Whether the WIT function is `async`. An async function lowered
* without `async` blocks; lowered with it, the call returns a subtask
* handle instead of waiting. The signature is the same either way —
* see docs/design/component-emission.md 1.6.
*/
isAsync: boolean;
new(this.paramTypes, this.paramNames, this.result, this.isAsync = false);
}
/**
* One function imported from a WIT interface. The core module imports it
* as `(import "<interfaceName>" "<fieldName>")`, which is what a Zena
* `@external("wasi:clocks/monotonic-clock@0.3.0", "now")` already
* compiles to.
*/
export final class ComponentImportFunc {
interfaceName: String;
fieldName: String;
signature: ComponentFuncType;
/** Whether to lower the call with the `async` canonical option. */
lowerAsync: boolean;
/**
* Whether the interface's types come from real WIT source rather
* than being derived from the Zena declaration — the declaration is
* then core-shaped (handles and pointers as `i32`s) and says nothing
* about the WIT types.
*/
var witProvided = false;
/**
* Whether lowering this function needs the canonical memory options:
* a `string` or `list` in the signature, a spilled parameter list,
* or an indirect result. Set from the WIT flattening once the
* document is parsed.
*/
var needsMemoryLower = false;
/**
* The core import's parameter and result counts, recorded for
* WIT-provided imports so `encode` can check the declaration against
* the interface's canonical flattening — the one shape both sides
* must agree on when the types cannot be compared directly.
*/
var coreParamCount = -1;
var coreResultCount = -1;
new(this.interfaceName, this.fieldName, this.signature, this.lowerAsync = false);
}
/**
* The core-module import namespace that means "these are canonical
* builtins, not a WIT interface". A builtin is a function the canonical
* ABI itself provides — creating a waitable set, dropping a subtask —
* so it has no interface to be imported from and no type to declare.
* The core module still imports them like anything else, which is why
* they need a namespace to be imported *from*.
*/
export let canonNamespace = "canon";
/**
* The core export names of the async entry and its callback. Internal
* to the toolchain: the component exports the entry as `main` — the
* program's `main` is what it runs, so the name the host calls does
* not change with how it is lifted — and the callback is not exported
* at the component level at all; the host reaches it only through the
* lift's `callback` option.
*/
export let componentEntryExport = "__zena_component_run";
export let componentCallbackExport = "__zena_component_callback";
/**
* The core exports the canonical options name, when a signature carries
* anything that lives in linear memory.
*
* They are exported under the names `wit-component` and `wit-bindgen`
* use for the same functions, so a component of ours reads the way
* every other one does. `cabi_post_return` is the program module's
* (`zena:wasi`'s); `realloc` and `memory` are the runtime
* memory module's.
*/
export let componentReallocExport = "cabi_realloc";
export let componentPostReturnExport = "cabi_post_return";
/** The runtime memory module's exports, aliased into the lift options. */
export let componentMemoryExport = "memory";
export let componentRuntimeReallocExport = "realloc";
/**
* The core-import namespace the program module uses for the runtime
* memory module's exports — its memory, and the `realloc` behind
* `zena:memory`'s delegating allocator. Reserved: `component-shape`
* does not treat it as a WIT interface, and the emitter satisfies it
* from the runtime core instance at instantiation.
*/
export let componentRuntimeNamespace = "zena:runtime";
/**
* The standard library module holding the guest side of the canonical
* ABI. Only the component target links it, so its presence in the
* program is itself the target check.
*/
/** The library the component ABI marshaling lives in. */
export let componentAbiModule = "wasi";
/**
* The module id of the file declaring the canonical ABI.
*
* `zena:wasi` is a facade: it re-exports three files and declares
* nothing itself, so rooting it roots nothing. Rooting the declaring
* file instead also keeps the async driver and the stream half out of
* a component that only moves strings.
*/
export let componentAbiUnitId = "zena:wasi/abi.zena";
/**
* The module id of the file declaring the async-import driver.
*
* Named for the same reason as `componentAbiUnitId`: the library
* entry point re-exports this file and declares nothing itself, so
* the rooting walk has to name the file that does the declaring.
*/
export let componentDriverUnitId = "zena:wasi/async.zena";
/**
* One canonical builtin the core module imports. `name` is both the
* builtin's spelling in the Component Model and the core import's field
* name.
*/
export final class ComponentCanonBuiltin {
name: String;
/** Only `task.return` carries one; it is the lifted task's result. */
result: i32;
/**
* The WIT type a typed builtin (`stream.*`, `future.*`) operates on,
* as declared (`stream<u8>`, `future`). The canon entry's immediate
* is a component type index, so the type is part of which builtin
* this is. Empty for the untyped builtins.
*/
canonType: String;
/**
* Whether a read/write builtin's canon entry carries the `async`
* option — return BLOCKED instead of waiting for the copy event.
*/
isAsync: boolean;
/**
* A typed `task.return`'s export: the name after `#` in
* `task.return#<name>`, when its `canonType` is a `return:`
* spelling. The return's `memory` option must equal that export's
* lift's, so the emitter reads the need off the export.
*/
returnsFor: String;
// -1 is `noComponentResult`, spelled out: a default initializer
// cannot name a module-level binding.
new(this.name, this.result = -1, this.canonType = "",
this.isAsync = false, this.returnsFor = "");
}
/**
* Whether lifting or lowering `signature` needs the canonical ABI's
* memory options — true as soon as a `string` is anywhere in it, in
* either direction. Arguments are written into guest memory before the
* call and results are read out of it after, and both are the guest
* allocator's business.
*/
export function componentSignatureNeedsMemory(
signature: ComponentFuncType
): boolean {
if (componentTypeNeedsMemory(signature.result)) {
return true;
}
for (let p in signature.paramTypes) {
if (componentTypeNeedsMemory(p)) {
return true;
}
}
return false;
}
/** One core-module export lifted to a component export. */
export final class ComponentExportFunc {
/** The core module's export name. */
coreName: String;
/** The component's export name, kebab-case. */
exportName: String;
signature: ComponentFuncType;
/**
* The core export to lift as this function's callback, or the empty
* string.
*
* A callback-lifted export does not run to completion: it returns a
* code telling the host either that it is done, or which waitable set
* to wake it on. The host re-enters through the callback when
* something in that set fires. That is what makes a timer a timer
* rather than a sleep — the process is not inside the guest while it
* waits. See docs/design/component-emission.md 1.7.
*/
callbackCoreName: String;
/**
* Whether the lifted type comes from the WIT through the encoder
* (`EncodedImports.exportTypeIndices`) rather than from `signature`
* — a declared world's async export, whose parameters and result may
* be any WIT type its wrapper marshals. `signature` then says
* nothing about the types.
*/
var witTyped = false;
/**
* For a WIT-typed export: whether its parameters or result move
* through linear memory, decided by the encoder from the WIT. The
* lift then carries the memory options, and so does the typed
* `task.return` that returns for it.
*/
var needsMemoryLift = false;
/**
* For a WIT-typed synchronous export: whether its result flattens
* past one core value and so comes back through a return area the
* wrapper stages, which `post-return` releases. Decided by the
* encoder from the WIT.
*/
var needsPostReturnLift = false;
/**
* The qualified name of the exported interface this function belongs
* to (`wasi:http/handler@0.3.0`), or empty for a world-level export.
* Exports of one interface are grouped into a component instance
* exported under that name; the declared world assigns it.
*/
var instanceName = "";
new(this.coreName, this.exportName, this.signature,
this.callbackCoreName = "");
/** The key `EncodedImports.exportTypeIndices` files this export under. */
typeKey(): String {
return `${this.instanceName}#${this.exportName}`;
}
/** Whether lifting this export needs a memory and a `realloc`. */
needsMemory(): boolean {
return this.needsMemoryLift ||
componentSignatureNeedsMemory(this.signature);
}
/**
* Whether the host must call `post-return` after reading the result.
*
* Only a result needs one: it is allocated by the guest during the
* call and nothing else can say when the host has finished copying it
* out. Arguments are the host's own buffers. An async export never
* does: its result goes through `task.return`, which the host reads
* before the call returns, and an async lift may not name one.
*/
needsPostReturn(): boolean {
if (this.callbackCoreName != "") {
return false;
}
return this.needsPostReturnLift ||
componentTypeNeedsMemory(this.signature.result);
}
}
/** The imports and exports a component wraps a core module in. */
export final class ComponentShape {
imports = new Array<ComponentImportFunc>();
builtins = new Array<ComponentCanonBuiltin>();
exports = new Array<ComponentExportFunc>();
/** The destructors of the resources the program provides. */
dtorExports = new Array<ComponentDtorExport>();
/**
* Whether the program module imports its memory from the runtime
* memory module — true exactly when it touches linear memory at all,
* since on this target the memory is never its own. The emitter must
* then be handed the runtime module's bytes to embed and instantiate
* first.
*/
var needsRuntimeMemory = false;
/**
* The imported interfaces' types and imports, encoded by the WIT
* type encoder — `buildComponentShape`'s last step, carried here so
* shape and encoding cannot drift apart. Null only for a shape with
* no imports.
*/
var encodedImports: EncodedImports | null = null;
}
// ============================================================================
// Names
// ============================================================================
/**
* Whether `name` is a valid Component Model kebab-case name: words of
* `[a-z][a-z0-9]*` joined by single `-`. Import and export names, and
* parameter names, are all checked by the format itself, so producing an
* invalid one is a hard error rather than a cosmetic slip.
*/
export function isKebabName(name: String): boolean {
if (name.length == 0) {
return false;
}
var wordStart = true;
for (var i = 0; i < name.length; i += 1) {
let c = name.getByteAt(i);
let isLower = c >= 97 && c <= 122; // a-z
let isDigit = c >= 48 && c <= 57; // 0-9
if (c == 45) { // '-'
if (wordStart) {
return false; // empty word: leading or doubled '-'
}
wordStart = true;
} else if (wordStart) {
if (!isLower) {
return false; // a word must open with a lowercase letter
}
wordStart = false;
} else if (!isLower && !isDigit) {
return false;
}
}
return !wordStart; // a trailing '-' leaves an empty word
}
/**
* A Zena identifier as a Component Model name: `camelCase` becomes
* `camel-case`. Zena identifiers cannot contain `-`, so a conversion is
* the only way a Zena name can reach the boundary at all.
*
* Returns the empty string when the identifier has no valid rendering
* (it starts with `_` or a digit, say). Callers decide what to do about
* that; nothing here invents a name.
*/
export function toKebabName(identifier: String): String {
let lowered = identifier.asciiLowerCase();
var out = "";
for (var i = 0; i < identifier.length; i += 1) {
let c = identifier.getByteAt(i);
if (c >= 65 && c <= 90 && i > 0) { // A-Z, not leading
out = out + "-";
}
out = out + lowered.sliceBytes(i, i + 1);
}
if (!isKebabName(out)) {
return "";
}
return out;
}
// ============================================================================
// Encoding
// ============================================================================
function writeName(buffer: ByteBuffer, name: String) {
buffer.writeULEB128(name.length);
for (var i = 0; i < name.length; i += 1) {
buffer.writeByte(name.getByteAt(i));
}
}
/**
* An import or export name: a `0x00` discriminant (the name carries no
* `implements` clause) followed by the name itself.
*/
function writeExternName(buffer: ByteBuffer, name: String) {
buffer.writeByte(0x00);
writeName(buffer, name);
}
/**
* `resultlist`: one unnamed result, or the empty vector of named ones.
*/
function writeResultList(buffer: ByteBuffer, result: i32) {
if (result == noComponentResult) {
// The other arm of `resultlist`: a vector of NAMED results, which
// the format still carries and which is empty here. The count byte
// is not optional — dropping it reads the next declaration's `0x40`
// as a result count.
buffer.writeByte(0x01);
buffer.writeByte(0x00);
} else {
buffer.writeByte(0x00); // one result, unnamed
buffer.writeByte(result);
}
}
/**
* A valtype as the encoder reports one: a primitive's code, stored
* negated (`0x79` for u32 arrives as -0x79), or a component type
* index. Signed on purpose in the index case: the position's grammar
* is `typeidx | primvaltype`, and the primitives are the negative
* SLEB128 space, so an unsigned index of 64..127 would collide.
*/
function writeEncodedValType(buffer: ByteBuffer, code: i32) {
if (code < 0) {
buffer.writeByte(0 - code);
} else {
buffer.writeSLEB128(code);
}
}
/**
* `functype ::= 0x40 ps:vec(<labelvaltype>) rs:<resultlist>`, and `0x43`
* for the async form — a distinct type, not a flag on the same one.
*/
function writeFuncType(buffer: ByteBuffer, signature: ComponentFuncType) {
buffer.writeByte(if (signature.isAsync) 0x43 else 0x40);
buffer.writeULEB128(signature.paramTypes.length);
for (var i = 0; i < signature.paramTypes.length; i += 1) {
writeName(buffer, signature.paramNames[i]);
buffer.writeByte(signature.paramTypes[i]);
}
writeResultList(buffer, signature.result);
}
/**
* A builtin field name without its `#<n>` disambiguator. A per-type
* builtin may be declared at several types — `future.read#0`,
* `future.read#1` — and each declaration is its own core import and
* canon entry, but the opcode is the base name's.
*/
export function canonBuiltinBaseName(name: String): String {
for (var i = 0; i < name.length; i += 1) {
if (name.getByteAt(i) == 35) { // '#'
return name.sliceBytes(0, i);
}
}
return name;
}
/**
* The canonical-builtin opcodes this emitter knows: the waitable-set
* family the async event loop needs, and the stream/future families
* (C6). Resources' builtins wait on the interop stages.
*/
function canonBuiltinOpcode(fieldName: String): i32 {
let name = canonBuiltinBaseName(fieldName);
if (name == "waitable-set.new") { return 0x1f; }
if (name == "waitable-set.wait") { return 0x20; }
if (name == "waitable-set.drop") { return 0x22; }
if (name == "waitable.join") { return 0x23; }
if (name == "subtask.drop") { return 0x0d; }
if (name == "task.return") { return 0x09; }
if (name == "stream.new") { return 0x0e; }
if (name == "stream.read") { return 0x0f; }
if (name == "stream.write") { return 0x10; }
if (name == "stream.drop-readable") { return 0x13; }
if (name == "stream.drop-writable") { return 0x14; }
if (name == "future.new") { return 0x15; }
if (name == "future.read") { return 0x16; }
if (name == "future.write") { return 0x17; }
if (name == "future.drop-readable") { return 0x1a; }
if (name == "future.drop-writable") { return 0x1b; }
if (name == "resource.drop") { return 0x03; }
if (name == "resource.new") { return 0x02; }
if (name == "resource.rep") { return 0x04; }
return -1;
}
/** Whether `name` is a canonical builtin this emitter can encode. */
export function isKnownCanonBuiltin(name: String): boolean {
return canonBuiltinOpcode(name) >= 0;
}
/**
* Whether `name` is a builtin whose canon entry carries a component
* type index — the stream and future families are per-type, so the
* type is part of which builtin the declaration names.
*/
export function isTypedCanonBuiltin(name: String): boolean {
return name.startsWith("stream.") || name.startsWith("future.") ||
name.startsWith("resource.");
}
/** Whether this typed builtin's canon entry also carries canonopts. */
function typedBuiltinTakesOpts(fieldName: String) {
let name = canonBuiltinBaseName(fieldName);
return name == "stream.read" || name == "stream.write" ||
name == "future.read" || name == "future.write";
}
/**
* Whether a typed builtin's payload moves through linear memory —
* which its read/write canon entries then name as an option. A
* `stream<u8>` spelling carries an element; a `future:<iface>#<p>`
* spelling carries one exactly when the payload text is non-empty.
*/
function canonTypeUsesMemory(canonType: String) {
if (canonType.startsWith("future:") || canonType.startsWith("stream:")) {
var hashAt = -1;
for (var i = 0; i < canonType.length; i += 1) {
if (canonType.getByteAt(i) == 35) { hashAt = i; break; } // '#'
}
return hashAt >= 0 && hashAt < canonType.length - 1;
}
return canonType.contains("<");
}
/** The WIT spelling of a primitive, as a component primvaltype, or 0. */
function witPrimByte(name: String): i32 {
if (name == "bool") { return 0x7f; }
if (name == "s8") { return 0x7e; }
if (name == "u8") { return 0x7d; }
if (name == "s16") { return 0x7c; }
if (name == "u16") { return 0x7b; }
if (name == "s32") { return 0x7a; }
if (name == "u32") { return 0x79; }
if (name == "s64") { return 0x78; }
if (name == "u64") { return 0x77; }
if (name == "f32") { return 0x76; }
if (name == "f64") { return 0x75; }
return 0;
}
/**
* The component type a typed builtin's `canonType` spells, as the type
* section body declaring it: `stream<u8>` → `0x66 0x01 0x7d`, bare
* `future` → `0x65 0x00`. Only primitive elements are emittable here —
* a richer element stages through linear memory, and describing it is
* the interop stages' business, not a builtin option string's.
*/
function canonTypeBody(canonType: String) {
var head = canonType;
var element = "";
var open = -1;
for (var i = 0; i < canonType.length; i += 1) {
if (canonType.getByteAt(i) == 60) { // '<'
open = i;
break;
}
}
if (open >= 0) {
if (!canonType.endsWith(">")) {
throw new Error(`Malformed canon type '${canonType}'`);
}
head = canonType.sliceBytes(0, open);
element = canonType.sliceBytes(open + 1, canonType.length - 1);
}
let body = new ByteBuffer();
if (head == "stream") {
body.writeByte(0x66);
} else if (head == "future") {
body.writeByte(0x65);
} else {
throw new Error(
`A canon type names a stream or future, not '${canonType}'`);
}
if (element == "") {
body.writeByte(0x00);
return body;
}
let prim = witPrimByte(element);
if (prim == 0) {
throw new Error(
"The canon type '" + canonType + "' has a non-primitive element; " +
"only primitive elements are emittable today.");
}
body.writeByte(0x01);
body.writeByte(prim);
return body;
}
/** One imported WIT interface and the functions the program calls on it. */
final class ImportedInterface {
name: String;
funcs = new Array<ComponentImportFunc>();
new(this.name);
}
/**
* Group imports by interface, preserving first-appearance order. The
* order is the emitted index order, so it must not depend on hash
* iteration — a module's bytes are a function of its source, not of the
* uid counter (see `resetWasmTypeUids`).
*/
function groupByInterface(
imports: GrowableArray<ComponentImportFunc>
) {
let groups = new Array<ImportedInterface>();
for (let imp in imports) {
var found: ImportedInterface | null = null;
for (let g in groups) {
if (g.name == imp.interfaceName) {
found = g;
}
}
if (found == null) {
let group = new ImportedInterface(imp.interfaceName);
group.funcs.push(imp);
groups.push(group);
} else {
found.funcs.push(imp);
}
}
return groups;
}
function writeSection(out: ByteBuffer, id: i32, payload: ByteBuffer) {
if (payload.length == 0) {
return;
}
out.writeByte(id);
out.writeULEB128(payload.length);
out.writeBuffer(payload);
}
/** The shim table's export name, and the fixup's import of it. */
let dtorTableExport = "$dtors";
/**
* The destructor shim: a core module holding a table of `n` function
* references and one trampoline per entry, `k(rep)` calling the k-th
* reference. Instantiated before the resource types, whose destructors
* name the trampolines; the fixup module below fills the table once
* the program's destructors exist. The shape wit-component emits.
*/
function dtorShimModule(n: i32) {
let m = new ByteBuffer();
m.writeU32(0x6D736100);
m.writeU32(1);
let types = new ByteBuffer();
types.writeULEB128(1);
types.writeByte(0x60); // func
types.writeULEB128(1);
types.writeByte(0x7f); // (param i32)
types.writeULEB128(0); // no result
writeSection(m, 1, types);
let funcs = new ByteBuffer();
funcs.writeULEB128(n);
for (var k = 0; k < n; k += 1) {
funcs.writeULEB128(0);
}
writeSection(m, 3, funcs);
let tables = new ByteBuffer();
tables.writeULEB128(1);
tables.writeByte(0x70); // funcref
tables.writeByte(0x01); // min and max
tables.writeULEB128(n);
tables.writeULEB128(n);
writeSection(m, 4, tables);
let exports = new ByteBuffer();
exports.writeULEB128(n + 1);
for (var k = 0; k < n; k += 1) {
writeName(exports, i32ToString(k));
exports.writeByte(0x00); // func
exports.writeULEB128(k);
}
writeName(exports, dtorTableExport);
exports.writeByte(0x01); // table
exports.writeULEB128(0);
writeSection(m, 7, exports);
let code = new ByteBuffer();
code.writeULEB128(n);
for (var k = 0; k < n; k += 1) {
let body = new ByteBuffer();
body.writeULEB128(0); // no locals
body.writeByte(0x20); // local.get
body.writeULEB128(0);
body.writeByte(0x41); // i32.const
body.writeSLEB128(k);
body.writeByte(0x11); // call_indirect
body.writeULEB128(0); // type 0
body.writeULEB128(0); // table 0
body.writeByte(0x0b); // end
code.writeULEB128(body.length);
code.writeBuffer(body);
}
writeSection(m, 10, code);
return m;
}
/**
* The destructor fixup: a core module importing the shim's table and
* the program's `n` destructors, whose one element segment writes the
* destructors into the table at instantiation.
*/
function dtorFixupModule(n: i32) {
let m = new ByteBuffer();
m.writeU32(0x6D736100);
m.writeU32(1);
let types = new ByteBuffer();
types.writeULEB128(1);
types.writeByte(0x60);
types.writeULEB128(1);
types.writeByte(0x7f);
types.writeULEB128(0);
writeSection(m, 1, types);
let imports = new ByteBuffer();
imports.writeULEB128(n + 1);
for (var k = 0; k < n; k += 1) {
writeName(imports, "");
writeName(imports, i32ToString(k));
imports.writeByte(0x00); // func
imports.writeULEB128(0); // type 0
}
writeName(imports, "");
writeName(imports, dtorTableExport);
imports.writeByte(0x01); // table
imports.writeByte(0x70); // funcref
imports.writeByte(0x01); // min and max
imports.writeULEB128(n);
imports.writeULEB128(n);
writeSection(m, 2, imports);
let elems = new ByteBuffer();
elems.writeULEB128(1);
elems.writeByte(0x00); // active, table 0, an offset expression
elems.writeByte(0x41); // i32.const
elems.writeSLEB128(0);
elems.writeByte(0x0b); // end
elems.writeULEB128(n);
for (var k = 0; k < n; k += 1) {
elems.writeULEB128(k);
}
writeSection(m, 9, elems);
return m;
}
/**
* Emit a component wrapping `coreModule`.
*
* The section order below is the order a validator reads them in, and
* each one's indices refer only to what came before:
*
* type one instance type per imported interface
* import one per interface, referencing its type
* alias one per imported function, out of its instance
* canon `canon lower` per imported function
* core instance one synthesized instance per interface, holding the
* lowered core functions under their field names
* core module the runtime memory module, when the program uses
* linear memory
* core instance instantiate it (it imports nothing)
* core module the Zena module, embedded verbatim
* core instance instantiate it against those
* alias pull each lifted export back out of the instance
* type one functype per export
* canon `canon lift` per export
* export the component's exports
*
* `runtimeModule` is the runtime memory module's bytes, compiled by the
* driver (`component-runtime.zena`); it is embedded and instantiated
* before the program module, which imports its `memory` and `realloc`
* under the reserved namespace. Required exactly when the shape says
* the program uses linear memory — the memory is never the program
* module's own on this target, so that a lowered import's canonical
* options can name it (docs/design/component-emission.md, 1.3).
*/
export function emitComponent(
coreModule: ByteBuffer,
shape: ComponentShape,
runtimeModule: ByteBuffer | null = null
): ByteBuffer {
let out = new ByteBuffer();
// Magic, then version 0x000d and layer 0x0001 — the layer is what
// distinguishes a component from a core module.
out.writeU32(0x6D736100);
out.writeU32(0x0001000D);
let groups = groupByInterface(shape.imports);
// --- the imported interfaces: instance types and imports -----------------
// The WIT type encoder's output, section by section — one instance
// type and one import per interface, `use`d types aliased between
// them. The emitter no longer writes instance types itself; what the
// encoder cannot express, nothing here can either.
let shapeEncoded = shape.encodedImports;
let encoded = if (shapeEncoded == null) {
new EncodedImports()
} else {
shapeEncoded as EncodedImports
};
if (groups.length > 0 && encoded.instanceCount == 0) {
throw new Error(
"This component imports interfaces, and the shape carries no " +
"encoding for them — buildComponentShape encodes them as its " +
"last step."
);
}
// The destructor shim comes before everything: the resource types in
// the encoder's pieces name its trampolines as core functions 0..n-1,
// and its instance is core instance 0. Every core module, instance
// and function index below shifts past it.
let dtorCount = encoded.dtorSlots.length;
let dtorInstances = if (dtorCount > 0) 1 else 0;
var moduleCount = 0;
if (dtorCount > 0) {
writeSection(out, ComponentSectionId.CoreModule as i32,
dtorShimModule(dtorCount));
moduleCount += 1;
let shimInst = new ByteBuffer();
shimInst.writeULEB128(1);
shimInst.writeByte(0x00); // instantiate
shimInst.writeULEB128(0); // the shim module
shimInst.writeULEB128(0); // no arguments
writeSection(out, ComponentSectionId.CoreInstance as i32, shimInst);
let trampolines = new ByteBuffer();
trampolines.writeULEB128(dtorCount);
for (var k = 0; k < dtorCount; k += 1) {
trampolines.writeByte(0x00); // sort: core
trampolines.writeByte(0x00); // core sort: func
trampolines.writeByte(0x01); // target: core instance export
trampolines.writeULEB128(0); // the shim instance
writeName(trampolines, i32ToString(k));
}
writeSection(out, ComponentSectionId.Alias as i32, trampolines);
}
let coreFuncBase = dtorCount;
for (let piece in encoded.pieces) {
writeSection(out, piece.id, piece.payload);
}
// A lowering that carries a string or list needs the canonical
// memory options, and they can only name things that already exist —
// so when any import needs them, the runtime memory module is
// embedded, instantiated and its two exports aliased before the
// lowers. Without such an import it stays where it was, just ahead
// of the program module, and the bytes of every existing component
// are untouched.
var lowersNeedMemory = false;
for (let imp in shape.imports) {
if (imp.needsMemoryLower) { lowersNeedMemory = true; }
}
// A typed read/write builtin moves elements through linear memory,
// and `waitable-set.wait` delivers its event pair through it, so
// their canon entries name the memory — the same early placement a
// memory-carrying lower forces.
// A typed `task.return` names the memory exactly when the export it
// returns for lifts with one.
let typedReturnNeedsMemory = (b: ComponentCanonBuiltin): boolean => {
if (!b.canonType.startsWith("return:")) { return false; }
for (let e in shape.exports) {
if (e.exportName == b.returnsFor && e.needsMemory()) { return true; }
}
return false;
};
for (let b in shape.builtins) {
if (typedBuiltinTakesOpts(b.name) && canonTypeUsesMemory(b.canonType)) {
lowersNeedMemory = true;
}
if (b.name == "waitable-set.wait" || typedReturnNeedsMemory(b)) {
lowersNeedMemory = true;
}
}
if (lowersNeedMemory && !shape.needsRuntimeMemory) {
throw new Error(
"An import needs the canonical memory options, but the program " +
"was emitted without linear memory. These cannot both be true."
);
}
var runtimeInstance = -1;
var earlyReallocCoreFunc = -1;
let writeRuntimeModuleAndInstance = () => {
if (runtimeModule == null) {
throw new Error(
"This program uses linear memory, so its component needs the " +
"runtime memory module, and the driver did not provide one. " +
"Compile it with compileComponentRuntimeModule and pass it to " +
"BinaryGenerator."
);
}
writeSection(out, ComponentSectionId.CoreModule as i32,
runtimeModule as ByteBuffer);
let runtimeInst = new ByteBuffer();
runtimeInst.writeULEB128(1);
runtimeInst.writeByte(0x00); // instantiate
runtimeInst.writeULEB128(moduleCount); // the runtime module, just embedded
runtimeInst.writeULEB128(0); // no arguments
writeSection(out, ComponentSectionId.CoreInstance as i32, runtimeInst);
moduleCount += 1;
};
if (lowersNeedMemory) {
writeRuntimeModuleAndInstance();
runtimeInstance = dtorInstances;
// The memory and the allocator, as core memory 0 and core func 0.
let early = new ByteBuffer();
early.writeULEB128(2);
early.writeByte(0x00); // sort: core
early.writeByte(0x00); // core sort: func
early.writeByte(0x01); // target: core instance export
early.writeULEB128(runtimeInstance);
writeName(early, componentRuntimeReallocExport);
early.writeByte(0x00);
early.writeByte(0x02); // core sort: memory
early.writeByte(0x01);
early.writeULEB128(runtimeInstance);
writeName(early, componentMemoryExport);
writeSection(out, ComponentSectionId.Alias as i32, early);
earlyReallocCoreFunc = coreFuncBase;
}
/** A group's component-level instance index, from the encoder. */
let instanceIndexOf = (name: String): i32 => {
if (let (true, index) = encoded.instanceIndices.get(name)) {
return index;
}
throw new Error(`the WIT encoder imported no instance named '${name}'`);
};
if (groups.length > 0) {
// --- alias: each imported function out of its instance ----------------
let aliases = new ByteBuffer();
var aliasCount = 0;
for (let group in groups) {
aliasCount += group.funcs.length;
}
aliases.writeULEB128(aliasCount);
for (var i = 0; i < groups.length; i += 1) {
for (let f in groups[i].funcs) {
aliases.writeByte(0x01); // sort: func
aliases.writeByte(0x00); // target: instance export
aliases.writeULEB128(instanceIndexOf(groups[i].name));
writeName(aliases, f.fieldName);
}
}
writeSection(out, ComponentSectionId.Alias as i32, aliases);
// --- canon: lower each imported function ------------------------------
let lowers = new ByteBuffer();
lowers.writeULEB128(aliasCount);
var funcIndex = 0;
for (let group in groups) {
for (let f in group.funcs) {
lowers.writeByte(0x01); // canon lower
lowers.writeByte(0x00);
lowers.writeULEB128(funcIndex);
var optionCount = 0;
if (f.lowerAsync) { optionCount += 1; }
if (f.needsMemoryLower) { optionCount += 2; }
lowers.writeULEB128(optionCount);
if (f.lowerAsync) {
lowers.writeByte(0x06); // async
}
if (f.needsMemoryLower) {
// The runtime memory module's, aliased ahead of this section.
lowers.writeByte(0x03); // memory
lowers.writeULEB128(0);
lowers.writeByte(0x04); // realloc
lowers.writeULEB128(earlyReallocCoreFunc);
}
funcIndex += 1;
}
}
writeSection(out, ComponentSectionId.Canon as i32, lowers);
}
// --- canon: the canonical builtins ---------------------------------------
// Not WIT functions: they have no interface, no import and no declared
// type, only an opcode. `task.return` carries the lifted task's own
// result; the stream/future builtins carry the type they operate on
// as a component type index — declared in a type section just ahead,
// continuing the index space the import front matter used — and the
// read/write pairs carry the memory their elements move through.
var canonTypeCount = 0;
if (shape.builtins.length > 0) {
// Resource types are not declared here: they are nominal — the
// imported instance's own — so the encoder alias-exported them and
// reported their indices in `canonTypeIndices`. `future:<iface>#`
// types went the same way — their payloads name the interface's
// types, which only aliases reach. Only the primitive-element
// stream/future spellings get a literal type section.
let canonTypes = new Array<String>();
for (let b in shape.builtins) {
if (b.canonType == "" || b.canonType.startsWith("resource:") ||
b.canonType.startsWith("future:") ||
b.canonType.startsWith("stream:") ||
b.canonType.startsWith("return:")) {
continue;
}
var known = false;
for (let t in canonTypes) {
if (t == b.canonType) { known = true; }
}
if (!known) { canonTypes.push(b.canonType); }
}
canonTypeCount = canonTypes.length;
if (canonTypes.length > 0) {
let types = new ByteBuffer();
types.writeULEB128(canonTypes.length);
for (let t in canonTypes) {
types.writeBuffer(canonTypeBody(t));
}
writeSection(out, ComponentSectionId.Type as i32, types);
}
let canonTypeIndexOf = (canonType: String): i32 => {
if (canonType.startsWith("resource:") ||
canonType.startsWith("future:") ||
canonType.startsWith("stream:") ||
canonType.startsWith("return:")) {
if (let (true, index) = encoded.canonTypeIndices.get(canonType)) {
return index;
}
throw new Error(`the encoder aliased no type for '${canonType}'`);
}
for (var i = 0; i < canonTypes.length; i += 1) {
if (canonTypes[i] == canonType) {
return encoded.typeCount + i;
}
}
throw new Error(`canon type '${canonType}' was not declared`);
};
let builtins = new ByteBuffer();
builtins.writeULEB128(shape.builtins.length);
for (let b in shape.builtins) {
let opcode = canonBuiltinOpcode(b.name);
if (opcode < 0) {
throw new Error(`Unknown canonical builtin '${b.name}'`);
}
builtins.writeByte(opcode);
if (canonBuiltinBaseName(b.name) == "task.return") {
if (b.canonType.startsWith("return:")) {
// The result type the encoder wrote from the WIT, and the
// memory option exactly when the export this returns for
// lifts with one: the canonical ABI traps on a mismatch
// between the two.
builtins.writeByte(0x00); // one result, unnamed
writeEncodedValType(builtins, canonTypeIndexOf(b.canonType));
if (typedReturnNeedsMemory(b)) {
builtins.writeULEB128(1);
builtins.writeByte(0x03); // memory
builtins.writeULEB128(0); // the runtime module's, memory 0
} else {
builtins.writeULEB128(0);
}
} else {
writeResultList(builtins, b.result);
builtins.writeULEB128(0); // no canonical options
}
}
if (b.name == "waitable-set.wait") {
builtins.writeByte(0x00); // not cancellable
builtins.writeULEB128(0); // the event pair lands in memory 0
}
if (isTypedCanonBuiltin(b.name)) {
builtins.writeULEB128(canonTypeIndexOf(b.canonType));
if (typedBuiltinTakesOpts(b.name)) {
// A `future:` payload or a `stream:` element may hold
// strings or lists, which a read lowers into the guest
// through `realloc`; a spelled `stream<u8>`'s bytes land in
// the caller's buffer and need none.
let withRealloc = (b.canonType.startsWith("future:") ||
b.canonType.startsWith("stream:")) &&
canonTypeUsesMemory(b.canonType);
var optionCount = 0;
if (b.isAsync) { optionCount += 1; }
if (canonTypeUsesMemory(b.canonType)) { optionCount += 1; }
if (withRealloc) { optionCount += 1; }
builtins.writeULEB128(optionCount);
if (b.isAsync) {
builtins.writeByte(0x06); // async: BLOCKED instead of waiting
}
if (canonTypeUsesMemory(b.canonType)) {
// The element moves through linear memory: the runtime
// memory module's, aliased as core memory 0 ahead of the
// lowers by the same trigger that placed it early.
builtins.writeByte(0x03); // memory
builtins.writeULEB128(0);
}
if (withRealloc) {
builtins.writeByte(0x04); // realloc
builtins.writeULEB128(earlyReallocCoreFunc);
}
}
}
}
writeSection(out, ComponentSectionId.Canon as i32, builtins);
}
// --- core instance: one per interface, plus one for the builtins --------
// The module imports each of these under the namespace it named in
// `@external`, so the instance an argument is bound to is chosen by
// that name rather than by position.
if (groups.length > 0 || shape.builtins.length > 0) {
let coreInstances = new ByteBuffer();
coreInstances.writeULEB128(
groups.length + (if (shape.builtins.length > 0) 1 else 0));
// Core function indices: the early realloc alias, when the lowers
// needed one, occupies index 0 ahead of the lowered functions.
var loweredIndex = coreFuncBase + (if (lowersNeedMemory) 1 else 0);
for (let group in groups) {
coreInstances.writeByte(0x01); // from inline exports
coreInstances.writeULEB128(group.funcs.length);
for (let f in group.funcs) {
writeName(coreInstances, f.fieldName);
coreInstances.writeByte(0x00); // core sort: func
coreInstances.writeULEB128(loweredIndex);
loweredIndex += 1;
}
}
if (shape.builtins.length > 0) {
coreInstances.writeByte(0x01);
coreInstances.writeULEB128(shape.builtins.length);
for (let b in shape.builtins) {
writeName(coreInstances, b.name);
coreInstances.writeByte(0x00); // core sort: func
coreInstances.writeULEB128(loweredIndex);
loweredIndex += 1;
}
}
writeSection(out, ComponentSectionId.CoreInstance as i32, coreInstances);
}
// --- core module: the runtime memory module, unless already placed ------
// When no lower needed memory it sits here, just ahead of the program
// module, and every existing component keeps its bytes; a
// memory-carrying lower moved it (and its aliases) ahead of the
// lowering section above instead.
if (shape.needsRuntimeMemory && runtimeInstance < 0) {
runtimeInstance = dtorInstances +
groups.length + (if (shape.builtins.length > 0) 1 else 0);
writeRuntimeModuleAndInstance();
}
// --- core module: the Zena module, verbatim -----------------------------
writeSection(out, ComponentSectionId.CoreModule as i32, coreModule);
let programModule = moduleCount;
moduleCount += 1;
// --- core instance: instantiate it against the synthesized instances ----
let instantiate = new ByteBuffer();
instantiate.writeULEB128(1);
instantiate.writeByte(0x00); // instantiate
// The program module's index: after the shim and the runtime module
// when there are any.
instantiate.writeULEB128(programModule);
instantiate.writeULEB128(
groups.length + (if (shape.builtins.length > 0) 1 else 0) +
(if (shape.needsRuntimeMemory) 1 else 0));
// The synthesized instances' indices shift by one when the runtime
// memory module was instantiated ahead of them.
let synthBase = dtorInstances + (if (lowersNeedMemory) 1 else 0);
for (var i = 0; i < groups.length; i += 1) {
// The argument name is the core module's import module name, which
// is the interface string itself.
writeName(instantiate, groups[i].name);
instantiate.writeByte(0x12); // core sort: instance
instantiate.writeULEB128(synthBase + i);
}
if (shape.builtins.length > 0) {
writeName(instantiate, canonNamespace);
instantiate.writeByte(0x12);
instantiate.writeULEB128(synthBase + groups.length);
}
if (shape.needsRuntimeMemory) {
// The program's memory import and the delegated realloc, both
// satisfied by the runtime instance's like-named exports.
writeName(instantiate, componentRuntimeNamespace);
instantiate.writeByte(0x12);
instantiate.writeULEB128(runtimeInstance);
}
writeSection(out, ComponentSectionId.CoreInstance as i32, instantiate);
if (shape.exports.length == 0) {
return out;
}
// The instantiated module is the last core instance, after the
// synthesized ones, the builtins' and the runtime memory module's.
let programInstance = synthBase + groups.length +
(if (shape.builtins.length > 0) 1 else 0) +
(if (shape.needsRuntimeMemory && !lowersNeedMemory) 1 else 0);
// The core function index space: the early realloc alias when the
// lowers needed one, then the lowered imports, then the builtins,
// then everything aliased back out of the program instance.
var coreFuncIndex = coreFuncBase + (if (lowersNeedMemory) 1 else 0);
for (let group in groups) {
coreFuncIndex += group.funcs.length;
}
coreFuncIndex += shape.builtins.length;
let firstExportCoreFunc = coreFuncIndex;
// Every distinct callback, aliased after the exports so a lift can
// name one. Distinct: several exports may share one callback, and
// aliasing it twice would be two core functions where one will do.
let callbackNames = new Array<String>();
for (let e in shape.exports) {
if (e.callbackCoreName != "") {
var seen = false;
for (let name in callbackNames) {
if (name == e.callbackCoreName) { seen = true; }
}
if (!seen) { callbackNames.push(e.callbackCoreName); }
}
}
let callbackCoreIndex = (name: String): i32 => {
for (var i = 0; i < callbackNames.length; i += 1) {
if (callbackNames[i] == name) {
return firstExportCoreFunc + shape.exports.length + i;
}
}
throw new Error(`No alias emitted for callback '${name}'`);
};
// A `string` anywhere in a signature means the canonical options: the
// memory the bytes live in, and the allocator that placed them. Both
// are the runtime memory module's — the one owner of the pages — and
// `post-return` is the program's, because reading the return area is
// its business. A signature that needs them without the runtime
// module existing is a contradiction: strings live in linear memory,
// and touching linear memory is what brings the runtime module in.
var needsMemory = false;
var needsPostReturn = false;
for (let e in shape.exports) {
if (e.needsMemory()) { needsMemory = true; }
if (e.needsPostReturn()) { needsPostReturn = true; }
}
if (needsMemory && !shape.needsRuntimeMemory) {
throw new Error(
"An export's signature carries a string, but the program module " +
"was emitted without linear memory. These cannot both be true."
);
}
// When the lowers already aliased the runtime module's realloc and
// memory, the lifts reuse them; otherwise they are aliased here,
// after the exports and callbacks.
let reallocCoreFunc = if (earlyReallocCoreFunc >= 0) {
earlyReallocCoreFunc
} else {
firstExportCoreFunc + shape.exports.length + callbackNames.length
};
let postReturnCoreFunc = firstExportCoreFunc + shape.exports.length +
callbackNames.length + (if (earlyReallocCoreFunc >= 0) 0 else 1);
// --- alias: pull each exported core function out of the instance --------
let exportAliases = new ByteBuffer();
var aliasTotal = shape.exports.length + callbackNames.length;
if (needsMemory && earlyReallocCoreFunc < 0) {
aliasTotal += 2; // realloc, then the memory itself
}
if (needsPostReturn) {
aliasTotal += 1;
}
exportAliases.writeULEB128(aliasTotal);
for (let e in shape.exports) {
exportAliases.writeByte(0x00); // sort: core
exportAliases.writeByte(0x00); // core sort: func
exportAliases.writeByte(0x01); // target: core instance export
exportAliases.writeULEB128(programInstance);
writeName(exportAliases, e.coreName);
}
for (let name in callbackNames) {
exportAliases.writeByte(0x00);
exportAliases.writeByte(0x00);
exportAliases.writeByte(0x01);
exportAliases.writeULEB128(programInstance);
writeName(exportAliases, name);
}
if (needsMemory && earlyReallocCoreFunc < 0) {
// The core function index space continues past the callbacks, so
// realloc and post-return are aliased in that order to match
// `reallocCoreFunc` above. The allocator and the memory are the
// runtime module's; `cabi_post_return` is the program's.
exportAliases.writeByte(0x00);
exportAliases.writeByte(0x00);
exportAliases.writeByte(0x01);
exportAliases.writeULEB128(runtimeInstance);
writeName(exportAliases, componentRuntimeReallocExport);
if (needsPostReturn) {
exportAliases.writeByte(0x00);
exportAliases.writeByte(0x00);
exportAliases.writeByte(0x01);
exportAliases.writeULEB128(programInstance);
writeName(exportAliases, componentPostReturnExport);
}
// Core memories have an index space of their own, so this is memory
// 0 whatever the functions did.
exportAliases.writeByte(0x00);
exportAliases.writeByte(0x02); // core sort: memory
exportAliases.writeByte(0x01);
exportAliases.writeULEB128(runtimeInstance);
writeName(exportAliases, componentMemoryExport);
} else if (needsPostReturn) {
exportAliases.writeByte(0x00);
exportAliases.writeByte(0x00);
exportAliases.writeByte(0x01);
exportAliases.writeULEB128(programInstance);
writeName(exportAliases, componentPostReturnExport);
}
writeSection(out, ComponentSectionId.Alias as i32, exportAliases);
// --- type: one functype per export --------------------------------------
// Except the WIT-typed ones, whose functype the encoder already
// wrote among the import front matter's types; the rest continue
// the component type index space after the front matter's types and
// the canon-builtin types section, when there was one.
let firstExportType = encoded.typeCount + canonTypeCount;
let exportTypeIndex = new Array<i32>();
let exportTypes = new ByteBuffer();
var declaredExportTypes = 0;
for (let e in shape.exports) {
if (let (true, index) = encoded.exportTypeIndices.get(e.typeKey())) {
exportTypeIndex.push(index);
continue;
}
exportTypeIndex.push(firstExportType + declaredExportTypes);
declaredExportTypes += 1;
writeFuncType(exportTypes, e.signature);
}
if (declaredExportTypes > 0) {
let section = new ByteBuffer();
section.writeULEB128(declaredExportTypes);
section.writeBuffer(exportTypes);
writeSection(out, ComponentSectionId.Type as i32, section);
}
// --- canon: lift each export --------------------------------------------
let lifts = new ByteBuffer();
lifts.writeULEB128(shape.exports.length);
for (var i = 0; i < shape.exports.length; i += 1) {
let e = shape.exports[i];
lifts.writeByte(0x00); // canon lift
lifts.writeByte(0x00);
lifts.writeULEB128(firstExportCoreFunc + i);
var optionCount = 0;
if (e.callbackCoreName != "") {
optionCount += 2; // async, and the callback to re-enter through
}
if (e.needsMemory()) {
optionCount += 2; // the memory, and the realloc that fills it
}
if (e.needsPostReturn()) {
optionCount += 1;
}
lifts.writeULEB128(optionCount);
if (e.callbackCoreName != "") {
// The callback form takes both options together: the lifted
// function is async, and this is what the host re-enters through.
lifts.writeByte(0x06); // async
lifts.writeByte(0x07); // callback
lifts.writeULEB128(callbackCoreIndex(e.callbackCoreName));
}
if (e.needsMemory()) {
lifts.writeByte(0x03); // memory
lifts.writeULEB128(0); // the one aliased above
lifts.writeByte(0x04); // realloc
lifts.writeULEB128(reallocCoreFunc);
}
if (e.needsPostReturn()) {
lifts.writeByte(0x05); // post-return
lifts.writeULEB128(postReturnCoreFunc);
}
lifts.writeULEB128(exportTypeIndex[i]);
}
writeSection(out, ComponentSectionId.Canon as i32, lifts);
// --- the destructor fixup ------------------------------------------------
// The shim's table and the program's destructors, aliased, gathered
// into a core instance the fixup module is instantiated against; its
// element segment writes the destructors into the table. From here
// on a dropped handle reaches the program.
if (dtorCount > 0) {
let coreFuncsSoFar = firstExportCoreFunc + shape.exports.length +
callbackNames.length +
(if (needsMemory && earlyReallocCoreFunc < 0) 1 else 0) +
(if (needsPostReturn) 1 else 0);
let fixupAliases = new ByteBuffer();
fixupAliases.writeULEB128(dtorCount + 1);
fixupAliases.writeByte(0x00); // sort: core
fixupAliases.writeByte(0x01); // core sort: table
fixupAliases.writeByte(0x01); // target: core instance export
fixupAliases.writeULEB128(0); // the shim instance
writeName(fixupAliases, dtorTableExport);
for (let slot in encoded.dtorSlots) {
var coreName = "";
for (let d in shape.dtorExports) {
if (d.resourceName == slot.name) {
if (coreName != "") {
throw new Error("Two of the program's resources are named '" +
slot.name + "'; their destructors cannot be told apart yet.");
}
coreName = d.coreName;
}
}
if (coreName == "") {
throw new Error("The program provides resource '" + slot.name +
"' of '" + slot.qualified + "', and its wrapper exports no " +
"destructor for it.");
}
fixupAliases.writeByte(0x00);
fixupAliases.writeByte(0x00); // core sort: func
fixupAliases.writeByte(0x01);
fixupAliases.writeULEB128(programInstance);
writeName(fixupAliases, coreName);
}
writeSection(out, ComponentSectionId.Alias as i32, fixupAliases);
let fixupArgs = new ByteBuffer();
fixupArgs.writeULEB128(1);
fixupArgs.writeByte(0x01); // from inline exports
fixupArgs.writeULEB128(dtorCount + 1);
writeName(fixupArgs, dtorTableExport);
fixupArgs.writeByte(0x01); // core sort: table
fixupArgs.writeULEB128(0); // the one core table aliased
for (var k = 0; k < dtorCount; k += 1) {
writeName(fixupArgs, i32ToString(k));
fixupArgs.writeByte(0x00); // core sort: func
fixupArgs.writeULEB128(coreFuncsSoFar + k);
}
writeSection(out, ComponentSectionId.CoreInstance as i32, fixupArgs);
writeSection(out, ComponentSectionId.CoreModule as i32,
dtorFixupModule(dtorCount));
let fixup = new ByteBuffer();
fixup.writeULEB128(1);
fixup.writeByte(0x00); // instantiate
fixup.writeULEB128(moduleCount); // the fixup module, just embedded
fixup.writeULEB128(1);
writeName(fixup, "");
fixup.writeByte(0x12); // core sort: instance
fixup.writeULEB128(programInstance + 1); // the arguments instance
writeSection(out, ComponentSectionId.CoreInstance as i32, fixup);
moduleCount += 1;
}
// --- instance + export ----------------------------------------------------
// Component function indices: the aliased imports come first, then the
// lifted exports. A world-level export is exported as the function
// itself. The functions of an exported interface are gathered into a
// component instance exported under the interface's name, so the host
// finds `handle` inside `wasi:http/handler@0.3.0`; the instances take
// the indices after the imported ones.
var firstExportFunc = 0;
for (let group in groups) {
firstExportFunc += group.funcs.length;
}
let instanceNames = new Array<String>();
for (let e in shape.exports) {
if (e.instanceName == "") { continue; }
var seen = false;
for (let name in instanceNames) {
if (name == e.instanceName) { seen = true; }
}
if (!seen) { instanceNames.push(e.instanceName); }
}
if (instanceNames.length > 0) {
// An interface with a resource of its own is a nested component,
// written first so the instance section can instantiate it: the
// components take indices in this order.
let nestedIndex = new Map<String, i32>();
for (let name in instanceNames) {
if (let (true, nested) = encoded.nestedExports.get(name)) {
nestedIndex[name] = nestedIndex.size;
writeSection(out, ComponentSectionId.Component as i32, nested.bytes);
}
}
let instances = new ByteBuffer();
instances.writeULEB128(instanceNames.length);
for (let name in instanceNames) {
if (let (true, nested) = encoded.nestedExports.get(name)) {
instances.writeByte(0x00); // instantiate a component
instances.writeULEB128(nestedIndex[name]);
var funcs = 0;
for (let e in shape.exports) {
if (e.instanceName == name) { funcs += 1; }
}
if (funcs != nested.funcImports.length) {
throw new Error("The nested component for '" + name +
"' imports " + i32ToString(nested.funcImports.length) +
" function(s), and the program lifts " + i32ToString(funcs));
}
instances.writeULEB128(nested.typeArgs.length + funcs);
for (let t in nested.typeArgs) {
writeName(instances, t.name);
instances.writeByte(0x03); // sort: type
instances.writeULEB128(t.index);
}
var k = 0;
for (var i = 0; i < shape.exports.length; i += 1) {
if (shape.exports[i].instanceName != name) { continue; }
writeName(instances, nested.funcImports[k]);
instances.writeByte(0x01); // sort: func
instances.writeULEB128(firstExportFunc + i);
k += 1;
}
continue;
}
instances.writeByte(0x01); // an instance from inline exports
// The interface's own types first, so the instance reads back
// with them, then its functions.
var ownTypes =
new GrowableArray<ExportedInstanceType>();
if (let (true, ts) = encoded.exportedTypes.get(name)) {
ownTypes = ts;
}
var count = ownTypes.length;
for (let e in shape.exports) {
if (e.instanceName == name) { count += 1; }
}
instances.writeULEB128(count);
for (let t in ownTypes) {
writeExternName(instances, t.name);
instances.writeByte(0x03); // sort: type
instances.writeULEB128(t.index);
}
for (var i = 0; i < shape.exports.length; i += 1) {
let e = shape.exports[i];
if (e.instanceName != name) { continue; }
writeExternName(instances, e.exportName);
instances.writeByte(0x01); // sort: func
instances.writeULEB128(firstExportFunc + i);
}
}
writeSection(out, ComponentSectionId.Instance as i32, instances);
}
let exports = new ByteBuffer();
var exportCount = instanceNames.length;
for (let e in shape.exports) {
if (e.instanceName == "") { exportCount += 1; }
}
exports.writeULEB128(exportCount);
for (var i = 0; i < shape.exports.length; i += 1) {
let e = shape.exports[i];
if (e.instanceName != "") { continue; }
writeExternName(exports, e.exportName);
exports.writeByte(0x01); // sort: func
exports.writeULEB128(firstExportFunc + i);
exports.writeByte(0x00); // no ascribed type
}
for (var k = 0; k < instanceNames.length; k += 1) {
writeExternName(exports, instanceNames[k]);
exports.writeByte(0x05); // sort: instance
exports.writeULEB128(encoded.instanceCount + k);
exports.writeByte(0x00); // no ascribed type
}
writeSection(out, ComponentSectionId.Export as i32, exports);
return out;
}
- zena-lang:wit-parser wit-parser.zena-lang.pkg.zena-lang.dev ~0.0.1-preview.1
| Version | Published | Dependencies | Approved by |
|---|---|---|---|
| 0.0.2-preview.1 | 2026-11-19 |
|
|