google-cloud 0.0.1-preview.1
Google Cloud clients: Firestore, Cloud Storage, Cloud DNS and Cloud KMS
google-cloud#
Google Cloud client libraries for Zena, built on wasi:http and @zena/protobuf so they compile for WebAssembly component targets as well as the zena CLI host.
Installation#
zena pkg add @zena/google-cloud
Libraries#
google-cloud/firestore— Firestore document, collection, query, batch, and transaction client.google-cloud/storage— Cloud Storage client for reading, writing, stating, and deleting objects.google-cloud/dns— Cloud DNS client for managing record sets in a managed zone.google-cloud/kms— Cloud KMS client for asymmetric signing and symmetric encryption/decryption.google-cloud/auth— Default credentials and project ID discovery from the GCE metadata server or environment variables.google-cloud/firestore-v1andgoogle-cloud/storage-v2— Generated protobuf messages and low-level service clients forgoogle.firestore.v1andgoogle.storage.v2.google-cloud/testing/fake-kmsandgoogle-cloud/testing/firestore-emulator— Test helpers for in-process fake Cloud KMS and the Firestore emulator.
Firestore#
import { Direction, Fields, Firestore, Operator, integerValue } from 'google-cloud/firestore';
let db = await Firestore.connect();
let post = db.collection('posts').doc('hello');
await post.set(new Fields().string('title', 'Hello').integer('views', 0));
let snapshot = await post.get();
if (let (true, views) = snapshot.getInteger('views')) {
await post.update(new Fields().integer('views', views + 1));
}
let popular = await db.collection('posts')
.where('views', Operator.GreaterThan, integerValue(100))
.orderBy('views', Direction.Descending)
.limit(10)
.get();
for (let p in popular) {
console.log(p.id);
}
Firestore.connect() discovers the project and credentials as described under Credentials and talks gRPC to the API. new Firestore({projectId, databaseId?, transport}) accepts explicit configuration instead.
- References:
db.collection(path),db.doc(path),collection.doc(id),doc.collection(id).collection.add(fields)creates a document with a random ID. - Writes:
setreplaces a document,createfails withALREADY_EXISTSwhen it exists,updatesets the named top-level fields and fails withNOT_FOUNDwhen it does not exist, anddeleteremoves it. Errors areRpcErrors carrying thegoogle.rpc.Code. - Transforms (applied by the server when the write commits):
Fields.serverTimestamp,increment,incrementDouble,maximum,minimum,arrayUnion, andarrayRemove. In anupdate,deleteFieldremoves a field. They apply to top-level fields and work in batches and transactions as well:post.update(new Fields().increment('views', 1).serverTimestamp('seen')). - Reads:
get()returns aDocumentSnapshot, whoseexistsisfalsefor a missing document.getString,getInteger,getDouble, andgetBooleantake a field path such asauthor.nameand return(true, value)or(false, _)when the field is absent or of another type.get(path)returns the rawValue.db.getAll(refs)reads several documents in one call in the order given. - Queries:
where,orderBy,limit,offset, andselecteach return a new query.get()collects the results;stream()yields each document as it arrives (for await);count()counts them on the server. - Cursors:
startAt,startAfter,endAt, andendBeforetake values for the orderings or aDocumentSnapshot. - Batches & Transactions:
db.batch()collects writes and commits them in one call.db.runTransaction(body)runsbodywith aTransaction, retrying up to five times on contention and rolling back ifbodythrows. - Deadlines:
Firestore.connect({timeout: milliseconds(10000)})gives each call a deadline, failing withDEADLINE_EXCEEDEDif exceeded.
let total = await db.runTransaction<i64>(async (tx: Transaction): Future<i64> => {
let counter = db.doc('counters/visits');
let snapshot = await tx.get(counter);
var n: i64 = 0;
if (let (true, stored) = snapshot.getInteger('n')) {
n = stored;
}
tx.set(counter, new Fields().integer('n', n + 1));
return n + 1;
});
Transports & Emulator#
Firestore.connect()uses gRPC (protobuf/grpc) over HTTP/2. WhenFIRESTORE_EMULATOR_HOSTis set tohost:port,Firestore.connect()connects to the Firestore emulator over cleartext HTTP/2 with theownertoken.Firestore.connectFallback()uses Google's gRPC fallback protocol (protobuf/fallback), which works on HTTP/1.1-onlywasi:httphosts such as stockwasmtime serve.new Firestore({projectId, transport})accepts anyProtoTransport.
Cloud Storage#
import { Storage } from 'google-cloud/storage';
let bucket = Storage.connect().bucket('my-bucket');
await bucket.write('notes/hello.txt', bytes, 'text/plain');
let read = await bucket.read('notes/hello.txt'); // null when absent
let object = await bucket.stat('notes/hello.txt'); // metadata, or null
await bucket.delete('notes/hello.txt'); // true if deleted
Storage.connect() uses the default credentials and talks gRPC to the Cloud Storage API (write streams data in chunks of at most 2 MiB). new Storage(transport) accepts any ProtoTransport, and storage.client exposes the generated StorageClient.
Cloud DNS#
import { CloudDns, RecordSet } from 'google-cloud/dns';
let zone = CloudDns.connect('my-project', 'my-zone');
await zone.put(new RecordSet('_atproto.alice.example.com.', 'TXT', 300, ['"did=did:plc:..."']));
let set = await zone.get('_atproto.alice.example.com.', 'TXT'); // null when absent
await zone.delete('_atproto.alice.example.com.', 'TXT'); // true if deleted
CloudDns uses the Cloud DNS REST API (https://dns.googleapis.com/dns/v1) with default credentials. Names are fully qualified (ending in a .), and put creates or replaces the record set of that name and type.
Cloud KMS#
import { sha256 } from 'zena:crypto';
import { CloudKms, ecPointFromPem } from 'google-cloud/kms';
// Asymmetric signing
let signingKey = CloudKms.connect(
'projects/p/locations/l/keyRings/r/cryptoKeys/k/cryptoKeyVersions/1'
);
let derSignature = await signingKey.asymmetricSign(sha256(message));
let pubKey = await signingKey.publicKey();
let uncompressedPoint = ecPointFromPem(pubKey.pem);
// Symmetric encryption (AEAD)
let symmetricKey = CloudKms.connect('projects/p/locations/l/keyRings/r/cryptoKeys/s');
let ciphertext = await symmetricKey.encrypt(plaintext, additionalData);
let decrypted = await symmetricKey.decrypt(ciphertext, additionalData);
Credentials#
defaultCredentials() (google-cloud/auth):
GOOGLE_OAUTH_ACCESS_TOKEN, when set in the environment, is used directly (for local runs:GOOGLE_OAUTH_ACCESS_TOKEN=$(gcloud auth print-access-token)).- Otherwise, the GCE metadata server (at
GCE_METADATA_HOSTwhen set, ormetadata.google.internal) supplies tokens for the default service account, caching each token until a minute before it expires.
defaultProjectId() reads GOOGLE_CLOUD_PROJECT or GCLOUD_PROJECT when set, falling back to the metadata server.
License#
MIT
/**
* A Firestore client.
*
* let db = await Firestore.connect();
* let post = db.collection('posts').doc('hello');
* await post.set(new Fields().string('title', 'Hello').integer('views', 0));
* let snapshot = await post.get();
* if (let (true, views) = snapshot.getInteger('views')) { ... }
*
* let popular = await db.collection('posts')
* .where('views', Operator.GreaterThan, integerValue(100))
* .orderBy('views', Direction.Descending)
* .limit(10)
* .get();
*
* Requests go through the generated `FirestoreClient` (`./firestore-v1.zena`)
* over a `ProtoTransport`. `Firestore.connect()` uses gRPC, which needs a
* host whose outgoing HTTP speaks HTTP/2, such as `zena`, and reaches
* the emulator when `FIRESTORE_EMULATOR_HOST` is set, as Google's own
* client libraries do. `Firestore.connectFallback()` uses Google's gRPC
* fallback protocol, which works on any `wasi:http` host. Values are the
* generated `Value` views; `Fields` and the `*Value` functions build them.
*/
import { AsyncIterable, AsyncIterator, AsyncStep } from 'zena:async';
import { HashMap } from 'zena:collections';
import { getEnv } from 'zena:cli';
import { Option } from 'zena:core';
import { randomInt } from 'zena:random';
import { Duration, milliseconds, sleep } from 'zena:time';
import { Credentials, ProtoBytes, ProtoTransport, ProtoView, RpcError } from 'protobuf/runtime';
import { FallbackTransport } from 'protobuf/fallback';
import { GrpcTransport } from 'protobuf/grpc';
import { AccessTokenCredentials, defaultCredentials, defaultProjectId } from './auth.zena';
import {
AggregationResult,
ArrayValue,
Cursor,
Document,
DocumentMask,
DocumentTransformFieldTransform,
DocumentTransformFieldTransformServerValue,
FirestoreClient,
Int32Value,
MapValue,
NullValue,
Precondition,
RunQueryResponse,
StructuredAggregationQuery,
StructuredAggregationQueryAggregation,
StructuredAggregationQueryAggregationCount,
StructuredQuery,
StructuredQueryCollectionSelector,
StructuredQueryCompositeFilter,
StructuredQueryCompositeFilterOperator,
StructuredQueryDirection,
StructuredQueryFieldFilter,
StructuredQueryFieldFilterOperator,
StructuredQueryFieldReference,
StructuredQueryFilter,
StructuredQueryOrder,
StructuredQueryProjection,
Timestamp,
TransactionOptions,
TransactionOptionsReadWrite,
Value,
Write
} from './firestore-v1.zena';
export {
StructuredQueryFieldFilterOperator as Operator,
StructuredQueryDirection as Direction,
ArrayValue,
MapValue,
Value,
Timestamp,
Document
} from './firestore-v1.zena';
export { RpcError } from 'protobuf/runtime';
/** The Firestore API's origin. */
export let FIRESTORE_ORIGIN = 'https://firestore.googleapis.com';
/** The project id `connect` uses with the emulator when none is set. */
let EMULATOR_PROJECT = 'demo-zena';
/** The field path that stands for a document's name. */
let NAME_FIELD = '__name__';
/** The alias `Query.count` gives its aggregation. */
let COUNT_ALIAS = 'count';
/** `google.rpc.Code` values the client acts on. */
let CODE_NOT_FOUND = 5;
let CODE_ABORTED = 10;
/** A string value. */
export function stringValue(value: String): Value {
return Value.of({stringValue: value});
}
/** A 64-bit integer value. */
export function integerValue(value: i64): Value {
return Value.of({integerValue: value});
}
/** A double value. */
export function doubleValue(value: f64): Value {
return Value.of({doubleValue: value});
}
/** A boolean value. */
export function booleanValue(value: boolean): Value {
return Value.of({booleanValue: value});
}
/** The null value. */
export function nullValue(): Value {
return Value.of({nullValue: NullValue.NullValue});
}
/** A timestamp value: seconds since the Unix epoch, and nanoseconds. */
export function timestampValue(seconds: i64, nanos: i32 = 0): Value {
return Value.of({timestampValue: Timestamp.of({seconds, nanos})});
}
/** A bytes value. */
export function bytesValue(value: ByteArray): Value {
return Value.of({bytesValue: value});
}
/** A reference to another document. */
export function referenceValue(document: DocumentReference): Value {
return Value.of({referenceValue: document.name});
}
/** An array value. */
export function arrayValue(values: Array<Value>): Value {
return Value.of({arrayValue: ArrayValue.of({values})});
}
/**
* A map value. Transforms and deletions apply to a document's top-level
* fields only, so `fields` must have none.
*/
export function mapValue(fields: Fields): Value {
if (fields.hasOperations) {
throw new Error('field transforms and deletions apply to top-level fields only');
}
return Value.of({mapValue: MapValue.of({fields: fields.toMap()})});
}
/**
* A document's fields, built one at a time:
* `new Fields().string('title', 'Hello').integer('views', 0)`.
*
* Besides values, a field can be given a transform, which the server
* applies to the stored document when the write commits
* (`serverTimestamp`, `increment`, `arrayUnion` and the rest), or, in an
* `update`, be deleted (`deleteField`). A transform or deletion replaces
* an earlier value, transform or deletion of the field. A field given a
* value after a transform or deletion has both, which the server
* rejects.
*/
export final class Fields {
#map: HashMap<String, Value> = new HashMap<String, Value>();
/**
* The transformed and deleted fields, made by the first of them, so a
* program with none carries no code for them.
*/
var #operations: WriteOperations | null = null;
/** Sets field `name` to `value`, replacing an earlier value. */
set(name: String, value: Value): Fields {
this.#map[name] = value;
return this;
}
string(name: String, value: String): Fields {
return this.set(name, stringValue(value));
}
integer(name: String, value: i64): Fields {
return this.set(name, integerValue(value));
}
double(name: String, value: f64): Fields {
return this.set(name, doubleValue(value));
}
boolean(name: String, value: boolean): Fields {
return this.set(name, booleanValue(value));
}
null_(name: String): Fields {
return this.set(name, nullValue());
}
timestamp(name: String, seconds: i64, nanos: i32 = 0): Fields {
return this.set(name, timestampValue(seconds, nanos));
}
map(name: String, fields: Fields): Fields {
return this.set(name, mapValue(fields));
}
array(name: String, values: Array<Value>): Fields {
return this.set(name, arrayValue(values));
}
/** Sets field `name` to the time the server commits the write. */
serverTimestamp(name: String): Fields {
return this.#transform(name, DocumentTransformFieldTransform.of({
fieldPath: quoteFieldName(name),
setToServerValue: DocumentTransformFieldTransformServerValue.RequestTime,
}));
}
/** Adds `by` to integer field `name`, which counts as 0 when it is absent. */
increment(name: String, by: i64): Fields {
return this.#transform(name, DocumentTransformFieldTransform.of({
fieldPath: quoteFieldName(name), increment: integerValue(by),
}));
}
/** Adds `by` to number field `name`, which counts as 0 when it is absent. */
incrementDouble(name: String, by: f64): Fields {
return this.#transform(name, DocumentTransformFieldTransform.of({
fieldPath: quoteFieldName(name), increment: doubleValue(by),
}));
}
/** Sets number field `name` to `value` when that is larger, or when the field is absent. */
maximum(name: String, value: Value): Fields {
return this.#transform(name, DocumentTransformFieldTransform.of({
fieldPath: quoteFieldName(name), maximum: value,
}));
}
/** Sets number field `name` to `value` when that is smaller, or when the field is absent. */
minimum(name: String, value: Value): Fields {
return this.#transform(name, DocumentTransformFieldTransform.of({
fieldPath: quoteFieldName(name), minimum: value,
}));
}
/** Appends to array field `name` each of `values` it does not hold yet. */
arrayUnion(name: String, values: Array<Value>): Fields {
return this.#transform(name, DocumentTransformFieldTransform.of({
fieldPath: quoteFieldName(name), appendMissingElements: ArrayValue.of({values}),
}));
}
/** Removes every occurrence of each of `values` from array field `name`. */
arrayRemove(name: String, values: Array<Value>): Fields {
return this.#transform(name, DocumentTransformFieldTransform.of({
fieldPath: quoteFieldName(name), removeAllFromArray: ArrayValue.of({values}),
}));
}
/** Deletes field `name` in an `update`. In `set` and `create`, the field is simply left out. */
deleteField(name: String): Fields {
this.#operation(name, null);
return this;
}
/** The names of the fields given values, in no particular order. */
names(): GrowableArray<String> {
let names = new GrowableArray<String>();
for (let entry in this.#map) {
names.push(entry.key);
}
return names;
}
/**
* Adds the quoted paths of the deleted fields to `paths`, and answers the
* transforms as the `update_transforms` of an encoded `Write`, to merge
* into the write's own encoding; empty when there are none.
*/
operationsPart(paths: GrowableArray<String>): ProtoView {
let operations = this.#operations;
return if (operations == null) ProtoView.empty() else operations.encode(paths);
}
/** Whether any field has a transform or is deleted. */
hasOperations: boolean {
get { return this.#operations != null; }
}
toMap(): HashMap<String, Value> {
return this.#map;
}
#transform(name: String, transform: DocumentTransformFieldTransform): Fields {
this.#operation(name, transform.protoView);
return this;
}
/** Records field `name`'s transform, or its deletion when `transform` is null. */
#operation(name: String, transform: ProtoView | null): void {
if (this.#map.has(name)) {
this.#map.delete(name);
}
let earlier = this.#operations;
let operations = earlier ?? new FieldOperations();
operations.add(name, transform);
this.#operations = operations;
}
}
/**
* The transforms and deletions of a `Fields`. An interface, so that the
* code behind it is only in programs that make a `FieldOperations`.
*/
interface WriteOperations {
/** Records field `name`'s transform, or its deletion when `transform` is null. */
add(name: String, transform: ProtoView | null): void;
/**
* Adds the deleted fields' quoted paths to `paths`, and answers the
* transforms as the `update_transforms` of an encoded `Write`; empty
* when there are none.
*/
encode(paths: GrowableArray<String>): ProtoView;
}
/** The transformed and deleted fields, in the order given; transforms are kept encoded. */
final class FieldOperations implements WriteOperations {
var #list: GrowableArray<FieldOperation> = new GrowableArray<FieldOperation>();
add(name: String, transform: ProtoView | null): void {
let kept = new GrowableArray<FieldOperation>();
for (let operation in this.#list) {
if (operation.name != name) {
kept.push(operation);
}
}
kept.push(new FieldOperation(name, transform));
this.#list = kept;
}
encode(paths: GrowableArray<String>): ProtoView {
let transforms = new GrowableArray<DocumentTransformFieldTransform>();
for (let operation in this.#list) {
let encoded = operation.transform;
if (encoded == null) {
paths.push(quoteFieldName(operation.name));
} else {
transforms.push(new DocumentTransformFieldTransform(encoded));
}
}
return if (transforms.length == 0) ProtoView.empty() else Write.of({updateTransforms: transforms}).protoView;
}
}
/** A field's transform, encoded, or its deletion when `transform` is null. */
final class FieldOperation {
name: String;
transform: ProtoView | null;
new(this.name, this.transform);
}
/**
* `name` as a Firestore field path segment: unchanged when it is a plain
* identifier, otherwise in backquotes with backquotes and backslashes
* escaped.
*/
export function quoteFieldName(name: String): String {
var plain = name.length > 0;
for (var i = 0; i < name.length; i += 1) {
let b = name.getByteAt(i);
let letter = (b >= 0x41 && b <= 0x5A) || (b >= 0x61 && b <= 0x7A) || b == 0x5F;
let digit = b >= 0x30 && b <= 0x39;
if (!letter && !(digit && i > 0)) {
plain = false;
}
}
if (plain) {
return name;
}
var out = '`';
for (var i = 0; i < name.length; i += 1) {
let c = name.sliceBytes(i, i + 1);
out = if (c == '`' || c == '\\') `${out}\\${c}` else `${out}${c}`;
}
return `${out}\``;
}
/** Options for `new Firestore`. */
export type FirestoreOptions = {
projectId: String,
/** `(default)` when absent. */
databaseId?: String,
/**
* How calls reach the API: a `GrpcTransport` or a `FallbackTransport`
* to `FIRESTORE_ORIGIN` with credentials, or a test's own. Only the
* transport a program names is compiled into it.
*/
transport: ProtoTransport,
};
/** Options for `Firestore.connect` and `Firestore.connectFallback`. */
export type ConnectOptions = {
/** `(default)` when absent. */
databaseId?: String,
/**
* For `connect`: each call's deadline after it starts
* (`GrpcTransport.setTimeout`); none when absent.
*/
timeout?: Duration,
};
function databaseIdOf(options: FirestoreOptions) {
if (let {databaseId} = options) {
return databaseId;
}
return '(default)';
}
function connectDatabaseId(options: ConnectOptions) {
if (let {databaseId} = options) {
return databaseId;
}
return '(default)';
}
/** The value of environment variable `name`, or null when it is unset or empty. */
function envValue(name: String): String | null {
let value = getEnv(name);
return if (value != null && value.length > 0) value else null;
}
/** Whether `e` is the server aborting a transaction because of contention. */
function isAborted(e: Error) {
return e is RpcError && e.code == CODE_ABORTED;
}
/**
* How long to wait before attempt `attempt + 1` of a transaction: a random
* time up to 50 ms doubled for each attempt so far, so transactions that
* contended do not meet again at once.
*/
function retryDelay(attempt: i32) {
// The doubling stops at 50 ms << 24, about nine days, where the next
// one would overflow an i32.
let doublings = if (attempt > 25) 24 else attempt - 1;
return milliseconds(randomInt(50 << doublings) as i64);
}
/** One Firestore database. */
export final class Firestore {
projectId: String;
databaseId: String;
/** The generated client every call goes through. */
client: FirestoreClient;
new(options: FirestoreOptions)
: projectId = options.projectId,
databaseId = databaseIdOf(options),
client = new FirestoreClient(options.transport);
/**
* A database of the project this program runs in, over gRPC, with the
* default credentials (see `defaultProjectId`, `defaultCredentials`).
* gRPC needs a host whose outgoing HTTP speaks HTTP/2, such as
* `zena` or `zena-run`; on other hosts, use `connectFallback`.
*
* When `FIRESTORE_EMULATOR_HOST` is set to the emulator's `host:port`,
* the database is the emulator's instead, reached over cleartext HTTP/2
* with the token `owner`, which bypasses security rules. The project id
* is then `GOOGLE_CLOUD_PROJECT` or `GCLOUD_PROJECT`, or `demo-zena`
* when neither is set.
*/
static async connect(options: ConnectOptions = {}): Future<Firestore> {
let databaseId = connectDatabaseId(options);
let emulator = envValue('FIRESTORE_EMULATOR_HOST');
if (emulator != null) {
var projectId = EMULATOR_PROJECT;
for (let name in ['GOOGLE_CLOUD_PROJECT', 'GCLOUD_PROJECT']) {
let value = envValue(name);
if (value != null) {
projectId = value;
break;
}
}
let transport = new GrpcTransport(`http://${emulator}`, new AccessTokenCredentials('owner'));
if (let {timeout} = options) {
transport.setTimeout(timeout);
}
return new Firestore({projectId, databaseId, transport});
}
let projectId = await defaultProjectId();
let transport = new GrpcTransport(FIRESTORE_ORIGIN, defaultCredentials());
if (let {timeout} = options) {
transport.setTimeout(timeout);
}
return new Firestore({projectId, databaseId, transport});
}
/**
* As `connect`, over Google's gRPC fallback protocol, which works on any
* `wasi:http` host, stock `wasmtime serve` among them. The emulator does
* not speak it, so this throws when `FIRESTORE_EMULATOR_HOST` is set.
*/
static async connectFallback(options: ConnectOptions = {}): Future<Firestore> {
if (envValue('FIRESTORE_EMULATOR_HOST') != null) {
throw new Error('FIRESTORE_EMULATOR_HOST is set, and the emulator does not speak the fallback protocol; use Firestore.connect()');
}
let projectId = await defaultProjectId();
let transport = new FallbackTransport(FIRESTORE_ORIGIN, defaultCredentials());
return new Firestore({projectId, databaseId: connectDatabaseId(options), transport});
}
/** `projects/<project>/databases/<database>`. */
database: String {
get { return `projects/${this.projectId}/databases/${this.databaseId}`; }
}
/** The prefix of every document name: `<database>/documents`. */
documentsRoot: String {
get { return `${this.database}/documents`; }
}
/** The collection at `path`, such as `posts` or `posts/hello/comments`. */
collection(path: String): CollectionReference {
return new CollectionReference(this, path);
}
/** The document at `path`, such as `posts/hello`. */
doc(path: String): DocumentReference {
return new DocumentReference(this, path);
}
/**
* Reads `documents` in one call, and answers their snapshots in the same
* order. A document that does not exist has a snapshot whose `exists`
* is false.
*/
async getAll(documents: Array<DocumentReference>): Future<GrowableArray<DocumentSnapshot>> {
return await readAll(this, documents, null);
}
/** A batch of writes, committed together. */
batch(): WriteBatch {
return new WriteBatch(this);
}
/**
* Runs `body` in a read-write transaction and commits its writes. When
* the server aborts the transaction because of contention, in a read
* in `body` or in the commit, the whole attempt runs again in a new
* transaction that names the old one, after a short random wait, up to
* `attempts` attempts in all. When `body` throws anything else, the
* transaction is rolled back and the error rethrown.
*/
async runTransaction<T>(body: (transaction: Transaction) => Future<T>, attempts: i32 = 5): Future<T> {
var retry: ProtoBytes | null = null;
var attempt = 1;
while (true) {
let readWrite = if (retry == null) TransactionOptionsReadWrite.of({})
else TransactionOptionsReadWrite.of({retryTransaction: retry});
let begun = await this.client.beginTransaction({
database: this.database,
options: TransactionOptions.of({readWrite}),
});
let transaction = new Transaction(this, begun.transaction);
var result: Option<T> = none;
var aborted = false;
try {
result = some(await body(transaction));
} catch (e) {
aborted = true;
await this.#rollback(transaction.id);
if (!isAborted(e) || attempt >= attempts) {
throw e;
}
}
if (!aborted) {
try {
await this.commit(transaction.writes, transaction.id);
return (result as Some<T>).value;
} catch (e) {
if (!isAborted(e) || attempt >= attempts) {
throw e;
}
}
}
await sleep(retryDelay(attempt));
retry = transaction.id;
attempt += 1;
}
throw new Error('unreachable');
}
/** Commits `writes`, in `transaction` when it is not null. */
async commit(writes: Array<Write>, transaction: ProtoBytes | null = null): Future<void> {
if (transaction == null) {
await this.client.commit({database: this.database, writes});
} else {
await this.client.commit({database: this.database, writes, transaction: transaction});
}
}
async #rollback(transaction: ProtoBytes): Future<void> {
try {
await this.client.rollback({database: this.database, transaction});
} catch (e) {
// The original error matters more than a failed rollback; the
// transaction expires on the server either way.
}
}
}
/** A document, which may not exist yet. */
export final class DocumentReference {
#db: Firestore;
/** The path below the database's documents: `posts/hello`. */
path: String;
new(db: Firestore, path: String) : #db = db, path = path;
/** The document's id, the last segment of its path. */
id: String {
get { return lastSegment(this.path); }
}
/** The full resource name: `projects/p/databases/d/documents/posts/hello`. */
name: String {
get { return `${this.#db.documentsRoot}/${this.path}`; }
}
/** The collection this document is in. */
parent: CollectionReference {
get { return new CollectionReference(this.#db, parentPath(this.path)); }
}
/** The subcollection `id` of this document. */
collection(id: String): CollectionReference {
return new CollectionReference(this.#db, `${this.path}/${id}`);
}
/** Reads the document. A snapshot of a document that does not exist has `exists` false. */
async get(): Future<DocumentSnapshot> {
try {
let document = await this.#db.client.getDocument({name: this.name});
return new DocumentSnapshot(this, document);
} catch (e) {
if (e is RpcError && e.code == CODE_NOT_FOUND) {
return new DocumentSnapshot(this, null);
}
throw e;
}
}
/** Replaces the document with `fields`, creating it if it does not exist. */
async set(fields: Fields): Future<void> {
await this.#db.commit([setWrite(this, fields)]);
}
/** Creates the document; fails with ALREADY_EXISTS if it exists. */
async create(fields: Fields): Future<void> {
await this.#db.commit([createWrite(this, fields)]);
}
/**
* Sets the top-level fields in `fields`, leaving the others; fails with
* NOT_FOUND if the document does not exist.
*/
async update(fields: Fields): Future<void> {
await this.#db.commit([updateWrite(this, fields)]);
}
/** Deletes the document. Deleting a document that does not exist succeeds. */
async delete(): Future<void> {
await this.#db.commit([deleteWrite(this)]);
}
}
/** A collection of documents. Its query methods start a `Query` over it. */
export final class CollectionReference {
#db: Firestore;
/** The path below the database's documents: `posts`, `posts/hello/comments`. */
path: String;
new(db: Firestore, path: String) : #db = db, path = path;
/** The collection's id, the last segment of its path. */
id: String {
get { return lastSegment(this.path); }
}
/** The document `id` in this collection. */
doc(id: String): DocumentReference {
return new DocumentReference(this.#db, `${this.path}/${id}`);
}
/** Creates a document with a new random id and `fields`, and returns it. */
async add(fields: Fields): Future<DocumentReference> {
let document = this.doc(randomId());
await document.create(fields);
return document;
}
/** A query over every document in this collection. */
query(): Query {
return new Query(this.#db, parentPath(this.path), this.id);
}
where(field: String, op: StructuredQueryFieldFilterOperator, value: Value): Query {
return this.query().where(field, op, value);
}
orderBy(field: String, direction: StructuredQueryDirection = StructuredQueryDirection.Ascending): Query {
return this.query().orderBy(field, direction);
}
limit(count: i32): Query {
return this.query().limit(count);
}
/** Every document in this collection, each as soon as it arrives. */
stream(): SnapshotStream {
return this.query().stream();
}
/** Every document in this collection. */
async get(): Future<GrowableArray<DocumentSnapshot>> {
return await this.query().get();
}
/** How many documents this collection holds. */
async count(): Future<i64> {
return await this.query().count();
}
}
/**
* Where a query's results start or end: values for its orderings, in
* order, or a document, whose values for the orderings are used.
*/
export type QueryPosition = Array<Value> | DocumentSnapshot;
/**
* One optional part of a query, encoded given the field paths the query
* is ordered by. The method that sets a part makes it, and a program
* reaches a part's code only through this interface, so the code to
* encode a part is only in programs that use it.
*/
interface QueryPart {
encode(paths: GrowableArray<String>): ProtoView;
}
/** The query's filters, joined by AND when there are several. */
final class WherePart implements QueryPart {
#filters: GrowableArray<StructuredQueryFilter>;
new(filters: GrowableArray<StructuredQueryFilter>) : #filters = filters;
encode(paths: GrowableArray<String>): ProtoView {
if (this.#filters.length == 1) {
return StructuredQuery.of({where: this.#filters[0]}).protoView;
}
return StructuredQuery.of({
where: StructuredQueryFilter.of({
compositeFilter: StructuredQueryCompositeFilter.of({
op: StructuredQueryCompositeFilterOperator.And,
filters: this.#filters,
}),
}),
}).protoView;
}
}
/** The fields to answer; none answers names only. */
final class SelectPart implements QueryPart {
#fields: Array<String>;
new(fields: Array<String>) : #fields = fields;
encode(paths: GrowableArray<String>): ProtoView {
let refs = new GrowableArray<StructuredQueryFieldReference>();
for (let field in this.#fields) {
refs.push(fieldReference(field));
}
if (refs.length == 0) {
refs.push(fieldReference(NAME_FIELD));
}
return StructuredQuery.of({select: StructuredQueryProjection.of({fields: refs})}).protoView;
}
}
/** A start (`isStart`) or end at `position`, just before it when `before`. */
final class BoundPart implements QueryPart {
#position: QueryPosition;
#before: boolean;
#isStart: boolean;
new(position: QueryPosition, before: boolean, isStart: boolean)
: #position = position, #before = before, #isStart = isStart;
encode(paths: GrowableArray<String>): ProtoView {
let cursor = cursorOf(this.#position, this.#before, paths);
return if (this.#isStart) StructuredQuery.of({startAt: cursor}).protoView
else StructuredQuery.of({endAt: cursor}).protoView;
}
}
/**
* A query over one collection. Each refining method returns a new query
* and leaves this one as it was.
*/
export final class Query {
#db: Firestore;
/** The path of the document the collection is under, or '' for a root collection. */
#parent: String;
#collectionId: String;
#filters: GrowableArray<StructuredQueryFilter> = new GrowableArray<StructuredQueryFilter>();
var #where: QueryPart | null = null;
/** At most this many results, or -1 for no limit. */
var #limit: i32 = -1;
/** The field paths ordered by, as written, and their directions. */
#orderPaths: GrowableArray<String> = new GrowableArray<String>();
#directions: GrowableArray<StructuredQueryDirection> = new GrowableArray<StructuredQueryDirection>();
var #offset: i32 = 0;
var #select: QueryPart | null = null;
var #start: QueryPart | null = null;
var #end: QueryPart | null = null;
/** Whether a cursor is a document, which orders the results by name too. */
var #documentCursor: boolean = false;
new(db: Firestore, parent: String, collectionId: String)
: #db = db, #parent = parent, #collectionId = collectionId;
/** The documents whose field `field` compares with `value` by `op`. */
where(field: String, op: StructuredQueryFieldFilterOperator, value: Value): Query {
let next = this.#copy();
next.#filters.push(StructuredQueryFilter.of({
fieldFilter: StructuredQueryFieldFilter.of({field: fieldReference(field), op, value}),
}));
next.#where = new WherePart(next.#filters);
return next;
}
/** Orders the results by `field`; later calls order ties. */
orderBy(field: String, direction: StructuredQueryDirection = StructuredQueryDirection.Ascending): Query {
let next = this.#copy();
next.#orderPaths.push(field);
next.#directions.push(direction);
return next;
}
/** At most `count` results. */
limit(count: i32): Query {
let next = this.#copy();
next.#limit = count;
return next;
}
/** Skips the first `count` results. */
offset(count: i32): Query {
let next = this.#copy();
next.#offset = count;
return next;
}
/** Only `fields` of each result, and its name; `select([])` answers names only. */
select(fields: Array<String>): Query {
let next = this.#copy();
next.#select = new SelectPart(fields);
return next;
}
/**
* The results from `position` on. With a document, the query must be
* ordered (`orderBy`) before the cursor is used, and results tied on
* every ordering are ordered by name.
*/
startAt(position: QueryPosition): Query {
return this.#bound(position, true, true);
}
/** The results after `position`; see `startAt`. For paging, pass the last snapshot of the page before. */
startAfter(position: QueryPosition): Query {
return this.#bound(position, false, true);
}
/** The results up to and including `position`; see `startAt`. */
endAt(position: QueryPosition): Query {
return this.#bound(position, false, false);
}
/** The results before `position`; see `startAt`. */
endBefore(position: QueryPosition): Query {
return this.#bound(position, true, false);
}
/** This query with a start (`isStart`) or end at `position`, just before it when `before`. */
#bound(position: QueryPosition, before: boolean, isStart: boolean): Query {
let next = this.#copy();
let part = new BoundPart(position, before, isStart);
if (isStart) {
next.#start = part;
} else {
next.#end = part;
}
if (position is DocumentSnapshot) {
next.#documentCursor = true;
}
return next;
}
/** The query as the API takes it. */
structuredQuery(): StructuredQuery {
// A document cursor gives a value for each ordering and then the
// document's name, so the query also orders by name, as the last
// ordering does, unless it already ends with that.
let paths = new GrowableArray<String>();
let directions = new GrowableArray<StructuredQueryDirection>();
for (var i = 0; i < this.#orderPaths.length; i += 1) {
paths.push(this.#orderPaths[i]);
directions.push(this.#directions[i]);
}
if (this.#documentCursor && (paths.length == 0 || paths[paths.length - 1] != NAME_FIELD)) {
let last = if (directions.length == 0) StructuredQueryDirection.Ascending else directions[directions.length - 1];
paths.push(NAME_FIELD);
directions.push(last);
}
let orders = new GrowableArray<StructuredQueryOrder>();
for (var i = 0; i < paths.length; i += 1) {
orders.push(StructuredQueryOrder.of({field: fieldReference(paths[i]), direction: directions[i]}));
}
let from = [StructuredQueryCollectionSelector.of({collectionId: this.#collectionId})];
// Each optional part is encoded on its own and the parts merged, so
// an absent part is left out rather than set to a default.
let parts = new GrowableArray<ProtoView>();
parts.push(StructuredQuery.of({from, orderBy: orders, offset: this.#offset}).protoView);
if (this.#limit >= 0) {
parts.push(StructuredQuery.of({limit: Int32Value.of({value: this.#limit})}).protoView);
}
for (let part in [this.#where, this.#select, this.#start, this.#end]) {
if (part != null) {
parts.push(part.encode(paths));
}
}
return new StructuredQuery(ProtoView.merge(parts));
}
/** The matching documents, each as soon as it arrives. */
stream(transaction: ProtoBytes | null = null): SnapshotStream {
let parent = this.#parentName();
let responses = if (transaction == null)
this.#db.client.runQuery({parent, structuredQuery: this.structuredQuery()})
else this.#db.client.runQuery({parent, structuredQuery: this.structuredQuery(), transaction: transaction});
return new SnapshotStream(this.#db, responses);
}
/** The matching documents. */
async get(): Future<GrowableArray<DocumentSnapshot>> {
return await collect(this.stream());
}
/** How many documents match, counted by the server without sending them. */
async count(): Future<i64> {
let aggregation = StructuredAggregationQueryAggregation.of({
alias: COUNT_ALIAS,
count: StructuredAggregationQueryAggregationCount.of({}),
});
let responses = this.#db.client.runAggregationQuery({
parent: this.#parentName(),
structuredAggregationQuery: StructuredAggregationQuery.of({
structuredQuery: this.structuredQuery(),
aggregations: [aggregation],
}),
});
var count: i64 = 0;
for await (let response in responses) {
let result = response.result;
if (result != null) {
let fields = result.aggregateFields;
if (fields.has(COUNT_ALIAS)) {
count = fields.getOr(COUNT_ALIAS, integerValue(0)).integerValue;
}
}
}
return count;
}
#parentName(): String {
return if (this.#parent.length == 0) this.#db.documentsRoot
else `${this.#db.documentsRoot}/${this.#parent}`;
}
#copy(): Query {
let next = new Query(this.#db, this.#parent, this.#collectionId);
for (let f in this.#filters) {
next.#filters.push(f);
}
if (this.#where != null) {
next.#where = new WherePart(next.#filters);
}
for (var i = 0; i < this.#orderPaths.length; i += 1) {
next.#orderPaths.push(this.#orderPaths[i]);
next.#directions.push(this.#directions[i]);
}
next.#limit = this.#limit;
next.#offset = this.#offset;
next.#select = this.#select;
next.#start = this.#start;
next.#end = this.#end;
next.#documentCursor = this.#documentCursor;
return next;
}
}
/**
* The API's cursor at `position`, just before it when `before`, in a
* query ordered by `paths`. A document's values are its fields at those
* paths, and its name for `__name__`.
*/
function cursorOf(position: QueryPosition, before: boolean, paths: GrowableArray<String>): Cursor {
if (position is DocumentSnapshot) {
let snapshot = position;
if (!snapshot.exists) {
throw new Error(`a query cursor's document does not exist: ${snapshot.ref.path}`);
}
let values = new GrowableArray<Value>();
for (let path in paths) {
if (path == NAME_FIELD) {
values.push(Value.of({referenceValue: snapshot.ref.name}));
} else {
let value = snapshot.get(path);
if (value == null) {
throw new Error(`a query cursor's document has no field ${path}, which the query orders by`);
}
values.push(value);
}
}
return Cursor.of({values, before});
}
return Cursor.of({values: position, before});
}
/** The documents a query returns, skipping the responses that carry none. */
export final class SnapshotStream implements AsyncIterator<DocumentSnapshot>, AsyncIterable<DocumentSnapshot> {
#db: Firestore;
#responses: AsyncIterator<RunQueryResponse>;
new(db: Firestore, responses: AsyncIterator<RunQueryResponse>) : #db = db, #responses = responses;
next(): AsyncStep<DocumentSnapshot> {
while (true) {
match (this.#responses.next()) {
case (1, let response, _): {
let document = response.document;
if (document != null) {
return (1, this.#snapshot(document), _);
}
}
case (2, _, let pending): {
return (2, _, this.#later(pending));
}
case _: {
return (0, _, _);
}
}
}
return (0, _, _);
}
[AsyncIterable.asyncIterator](): AsyncIterator<DocumentSnapshot> {
return this;
}
/** The next document, from `pending` or the responses after it. */
async #later(pending: Future<Option<RunQueryResponse>>): Future<Option<DocumentSnapshot>> {
var next = await pending;
while (true) {
if (next is Some<RunQueryResponse>) {
let document = next.value.document;
if (document != null) {
return some(this.#snapshot(document));
}
} else {
return none;
}
next = await nextResponse(this.#responses);
}
return none;
}
#snapshot(document: Document): DocumentSnapshot {
let name = document.name;
let root = `${this.#db.documentsRoot}/`;
let path = if (name.startsWith(root)) name.sliceBytes(root.length, name.length) else name;
return new DocumentSnapshot(new DocumentReference(this.#db, path), document);
}
}
/** The next element of `responses`, waiting for it if it is still coming. */
async function nextResponse(responses: AsyncIterator<RunQueryResponse>): Future<Option<RunQueryResponse>> {
match (responses.next()) {
case (1, let response, _): {
return some(response);
}
case (2, _, let pending): {
return await pending;
}
case _: {
return none;
}
}
}
async function collect(stream: SnapshotStream): Future<GrowableArray<DocumentSnapshot>> {
let out = new GrowableArray<DocumentSnapshot>();
for await (let snapshot in stream) {
out.push(snapshot);
}
return out;
}
/** A document as it was read. */
export final class DocumentSnapshot {
ref: DocumentReference;
/** The document as the API returned it, or null when it does not exist. */
document: Document | null;
new(this.ref, this.document);
exists: boolean {
get { return this.document != null; }
}
id: String {
get { return this.ref.id; }
}
/** When the document was last written, or null when it does not exist. */
updateTime: Timestamp | null {
get {
let document = this.document;
return document?.updateTime;
}
}
/**
* The value at `path`: a field name, or names joined with `.` to reach
* into map values. Null when there is no such field.
*/
get(path: String): Value | null {
let document = this.document;
if (document == null) {
return null;
}
let names = path.split('.');
var fields = document.fields;
for (var i = 0; i < names.length; i += 1) {
if (!fields.has(names[i])) {
return null;
}
let value = fields.getOr(names[i], nullValue());
if (i == names.length - 1) {
return value;
}
let map = value.mapValue;
if (map == null) {
return null;
}
fields = map.fields;
}
return null;
}
/** The string at `path`: `(true, value)`, or `(false, _)` when it is absent or not a string. */
getString(path: String): inline (true, String) | inline (false, _) {
let value = this.get(path);
if (value != null && value.hasStringValue) {
return (true, value.stringValue);
}
return (false, _);
}
/** The integer at `path`: `(true, value)`, or `(false, _)` when it is absent or not an integer. */
getInteger(path: String): inline (true, i64) | inline (false, _) {
let value = this.get(path);
if (value != null && value.hasIntegerValue) {
return (true, value.integerValue);
}
return (false, _);
}
/** The double at `path`: `(true, value)`, or `(false, _)` when it is absent or not a double. */
getDouble(path: String): inline (true, f64) | inline (false, _) {
let value = this.get(path);
if (value != null && value.hasDoubleValue) {
return (true, value.doubleValue);
}
return (false, _);
}
/** The boolean at `path`: `(true, value)`, or `(false, _)` when it is absent or not a boolean. */
getBoolean(path: String): inline (true, boolean) | inline (false, _) {
let value = this.get(path);
if (value != null && value.hasBooleanValue) {
return (true, value.booleanValue);
}
return (false, _);
}
/** The timestamp at `path`, or null when it is absent or not a timestamp. */
getTimestamp(path: String): Timestamp | null {
let value = this.get(path);
return value?.timestampValue;
}
}
/** Writes collected and committed together. */
export final class WriteBatch {
#db: Firestore;
#writes: GrowableArray<Write> = new GrowableArray<Write>();
new(db: Firestore) : #db = db;
set(document: DocumentReference, fields: Fields): WriteBatch {
this.#writes.push(setWrite(document, fields));
return this;
}
create(document: DocumentReference, fields: Fields): WriteBatch {
this.#writes.push(createWrite(document, fields));
return this;
}
update(document: DocumentReference, fields: Fields): WriteBatch {
this.#writes.push(updateWrite(document, fields));
return this;
}
delete(document: DocumentReference): WriteBatch {
this.#writes.push(deleteWrite(document));
return this;
}
/** Commits every write, all or none. */
async commit(): Future<void> {
await this.#db.commit(this.#writes);
}
}
/**
* A read-write transaction, as `Firestore.runTransaction` hands it to its
* body. Reads go to the server at once; writes are kept and committed when
* the body returns.
*/
export final class Transaction {
#db: Firestore;
/** The transaction's id. */
id: ProtoBytes;
/** The writes to commit. */
writes: GrowableArray<Write> = new GrowableArray<Write>();
new(db: Firestore, id: ProtoBytes) : #db = db, id = id;
/** Reads `document` in this transaction. */
async get(document: DocumentReference): Future<DocumentSnapshot> {
try {
let found = await this.#db.client.getDocument({name: document.name, transaction: this.id});
return new DocumentSnapshot(document, found);
} catch (e) {
if (e is RpcError && e.code == CODE_NOT_FOUND) {
return new DocumentSnapshot(document, null);
}
throw e;
}
}
/** Reads `documents` in this transaction, in one call; see `Firestore.getAll`. */
async getAll(documents: Array<DocumentReference>): Future<GrowableArray<DocumentSnapshot>> {
return await readAll(this.#db, documents, this.id);
}
/** Runs `query` in this transaction. */
async query(query: Query): Future<GrowableArray<DocumentSnapshot>> {
return await collect(query.stream(this.id));
}
set(document: DocumentReference, fields: Fields): Transaction {
this.writes.push(setWrite(document, fields));
return this;
}
create(document: DocumentReference, fields: Fields): Transaction {
this.writes.push(createWrite(document, fields));
return this;
}
update(document: DocumentReference, fields: Fields): Transaction {
this.writes.push(updateWrite(document, fields));
return this;
}
delete(document: DocumentReference): Transaction {
this.writes.push(deleteWrite(document));
return this;
}
}
function documentOf(ref: DocumentReference, fields: Fields) {
return Document.of({name: ref.name, fields: fields.toMap()});
}
/** `write` with the transforms in `fields` merged in. */
function withTransforms(write: Write, fields: Fields) {
if (!fields.hasOperations) {
return write;
}
// Deletions do not apply outside an update, so their paths are dropped.
let part = fields.operationsPart(new GrowableArray<String>());
return new Write(ProtoView.merge([write.protoView, part]));
}
function setWrite(ref: DocumentReference, fields: Fields): Write {
return withTransforms(Write.of({update: documentOf(ref, fields)}), fields);
}
function createWrite(ref: DocumentReference, fields: Fields): Write {
return withTransforms(Write.of({
update: documentOf(ref, fields),
currentDocument: Precondition.of({exists: false}),
}), fields);
}
/**
* An update of the fields with values and the deleted fields, which the
* mask names, then the transforms, which it does not: the server applies
* them after the update.
*/
function updateWrite(ref: DocumentReference, fields: Fields): Write {
let paths = new GrowableArray<String>();
for (let name in fields.names()) {
paths.push(quoteFieldName(name));
}
let transforms = fields.operationsPart(paths);
let write = Write.of({
update: documentOf(ref, fields),
updateMask: DocumentMask.of({fieldPaths: paths}),
currentDocument: Precondition.of({exists: true}),
});
return if (transforms.byteLength == 0) write else new Write(ProtoView.merge([write.protoView, transforms]));
}
/** Reads `documents` in one call, in `transaction` when it is not null; see `Firestore.getAll`. */
async function readAll(
db: Firestore,
documents: Array<DocumentReference>,
transaction: ProtoBytes | null
): Future<GrowableArray<DocumentSnapshot>> {
let out = new GrowableArray<DocumentSnapshot>();
if (documents.length == 0) {
return out;
}
let names = new GrowableArray<String>();
for (let document in documents) {
names.push(document.name);
}
let responses = if (transaction == null)
db.client.batchGetDocuments({database: db.database, documents: names})
else db.client.batchGetDocuments({database: db.database, documents: names, transaction: transaction});
// The responses come in any order.
let found = new HashMap<String, Document>();
for await (let response in responses) {
let document = response.found;
if (document != null) {
found[document.name] = document;
}
}
for (let document in documents) {
if (let (true, read) = found.get(document.name)) {
out.push(new DocumentSnapshot(document, read));
} else {
out.push(new DocumentSnapshot(document, null));
}
}
return out;
}
function deleteWrite(ref: DocumentReference): Write {
return Write.of({delete: ref.name});
}
/** A reference to the field at `path`, names joined with `.`, each quoted as needed. */
function fieldReference(path: String): StructuredQueryFieldReference {
var quoted = '';
for (let name in path.split('.')) {
quoted = if (quoted.length == 0) quoteFieldName(name) else `${quoted}.${quoteFieldName(name)}`;
}
return StructuredQueryFieldReference.of({fieldPath: quoted});
}
function lastSegment(path: String): String {
let segments = path.split('/');
return segments[segments.length - 1];
}
/** `path` without its last segment, or '' when it has one segment. */
function parentPath(path: String): String {
var i = path.length - 1;
while (i >= 0) {
if (path.getByteAt(i) == 0x2F) {
return path.sliceBytes(0, i);
}
i -= 1;
}
return '';
}
/** A random 20-character document id, as Firestore's own clients make. */
function randomId(): String {
let alphabet = 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789';
var id = '';
for (var i = 0; i < 20; i += 1) {
let n = randomInt(62);
id = `${id}${alphabet.sliceBytes(n, n + 1)}`;
}
return id;
}
- protobuf protobuf.zena-lang.pkg.zena-lang.dev ~0.0.1-preview.1
| Version | Published | Dependencies | Approved by |
|---|---|---|---|
| 0.0.1-preview.1 | 2026-11-19 |
|
|