Meta-programming
The meta keyword marks what runs at compile time. At that point, types, declarations and code are values that are read, transformed and built with the ordinary language.
5.1 Constructs
Section titled “5.1 Constructs”| Form | Role |
|---|---|
meta val X = expr |
Constant computed at compile time: an integer, a floating-point number, a boolean, a character, a string, the variant of an enum without payload, or an array of integers of any size or of characters, stored in the program’s data; meta val K: UInt32[] = [0x428a2f98, …] states the type of its elements. If it is an integer, it can give the size of a fixed array, UInt8[X]. Inside a function, it is a constant of its block |
meta fn f(...) |
Function executable only at compile time |
meta if / meta for |
Conditional or unrolled code, in the body of a function; meta if also selects declarations of the module |
fn f(meta n: Int) |
Parameter known at compile time, of a function or a method: f is specialized by value |
quote { ... } |
Code fragment: an expression (Expr, or Expr<T> when its type T is known), declarations (Decls), statements (Stmt) |
${ ... } in a quote |
Splice of a meta value |
@f(args) on a declaration |
Call of the meta fn f |
#C(args) on a type |
Adoption of contract C, implemented by its generator (section 4) |
`ns(options)` after a field’s type |
Inert tag, read by meta fns (5.6) |
The condition of a meta if, the sequence of a meta for, the argument of a meta parameter are meta code: they see the block’s constants, meta val and meta parameters, and no runtime variable. The discarded branch of a meta if is not checked: it may refer to what exists only on another target. A meta for iterates over a range of integers (0..n, step included) or an array of constants, and checks one copy of its body per value, in which the variable is a constant; break and continue do not leave it, since there is no loop anymore once it is unrolled, but with a label they target an enclosing runtime loop, continue 'lines. A function with meta parameters is a template: each set of values gives a specialization of it, checked and lowered separately, whose signature may refer to its parameters, fn zeros(meta n: Int): UInt8[n]. A method can have them too, v.at(-1), generic or not.
Among the declarations of a module, meta if keeps those of one branch: they are the module’s declarations, imports included, and those of the other are not checked. It is decided before imports. else meta if chains conditions, at module level as in a function. The condition is a Bool, which often reads the build target: Build.target.arch (.x86_64, .arm64), Build.target.os (.linux, .macos, .windows) and Build.profile (.dev with weaver build -dev, .release otherwise), variants of the enums Arch, Os and Profile of core, which only meta code reads. The target is the machine that compiles, or the one that -target names: weaver build -target arm64-linux compiles for it, which asks clang for its headers and C libraries, and weaver expand -target x86_64-windows shows the code it keeps.
meta if Build.target.os == .macos && Build.target.arch == .arm64 { val pageSize = 16384} else meta if Build.target.os == .windows { val pageSize = 65536 // the allocation granularity} else { val pageSize = 4096}fn pow(x: Int, meta n: Int): Int { var r = 1 meta for _ in 0..n { r *= x } // pow(x, 3) is x * x * x, without a loop return r}In a quote, a splice alone on its line splices statements (Stmt, or an array of Stmt or of Expr), or declarations in a quote of declarations; where statements are expected, a Stmt result or an element of a Stmt[], a quote made of such splices alone, of one expression, or of nothing, quote { }, is statements; elsewhere, statements stand for their block. A for that ends a splice yields the array of the values of its body: ${ for f in t.fields { quote { … } } } splices one quotation per field. ${name} in place of the name of a binding, a pattern, a function or a parameter, val ${name} = …, for ${v} in …, fn ${name}(${arg}: Int), names what the generated code binds or declares, a type, an enum or a contract among them, type ${name} = …, enum ${name} = …, contract ${name} { … }, and after a dot the member read, self.${f.name}; ident(name) quotes a name as an expression, where ${name} would give its string. Among the members of a quoted impl, a splice adds functions, an FnDecl or Decls that hold nothing else, next to those written there: impl ${t} { ${getters(t)} fn describe(self): String = … }; among those of a quoted contract, its methods, without a body: contract ${name} { ${methods} }.
5.2 Types are values
Section titled “5.2 Types are values”The type Type exposes name, fields, variants, contracts and attributes (the names of the contracts it adopts, already or through a #C awaiting its generator, and those of its @X annotations), and constructors: Option(t), Array(t), Record([Field("x", Int), …]), and Named("Expr"), a type by its name, for one the generated code is only declaring. In meta code, a type name is a value, Counts.fields. What TypeScript needs a type sub-language for is written here with ordinary functions:
meta fn Partial(t: Type): Type = t.mapFields(f => f.withType(Option(f.type)))
meta fn Pick(t: Type, names: String[]): Type = t.filterFields(f => names.contains(f.name))
type UserPatch = Partial(User)type UserSummary = Pick(User, ["id", "name"])A Field exposes name, type, doc, tag(T), withType and withName; a Type also exposes doc, isOption, isArray, isEnum, isParam (a type parameter, the T of a template), args (the element of an array or an optional, the arguments of an instance: Option(Int).args is [Int]) and variants, each Variant of which has name, doc and fields, and v.construct(f => …), the call that constructs it, E.V(a: …, b: …). t.annotations gives its @X(…) annotations with their arguments: each Annotation has name, args, the quoted expressions of its arguments, labels, their labels ("" when unlabeled), and arg(label), the argument with that label; @table("users", schema: "public") has args of two expressions and labels ["", "schema"]. A literal argument is read as a meta value: a.arg("schema").string() is "public", and .int(), .float(), .bool() and .char() read the other literals, -3 included; another expression, 1 + 2, stops compilation. t.matchVariants(subject, (v, x) => …, inPlace: true) writes a match that has one arm per variant, x denoting the value it carries, bound as mut with inPlace: this is how #Reader and #Writer delegate to the active variant of an enum. t.matchCases(subject, (v, i, xs) => …) also has one arm per variant, i its rank and xs its fields, bound in order; t.matchPairs(a, b, (v, xs, ys) => …, otherwise) matches two values at once, one arm when they have the same variant, otherwise for the rest: this is how #Hash and #Eq handle an enum. doc is the /// comment of the declaration: the same text serves the IDE, weaver doc and, through #Cli, the command-line help. t.construct(f => …) writes a record literal, field by field, from a quoted expression for each one: this is how the generator of Default writes default() (section 4).
A computed type is a call of a meta fn: type X = F(…) expects a record from its meta fn, enum X = F(…) an enum, which Enum([…]) and Variant(name, fields) construct. enum ShapeTag = Tags(Shape) thus keeps the names of Shape’s variants, without their payloads. A computed type may have parameters. It is first computed once, its parameters unknown: type Pair<T> = Twice(T), whose meta fn places T without reading it, becomes a generic record like those one writes, which carries its annotations (#Eq #Show), is shared (sync type), and whose literal infers the arguments, Pair{first: 1, second: 2}. A meta fn that reads its parameter, its fields or its name, type Patch<T> = Optionals(T), is computed at each instance, receiving the type that the parameter receives: Patch<User> reads the fields of User. The expected type gives its arguments; in a generic function, Patch<T> is passed as is, without fields for it to read, and each monomorphization computes its instance. Its annotations run on each instance, once its fields are computed: #Eq #Show type Patch<T> = Optionals(T) gives impl Eq for Patch<User>, generated in a view of the file that writes the instance, which names its arguments, and of the template’s, which names the generators; an instance whose fields refuse them is reported where it is written. In a generic function, Patch<T> adopts the contracts of its #C annotations, p == q, "${p}", or a bound P: Eq, each instance calling its own impl; a shared type, computed once, still cannot read T. A shape constraining T says which fields it will have: in fn rename<T: { name: String, .. }>(x: T, p: Patch<T>), the meta fn reads the shape’s fields as those of T, and p.name is a String?, read by name in each instance. Those are only some of the instance’s fields: a literal, a with or a record pattern of Patch<T> is refused there, and each call checks that its instance, Patch<User>, has these fields, of these types.
5.3 Annotations
Section titled “5.3 Annotations”An annotation @f(args) is a call of a meta fn, which receives the annotated declaration and then its arguments. Two signatures are accepted:
meta fn f(t: Type, …): Declsadds declarations next to the annotated type (@builder,@client(Api)): functions andimpls, and types, enums, contracts and constants of the module, which the rest of the module names (below).meta fn f(f: FnDecl, …): FnDecltransforms the annotated function (@route,@trace);(FnDecl, Decls)also adds declarations next to it, the table of a@memoize.
An FnDecl exposes name, params (each Param has name, type and expr, its name as an expression), result and body, its body as an expression; f.withBody(stmts) gives it a body, f.withName(n) another name, and f.forwardTo(n) writes the call of function n with its parameters, self.n(…) for a method. A function annotation also applies to a method, of an impl or of an adoption, and to a generic function, whose parameter types refer to T as a type parameter; the impl that contains it is declared once its methods are transformed, and the functions of the Decls returned for a method join its impl, or, for an adoption method, an impl of the type, the rest going to the module. The returned body evaluates to its last expression, like = { … }. Several annotations apply from the one nearest to fn to the farthest: @a @b fn f is a(b(f)). The written body keeps its positions in the generated code, and its errors are reported there; the rest takes the position of the annotation. An annotation on a generic type receives its template: ${t} is written there as Pair<T>, and impl ${t} { … } declares its parameters.
meta fn trace(f: FnDecl): FnDecl = f.withBody(quote { print("→ ${${f.name}}") ${for p in f.params { quote { print(" ${${p.name}} = ${${p.expr}}") } }} ${f.body}})Declarations an annotation writes. A quote of declarations declares a type from a value the meta code built: type ${name} = ${t} takes the fields of the record t, Record([…]), those of a pub type visible as it is; enum ${name} = ${t} the variants of the enum t, Enum([…]); contract ${name} { ${methods} } its methods. They refer to one another, and to themselves, by Named: Field("args", Array(Named("JsExpr"))), read where the declaration is spliced. record(Named("JsCall"), ["name", "args"], [n, xs]) writes a literal of such a record, JsCall{name: …, args: …}, each field a quoted expression, as call writes a call. The module’s own declarations name what an annotation wrote, in a signature, a field, an impl of a generated contract: a function, a type, a contract or an impl that names a type or a contract neither the module nor its imports declare waits, in a module that has annotations, until they have run, and is declared then, its own annotations running in turn; what names a declaration that waits waits with it. What still names nothing known is reported once every annotation has run, unknown type, as it always was. A meta fn does not wait: the annotations call it. An annotation on a namespace, @grammar(g) namespace Json { }, adds functions to it, in an impl ${t} it writes, without self as the namespace’s own, and declarations beside it.
meta fn nodes(t: Type, names: String[]): Decls { val out: Decls[] = mut [] val variants: Variant[] = mut [] for n in names { val node = "${t.name}${n}" out.push(quote { type ${node} = ${Record([Field("inner", Array(Named("${t.name}Node")))])} }) variants.push(Variant(n, [Field("node", Named(node))])) } val all = "${t.name}Node" return quote { ${out}
enum ${all} = ${Enum(variants)} }}
@nodes(["Call", "Name"])type Js = { text: String }
fn depth(n: JsNode): Int = match n { // JsNode, JsCall and JsName: written by `@nodes` Call(c) => 1 + c.inner.seq().fold(0, (d, x) => d.max(depth(x))) Name(_) => 1}Arguments. An annotation’s arguments are literals and meta expressions, which the meta phase evaluates. A parameter declared lazy, meta fn timeout(f: FnDecl, lazy limit: Expr<Duration>): FnDecl, takes instead an expression the program computes when it runs, @timeout(5.seconds()): the meta fn receives it unevaluated, and may only splice it into the code it generates; reading its value during the expansion is a compile error. The expansion thus never depends on a value of the program, and the phases stay apart: expanding never needs the code it produces. Any annotation may declare such parameters, a test framework outside std as well as weaver.test.
@ acts without promising anything; # (section 4) declares a conformance, which the compiler checks.
#Encode #Eq @buildertype User = { ... }
contract Encode { fn encode(self, w: mut JsonWriter) meta fn derive(t: Type): Decls = quote { fn encode(self, w: mut JsonWriter) = w.object(() => { ${for f in t.fields { quote { w.field(${f.name}, self.${f.name}) } }} }) }}
val u = Json.decode<User>(body)! // generated decoder, zero reflection5.4 Prefixed literals
Section titled “5.4 Prefixed literals”A literal name"..." calls the meta fn name(lit: Template): Expr<T>. It is the only syntactic extension allowed in v1. The Template has text, parts, the pieces of text, and args, the expressions of its interpolations, quoted: they remain the caller’s code, which the meta fn splices without evaluating them. Its text is raw: regex"\d+\.txt" keeps its backslashes. The returned expression takes the place of the literal and is checked there, with type T; it also sees the names of the meta fn’s module. compileError(message) rejects it, at the position of the literal. name""", a block, writes it over several lines, its margin taken off as a plain block’s (section 2); its text is raw as on one line, and its interpolations are the meta fn’s too, sql"""…""" taking its parameters as sql"…" does. A ${ that the meta fn reads as text is written $${. Only b"…" is part of the language: bytes, b"GET /\r\n", where \xff writes an arbitrary one, without interpolation.
val q = sql"SELECT id, name FROM users WHERE id = ${id}" // Query<{ id: Uuid, name: String }>val r = regex"^[a-z]+@[a-z.]+$" // compiled automaton5.5 Guarantees of meta code
Section titled “5.5 Guarantees of meta code”- Typed and hygienic. A
quoteis checked where it is written, as far as its names allow, then as ordinary code where it is spliced. Where it is written, each splice is checked according to its place, an expression, a type, statements or declarations; anExpr<T>takes type T, and an expression made of literals, operators, calls, members, indices, tuples, arrays, records and typed splices gets a type:quote { ${n} * 2 }, n being anExpr<Int>, is anExpr<Int>, as isquote { ${s}.byteLen() + count() }, s being anExpr<String>and count a function of the generator, andif ${e},val x: Int = ${e}or the result of a quoted function expect the type of their place. AnExpr<T>can be passed where anExpris expected; anExprcan be passed where anExpr<T>is, its type being checked at the splice. A name it writes denotes a variable of the scope where it is spliced, if there is one, otherwise what it denotes in the file of the meta fn, its declarations, private ones included, and its imports, and only failing that what the code where it is spliced sees:helper()calls the generator’s function even if the program declares one of the same name. The user’s code that it splices, the argument of a prefixed literal or the body of an annotated function, keeps its names, andident(name)quotes a name of the site. Where the quote is written, what is only found at expansion, a variable, or a function or a method of the site, leaves it without a type; the rest is checked there, and an error that does not depend on the expansion is reported there:quote { twice(${s}) }is rejected if twice expects an Int. An error in generated code is reported at the line of thequotethat wrote it, followed by the annotation that requested it, except for the code that core and the standard library generate, reported at the annotation. Its local names capture nothing: what itsvals, its patterns and its lambdas bind is renamed at each expansion (outbecomesout__1), so that it neither shadows nor captures the code where it is spliced; a pattern name that denotes a variant of the matched type,payload is Null, remains one. The quotes of a single expansion share their names: avalof aquoteof statements can be read in thequotethat splices it, or in those its splices compute. The parameters of a quoted function, its signature, keep theirs. In an enum’simpl,Self.V(…)constructs a variant without naming the type. - Deterministic. No clock, no randomness, no network.
- Declared inputs.
Build.read("api.yaml")reads a file, relative to the source file that requests it, and records it as a dependency of the incremental cache, by the hash of its content;Build.env("LOG_LEVEL")returns an environment variable, aString?, and records it likewise.Build.sources("net")returns a fingerprint of the sources of modulenetand of every module it imports, directly or not, and core’s;Build.sources(""), of the whole program, its own files included: a program names its own version by it, or that of a part several of its programs share, which changes with any of those files, and the module calling it is checked again then, an expansion reading it never kept. The cache keeps the generated declarations, the expressions of prefixed literals, the values ofmeta vals and the fields of computed types;weaver cacheshows it,weaver cache cleanempties it. - Bounded. An execution budget per call, with a clear error when it is exceeded.
- Inspectable. All generated code is visible with
weaver expandand in the IDE;weaver expandalso shows the branch ameta ifkeeps, the iterations of ameta forand the value of ameta val. - A string belongs to the generated code. In a
quote,${…}inside a string is the program’s interpolation, evaluated at run time. A meta value is prepared outside thequoteand spliced as a value: an integer, a floating-point number, a boolean, a character, a string, or an array of these values, which becomes its literal (fn names(): String[] = ${names}):
meta fn Show(t: Type): Decls { val prefix = "${t.name} #" // meta string, computed at compile time return quote { impl Show for ${t} { fn show(self): String = ${prefix} + "${self.id}" // self.id: at run time } }}meta val Api = OpenApi.load(Build.read("api.yaml"))@client(Api) type PetStore // typed methods generated from the schema5.6 Field tags
Section titled “5.6 Field tags”A tag is inert data attached to a field or to an enum variant, placed between backticks at the end of the declaration, as in Go. @ and # act (they call a meta fn), the tag describes: it does nothing as long as no meta fn reads it.
#Encode #Rowtype User = { id: Uuid `json(name: "user_id") sql(column: "id", primaryKey)` email: Email? `json(omitEmpty) sql(unique)`}- One namespace per type. A library declares the shape of its tags with a record annotated
@tag:@tag("json")type JsonTag = { name: String?, omitEmpty: Bool }json(name: "user_id")is a literal ofJsonTag, checked at compile time: an unknown option name or an ill-typed value is an error, and the IDE offers completion. - Options without a value.
omitEmptyis shorthand foromitEmpty: true. An omitted option takes the empty value of its type:None,false,0,""or[]; an option without an empty value must be given. When each one has one, the type adopts Default by itself:JsonTag.default()is the tag without options,json(). - Several namespaces on one element. Tags are separated by spaces; a given namespace may appear only once per field or per variant. When they do not fit on the line, they move to the following lines, one per line, indented under their field:
What documents a field does not go in a tag: thetype User = {id: Uuid`json(name: "user_id")``sql(column: "id", primaryKey, references: "accounts.id")`}
///comment above it, which the meta fn reads throughf.doc, is the help of#Cli. - Reading. In a
meta fn,f.tag(JsonTag)returns the tag of fieldf, orNone;v.tag(ErrorTag)does the same for a variantvoft.variants(example in 8.1). - No orphan tag. A tag that no
meta fnapplied to the type reads triggers a warning, like an unused import.
// The generator of the Encode contract.meta fn derive(t: Type): Decls = quote { fn encode(self, w: mut JsonWriter) = w.object(() => { ${for f in t.fields { val tag = f.tag(JsonTag) ?? JsonTag.default() quote { w.field(${tag.name ?? f.name}, self.${f.name}, omitEmpty: ${tag.omitEmpty}) } }} })}