Compilation and tooling
The compiler is a service: the same front-end serves compilation, the IDE and weaver expand. Meta code runs between type checking and memory analysis, until no new declaration is generated.
flowchart LR
subgraph FE["Front-end shared with the language server (IDE)"]
A["Source<br/>.wft modules"] --> B["Syntax<br/>syntax tree"] --> C["Names and types<br/>checking"]
C -- "types, AST" --> M["Meta execution<br/>interpreter"]
M -- "declarations" --> C
end
C -- "complete, typed program" --> D["Ownership<br/>linearity, mut"]
D --> E["Memory<br/>RC, copies, arenas"] --> F["Code generation<br/>LLVM, -O0 in development"] --> G["Native binary<br/>no VM"]
Type checking and meta execution alternate: an annotation receives a type, its meta fn returns declarations, which are type-checked in turn. Meta code can call ordinary functions, run by the interpreter, as long as they touch neither pointers, nor the system, nor resources. The ownership and memory analyses therefore only see complete code.
- Development builds.
weaver build -devcompiles without optimization, with debug information: three times faster than an optimized build (find: 0.96 s of clang against 2.2 s), for a program 1.6 times slower. A build compiles each module apart, optimized-O2or under-devnot, and keeps its objects: an edit recompiles the modules it changes, then links them all, optimized with ThinLTO, which inlines small functions across modules and keeps what it optimized;weaver build -wholecompiles the whole program in one piece, much slower, for a program up to 15 % faster. The front, from parsing to LLVM IR, takes only 160 ms of the 2.3 s of a find build; a second backend, Cranelift, will only be considered if unoptimized compilation stays too slow. - Incremental cache. Each expansion is kept with its inputs: the call that requests it, the functions it runs (their text and what their names designate), the structure of the types it reads, and the files declared by
Build.read. An expansion none of whose inputs changed no longer runs;weaver expandshows the code it generates. - Traceable generated code.
weaver build -g, or-dev, writes debug information (DWARF): gdb shows the Weaver source, its lines, the arguments and the variables, including in core and the standard library, whose sources the compiler writes into its cache. A function is named there by its path, which a gdb command quotes between apostrophes,break 'core.panic'. The executable carries its gdb pretty-printers, whichadd-auto-load-safe-pathloads: a string reads as text, bytes asb"…", an array as its elements, an enum as its variant and its fields,Rect = {w = 2, h = 3}. The code a meta fn generates shows at its line in thequotethat wrote it, like a function inlined into the annotation that requested it: underArgs.parse, at the line of#Cli, gdb showsweft.cli.Cli.deriveat its own line in weft/cli/cli.wft. Compile errors in generated code are reported at the line of thequotethat wrote it, followed by the annotation (5.5). A panic writes, under its message, the Weaver functions on the stack, from the nearest call to the farthest, and with debug information the file and line of each call,in pick (main.wft:16), including the functions the optimizer inlined, which it reads from that information; without it, an inlined function does not appear.WEAVER_BACKTRACE=0silences it. - The editor.
weaver lspis the Language Server Protocol server. It checks the whole program separately, after the last modification, with the texts the editor has not saved, and answers from the last successful analysis; core and the std modules the program reaches stay checked from one analysis to the next, the program being checked on top of them: a file’s program is the folder of its weaver.toml, dependencies included, a module of the standard library, or the file alone. It provides diagnostics, hover on a name (type of a variable, signature and doc of a function, type and doc of a field, declaration and doc of a type), and on an annotation the code it generates; go to definition, all the way into the standard library sources; references and rename, which cross open packages, from a program to its dependency, and rewrite a shorthand fieldPoint{x}in full; renaming a contract renames its adoptions and its constraints, renaming its method renames that of each adoption; completion of the members of a value, even at the end of a call or an index,xs.first()?., of a type or a module, and of visible names; the symbols of a file and of the workspace, including those of the branch a module’smeta ifkeeps, the other being greyed out;weaver fmtformatting; and as inlay hints, the type of a binding that does not write it and the cost of each line (6.5). A name from the standard library or core cannot be renamed. - Assistants.
weaver skill [project]installs the Weaver skill for Claude Code into.claude/skills/weaver: a guide to the language, the command and the pitfalls, the SPEC, the grammar, and the core and std docs of the compiler that installs it. Running it again updates it. - Profile-guided optimization.
weaver build -pgo geninstruments the executable, which on exit, even throughExit.codeor a panic, writes the profile of its run (default_*.profraw, or$LLVM_PROFILE_FILE);weaver build -pgo profiles, a.profdata,.profrawfiles or their folder, compiles according to that profile: branch direction, what gets inlined, placement of hot code. A profile from an earlier version still serves, where it applies. - A single tool.
weaver build,run,test,fmt,expand,doc,new,update,publish,yank,lsp,skill. Options are written before or after the program,weaver build prog.wft -o prog; forrun, what follows the program goes to the program.weaver new namecreates a project in the foldername: its weaver.toml and amain.wftthat says hello;weaver new -lib namea library, a documented function whose example and test pass underweaver test. The formatter is canonical and has no options, likegofmt: it only changes the whitespace between tokens, keeps the author’s comments and line breaks, collapses several blank lines into one, and aligns into columns the consecutive lines that lend themselves to it (arm arrows, fields,=of bindings, tags, end-of-line comments). Like golines, it breaks a line longer than 100 columns: a function’s body moves after its=, a statement’s block opens, a condition breaks after its||or&&, then the widest parenthesized group puts one element per line. A chain of at least two method calls that is too long,xs.filter(…).map(…), breaks before each.. What it writes parses back to an identical tree, otherwise it writes nothing;weaver fmt -checklists the files that are not formatted. On a Markdown file, it formats the```weaverblocks and the ones without a language that parse, and leaves the text alone; this specification is checked that way. - Builds. A
[build.<name>]section of weaver.toml records a wholeweaver buildcommand, so as not to repeat it:main, the program, a folder or a list of.wftfiles, relative to the manifest,kind,"bin","lib"for a library for C, or"extension", and its options,out,dev,g,sanitize,tzdata,pgo,target,panic.[build.find]main = "cmd/find"pgo = "profiles/find"[build.find-arm]main = "cmd/find"target = "arm64-linux"out = "dist/find-arm64"weaver build findstands for the command, and so doweaver run find args…andweaver expand find; a command-line option adds to the alias’s options, or replaces its own,weaver build find -dev. An alias takes precedence over a folder of the same name, which./finddesignates.weaver buildwithout arguments compiles all the project’s aliases, or, if it declares none, the program at its root. An alias’s output isbin/<name>under the root of the project, or of the workspace it is a member of, unlessoutor-o; a folder compiled by its path also goes into its project’sbin/, and a lone file next to it.weaver publishleavesbin/and theoutpaths out of the archive, andweaver newwrites a.gitignorethat ignores it. - Tests in the language. A test is a function marked
@test("decoding a User") fn decodesUser() { … }, in the file under test, whose names it sees, private ones included;fn f() !when it propagates errors.weaver testruns them in as many processes as cores at once, each test behind a barrier (8.1, catching a panic): a panic, a false assertion or aProcess.exitfails the test alone, and a test that stops its process fails while the tests after it run in a new one;-run patternselects those whose name contains it. It runs those of the program and of the modules of its package, imported or not, every folder of .wft files under its root excepttestdata, not those of its dependencies or the standard library; a module’s file namedtests.wftor…_tests.wftholds what only its tests use, types and helpers included, and is left out of a dependency or of the standard library that a program uses;weaver buildsticks to the modules the program imports;weaver test weaverruns those of the standard library and the examples of its doc,weaver test weaver.fsthose of one module,weaver test corethe examples of the prelude’s doc, named “core: doc of String.replace, example 1”. A test fails on an error returned by!orfail, whose message is displayed, on a false assertion, or on a panic, reported with where it arose. Annotations say what it expects:@expectPanic("out of bounds")passes if it panics with a message containing the text, and fails if it ends normally, panics with another message, or fails an assertion;@expectExit(3)if it callsProcess.exit(3);@skip("reason")leaves it out, andskip("reason"), called in it, stops it there: a skipped test counts neither as passed nor as failed.weaver test -covermeasures what the tests run: the lines of the functions under test, those of the package or of the std module tested, and the branches of eachif,while,match,&&and||, a function no test calls counting as not covered; it prints the share of each file, and-coverprofile filewrites them in the lcov format, which editors and CI tools read.weaver.testprovidesassert(cond, message), whose message islazy,assertEq(actual, expected)andassertNe(actual, other), which show the values, andassertErr(r), the error of a Result the test expects to fail; each records where it is called. Ameta fnmarked@testchecks generated code: it runs in the interpreter during the compilation ofweaver test, calls the meta fns, and compares their result to the expected text,assertEq(Describe(Counts).text(), "impl Counts { … }");.text()writesDecls, anExpr,Stmts or aTypeinweaver fmtformat. A@suite("store")type groups tests that share its value: itsfn new()builds it before the first, its disposal follows the last, its methods marked@beforeEachand@afterEachrun around each test, and its tests are its methods marked@test, takingselformut self; they run in one process, in order. A@parallelsuite, whose tests takeselfand which has no@beforeEachnor@afterEach, lets them run in several processes, each building its own value.@cases(table)repeats a test for each element of a table, which its parameters receive, a tuple in order:@test("parses ${text}") @cases([("42", 42), ("-7", -7)]) fn parses(text: String, want: Int); the table is computed when the tests run, and each case, named by the description’s interpolation, is a test of its own, two cases named alike failing.@timeout(5.seconds()), on a test or a suite, bounds each test, a minute by default, which-timeout 30schanges,0removing it: past it, the test fails, and the tests after it run in a new process.weaver test -listprints the tests, their cases named, without running them;-report junit.xmlwrites the results in JUnit’s format, which continuous integration reads, a suite per program tested, a case per test, a failure’s message and output, a program that does not compile a failed case. These annotations’ arguments are computed by the program, as a meta fn’slazyparameters are (5.3); std/test/testing.md gives the details. - Documentation. A
///comment documents the declaration that follows it: a type, a field, a variant, a function, asubscript. It is Markdown, read by the IDE,weaver docand meta code (t.doc,f.doc). A function’s doc can carry sections, under a fixed set of English headings, which the compiler checks:/// Reads up to and including `sep`, and appends those bytes to `into`.////// # Parameters/// - `sep`: the byte that ends the read, included./// - `into`: receives the bytes read, after what it already holds.////// # Returns/// The number of bytes appended, 0 at the end of the stream.////// # Errors/// - `Os`: the system refused the read; `reason` says why./// - `BrokenPipe`: the other end of the pipe was closed.////// # Examples/// ```weaver/// var line = Bytes()/// val n = Stdin.lock().readUntil(b'\n', into: &line)!/// ```pub fn readUntil(mut self, sep: UInt8, into: mut Bytes): Int ! IoError# Parametersis a list each of whose entries starts with the name of a parameter in backticks: a name the signature does not declare is an error. A parameter may be left out;selfis not listed.# Returnsdescribes the returned value, in free text.# Errorsis a list each of whose entries names a variant of the function’s error type, and of it alone (of the union, when there is one): a variant of another type is an error, which keeps the doc accurate when the error type changes. An error without variants, aUserErrorbuilt on the spot, has no case to name: its entries are free text,- the text is not a number.; in a union, they stand beside the named ones.# Panicssays when the function stops, in free text.# Examplescontains```weaverblocks, which are tests:weaver testcompiles and runs them, under the name “doc of File.open, example 1”, and a doc whose example no longer compiles is an error. A type’s, an enum’s, a contract’s or an impl’s doc has them too, “doc of IpAddr, example 1”, an impl’s named by its contract, “doc of Deque: Traversable, example 1”; an example that a test cannot run, one that reaches the network or serves until cancelled, is a code block outside# Examples. An example is the body of a function without arguments, which may fail through!: it sees the module’s names and its file’s imports, andassertandassertEqwithout importing them. An error in it is reported at its line in the file.- Any other top-level heading in a function’s doc is a warning; lower-level headings are free.
weaver docchecks these sections and writes the documentation of thepubitems of the program and of the modules of its package, imported or not, or of the standard library (weaver doc weaver,weaver doc weaver.fs), a library being documented withoutmain: their signatures, their doc and its sections, public fields, variants, methods filed under their type, its subscripts, even those written in the adoption ofIndex, and the contracts it adopts, throughimplor through#C. A module’smeta ifcounts there as the branch it keeps for the target, that of the machine or of-target. It writes it as JSON, the model other tools read, the format a published package’s documentation takes (registry): itsformat, a version a newer reader still reads and an older refuses, the package’s full name and version, and paths relative to the package’s root; and lays it out in Markdown or HTML, one file per module plus an index (-format markdown|html,-o folder);-from doc.jsonlays out an already written JSON,-allalso documents what is notpub. A package’s prose goes along: its root’sREADME.md, its landing page, which opens the index, and the Markdown files of itsdoc/directory, pages of its modules shown before their items; aREADME.mdin it presents the module of its directory,doc/net/http/README.mdthat ofnet.http,doc/README.mdthe root module, its text the module’s description: a file’s opening//comment, its maintainers’, is not. A page may open with a front matter between---lines,title: Writing tests,module: weaver.test,order: 10: its title, its first#heading or its file name otherwise; the module it belongs to, the package’s root by default; its rank among that module’s pages, then its title. Another key, a module the package does not have or an order that is no integer is an error, ofweaver doc, and ofweaver fmt -checkfor a page of adoc/directory, the module aside. A page’s code blocks are formatted and parsed as any Markdown file’s, not run.
- Modules. A folder is a module: its files see each other’s names, without
pub.privaterestricts a name to its file,pubopens it to other modules. A subfolder is a module in its own right, imported by its path from the project root, the folder ofweaver.toml:import text.utf8fortext/utf8/. Imports are specific to each file: each one says where the names it uses come from.weaver build app/compiles the module of a folder,weaver build tool.wfta lone file. - Member visibility. A field and a function of an
implare only opened outside their package bypub:type Point = { pub x: Int, pub y: Int, cache: Int },pub fn norm(self). The package is the boundary, not the module: the standard library, a dependency, or the program and its folders see everything of their own types. Outside the package, a hidden field cannot be read, written or matched, and a literal of the type is refused, which preserves a constructor’s invariants:LocalDate{…}would bypassLocalDate.of, which checks the date.private fnrestricts a function of a type to its file. The code a meta fn generates sees what its package sees. A contract’s functions, and a type’s disposal, do not needpub. - Facades.
pub import geom.{Point, Circle}imports these names and also presents them: whoever imports the module finds them there,import shapes.{Point}, without knowing the module they come from.pub importpresents names, not a whole module. - Module constants.
pub meta val Max: Int32 = 15presents a compile-time constant, which another module reads qualified,limits.Max.val monthNames = ["January", …], orpub val, declares a module value, computed once beforemain, in source order: a constant only reads those preceding it, and meta code, which runs at compile time, does not see it (meta valis its own constant). It does not hold a resource. A modulevaris refused: state shared between tasks goes through aShared<T>,val hits = Shared(Map<String, Int>())(8.3). - Dependencies.
weaver.tomlnames the project and its dependencies:[package] name = "app",[dependencies] json = { path = "../json" }. A dependency is imported by its name, then its path within it:import json.{Value},import json.lexer. A project folder that would bear its name, or that ofweaver, is refused: the import would be ambiguous.- Dependencies of tests.
[dev-dependencies], written as[dependencies]is, holds what the package’s tests alone use, a test helper or a reference to compare against: imported only by its test files,tests.wftor…_tests.wft, whichweaver buildleaves out of the programs that use the package, an import from another file being an error. They are resolved and locked with the package’s own, but a dependency’s are never read, nor published as a dependency of its version: those who use the package do not fetch what its tests need. A package in both tables is an error.weaver add -dev publisher/namewrites one there. - The registry. A published package is named by its publisher, the account that publishes it, then its name:
[dependencies] "gerard/json" = "1.4". It is imported by its qualified name,import gerard.json.{Value},import gerard.json.lexer, andimport gerard.jsonbindsjson. The publisher is a Weaver name, in lowercase letters, digits and_;weaveris reserved, and a program folder that would bear its name is refused. The constraint follows Cargo:1.4, or^1.4, takes any compatible version from 1.4.0 on, with the same first non-zero number;~1.4the 1.4.x versions,=1.4.2that one,>=1.2, <3a range,*all. A prerelease,2.0.0-rc.1, is only taken when named. Resolution keeps for each package the most recent version that satisfies all the constraints of the program and its dependencies, a single one per package: two constraints no version satisfies together are an error, which names them along with those who set them. Two packages with the same name, from different publishers, can meet in a program, each under its qualified name. - The official extensions are published by the publisher
weft:weft.width, the display width of a character. What needs neither the compiler nor the runtime lives outside the standard library, on its own release schedule. - On disk or from git, a package that bears a publisher name is already imported by its qualified name:
"weft/width" = { path = "../weft/width" }givesimport weft.width. - weaver.lock, next to the program’s weaver.toml, freezes the resolution: each version, the hash of its archive, and the packages it depends on. A build the lock satisfies does not consult the registry, except for an archive missing from the cache: the cache keeps each archive by its hash, and redoes from it an extraction that no longer matches it.
weaver update, orweaver update json, resolves again. - Publishing.
weaver publishputs the package in an archive, its files that git tracks or does not ignore, with no hidden file, no weaver.lock, and no nested package; it checks it as it is, extracted separately, its dependencies resolved from the registry, its tests run there unless-no-test, its documentation written as JSON, then sends both to the registry’s server with a token of its publisher, whichweaver loginkeeps or$WEAVER_TOKENgives;-nstops before sending and writes them,-o dir. A published package carries in[package]its namepublisher/name, itsversion, itslicense, an SPDX expression, and itsdescription; it may carryauthors,repository,homepage,documentation,readmeand up to fivekeywords. It only depends on the registry. A published version never changes:weaver yank gerard/json@1.4.2withdraws it from future resolutions, a program whose lock records it keeping it, and-undoputs it back. - The registry is static:
https://wefts.weaver-lang.devby default, another http(s) address or a folder given by$WEAVER_REGISTRY.index/publisher/nameholds one JSON line per version there (its version, the hash of its archive, its dependencies and their constraints, whether it is yanked), which resolution reads alone, its signature beside it,index/publisher/name.sig, ed25519;archives/publisher/name/name-1.4.2.tar.gzthe archive, its hash the index’s. wv carries the official registry’s public key;$WEAVER_REGISTRY_KEYnames another registry’s, a mirror’s or a test’s, and a registry without one is refused.wv publishsends a package to a registry’s server,wv yankmarks a version. - Workspace.
weaver-workspace.toml, at the root of several packages, makes them a workspace:[workspace] members = ["weft/*", "examples/*"], patterns each of whose folders that has its own weaver.toml is a member,exclude = […]removing some, and[dependencies]the versions the members share. A member depends on another, or on a dependency the workspace pins, through"weft/cli" = { workspace = true }, with no path or version: the member of that name if there is one, otherwise the[dependencies]line; another name is an error, as is{ workspace = true }outside a member. Members share weaver.lock andbin/, at the workspace root. Publishing a member writes into the archive the version that{ workspace = true }designates, the member’s or the one the workspace pins. At the root,weaver buildcompiles each member’s aliases, its program if it declares none, and checks a library;weaver testtests each one;weaver build findandweaver run findlook for the alias among all members, and an alias that two members declare requires choosing with-p,weaver build -p weft/cli. From a member’s folder, commands target that member. - Git. A dependency can also be pulled from a repository,
json = { git = "https://…/json.git", rev = "v1.2" }: the repository is kept as a mirror in the cache, and the commitrevdesignates is checked out from it. weaver.lock records that commit and the hash of the content: the next build reuses them without consulting the repository, and refuses content that has changed;weaver updateresolvesrevagain.
- Dependencies of tests.
- Imports.
import weaver.io.{File, IoError}imports names from a module,import weaver.fs.Metadataa single one,import weaver.fsthe whole module, which is then cited:fs.Metadata,fs.Fs.readText(p). Onlypubitems come out. A type’s functions (impl) come with it: importingFilegivesFile.open(p)andf.read(&buf). The variants of an imported enum are used without qualifying it.asrenames what is imported, a name as well as a module:import weaver.io.{File as IoFile, IoError},import weaver.fs.Metadata as FsMeta,import json as jthenj.parse(s). Two imports that would give the file the same name, or an import that would reuse a name of the module, are refused at the import, and the message suggestsas:import geom.{Point}thenimport draw.{Point as Pixel}. A renamed type keeps everything that comes with it, its functions and its variants; messages name it by its qualified name,weaver.io.IoError, the same in every file. A type can also be written by its full path without importing it,fn main() ! weaver.io.IoError, and its functions through the module that holds it,import shapes as shthensh.Circle.of(2). - Namespaces.
namespace Env { … }groups under a name functions that have no value to carry:pub namespace Env { pub fn get(name: String): String? … }, called asEnv.get("HOME"), like a type’s function. A namespace is imported like a type,import weaver.env.{Env}, or reached through its module,env.Env.get(…); its functions, however, are not imported on their own, so that a generic name,get,read,remove, always reads together with what gives it its meaning. Inside the block, its functions are visible without naming it, and take precedence over module functions that would bear the same name. A namespace has no values: it is not used as a type,val e: Env, nor constructed,Env{}; its functions have noselfnorsubscript, and are all declared in its block, which noimplextends. Its doc and visibility are written like a type’s, andweaver docpresents it with its functions. std thus groups the operations of its domains,Fs,Env,Net,Process,System, which its root, theweavermodule, presents:import weaver.{Env, Fs}is equivalent toimport weaver.env.{Env}thenimport weaver.fs.{Fs};Cin weaver.c,Hexin weaver.crypto. - Prelude. The language’s types (
Int,UInt8,Bool,Char,String,Bytes,Array(T[]),Option,Result,Ordering…) are always visible, with their methods: they are the primitives every program needs. Everything else is imported, library functions and collections included:import weaver.io.print,import weaver.collections.Map. - C FFI.
extern "C" fn write(fd: Int, data: Ptr<UInt8>, n: Int): Intdeclares a C function, which is only called inside anunsafeblock. It exchanges integers of any size, floats, booleans and pointers, and by value C records, whose fields are of these types, fixed arrays or other C records: they have the layout of a C struct, and a shim, which clang compiles for the target, passes them as the platform ABI requires. APtr<T>parameter also accepts&x, the address of a variable of type T for the duration of the call, like amutparameter. A variadic C function ends its parameters with...,extern "C" fn printf(format: Ptr<UInt8>, ...): Int32: its extra arguments, which the call does not name, are integers, floats, booleans, pointers or C records, promoted as in C, a Float32 to double, an integer shorter than anintor a boolean toint; an integer literal there is an Int, which%ldreads, and42.toInt32()passes anint. A call that exchanges a struct by value, among its fixed or its extra arguments or as its result, goes through a shim of its own, written for its arguments’ types: C cannot pass...on, but the shim calls the function as C does.cFunction(f)gives the address, aPtr<UInt8>, of a program function f that C will call back, the comparator ofqsortfor instance: a wrapper with the C ABI that calls f. f is neither generic, nor meta, nor async, and exchanges integers, floats, booleans, pointers and C records by value: for a record, the address C receives is that of a C function the shim defines, which clang compiles for the platform’s ABI and which passes each record on by address; a panic in f stops the program as anywhere else, without unwinding the C frames. - Low-level bodies.
unsafe fn atomicAdd(cell: Ptr<Int>, n: Int): Int = llvm"""…"""writes a function’s body in LLVM IR, which the compiler copies into an internal function that LLVM inlines into its callers: an atomic, an intrinsic or a pointer operation then costs neither a call nor a pass through the C bridge. Open to any package, asunsafeis, an extension needing theunsafecapability: its form, LLVM’s textual IR within the constraints below, is a public interface.unsafe fn atomicAdd(cell: Ptr<Int>, n: Int): Int = llvm"""%old = atomicrmw add ptr %cell, i64 %n seq_cst, align 8%new = add i64 %old, %nret i64 %new"""- The function is neither generic nor variadic; like
extern "C", its parameters, read-only, and its result are integers, floats, booleans or pointers, Int ani64, Bool ani1,Ptr<T>aptr. The body names its parameters%cell, and no symbol but LLVM’s intrinsics,@llvm.…, which adeclareline of the text declares: it neither calls the program nor C, reads no global, and does not allocate (@assert(noalloc)counts it as such). The call requiresunsafe, and meta code cannot run it. - LLVM reads the bodies on their own before the program, once per set of bodies (the object cache keeps the verdict): an error is reported at its line and column in the body.
- The function is neither generic nor variadic; like
- Extensions. Separately compiled code, which a host loads, starts, stops and reloads while running,
weaver.extension: hot reload of a server in development, or a large application that only loads what the session needs.[build.hello] kind = "extension", orweaver build -extension, producesbin/hello.wvx, a shared object that does not embed the runtime: it uses the host’s, and is held to the host’s ABI, whose versions it states and the host checks at load time, whatever build of the compiler made each.// The extension: its start function, which the host calls when starting it.fn start(ctx: Context) ! ExtensionError {val log = ctx.require<any Log>()! // what the host provides to itctx.provide<any Greeter>(Polite{log}) // what it provides to the host}// The host.val ext = load("bin/hello.wvx")!ext.provide<any Log>(console)ext.start()!val greeter = ext.service<any Greeter>()print(greeter.greet("world")!) // a Greeter method, which also returns ExtensionErrorext.reload()! // the new version, recompiled in the meantime- Services are
any Cvalues exchanged by the host and the extension, under the qualified name of the contract C and the fingerprint of its shape: its methods, in order, with the types of their parameters and results, and the records and enums those exchange, down to their fields. Both sides must have compiled the same contract, otherwise the service is not found; what changes neither a method nor the data it carries, another function of the API package, a comment, a module one side does not import, changes no service. They are resolved when the extension starts; a dynamic registry can be written on top, as a library. - The handle
Service<any C>calls C’s methods:s.m(args)returns m’s result, or ExtensionError,Stoppedif the extension is stopped,Failedif it has failed. It does not hold the service, which never leaves the call: after a reload, it calls the new version. What crosses the boundary, arguments and results, is made of data, or of handles: a function, anany, a pointer, a shared object or a resource would point into the code of the other side, which may disappear, and compilation refuses them. - The lifecycle:
startsets up the extension’s constants then calls itsstartfunction;stoprefuses new calls, waits for those in progress, calls itsfn stop()function if there is one, and disposes of its services, while its code is still there;unloadunloads its code,reloadreloads it from its file, with what the host was providing to it, then starts it;state()says where it stands. Stopping also cancels the tasks it launched, and waits for them to finish, at their next cancellation point: its code is unloaded afterwards, unless it created a pool, whose threads could run it, or athread var, whose disposal will call it. - A registry holds the extensions a host loads together, which exchange services with each other as with the host:
val plugins = Registry.new(),plugins.load(path)in place ofExtension.load,plugins.provide<any Log>(console)for what the host provides to them all, andplugins.service<any C>(), the handle of the service whichever of them provides. An extension takes another’s service by the same handle,ctx.service<any C>()in itsstart, which it keeps: unlike the host, another extension may stop, fail or be replaced, so its service is called under its own barrier, its methods also returning ExtensionError, and is looked for at each call among the extensions started then, the first loaded if several provide it. A panic in the extension called fails that one, and the caller receivesFailed; a call in progress is one its stop waits for, wherever it comes from. - The events of a service:
s.isProvided()says whether the service of a handle is provided now, ands.ready()waits until it is, at once if so;var w = s.watch()thenw.next()waits for each of its arrivals and departures, in order, the first its arrival if it is provided. A provider reloaded, or replaced by another, is a departure then an arrival. Both take a deadline,ready(by: clock.timer(5.seconds())), and then return false, or None, once it passed; they are cancellation points, and return the same when the task is cancelled: an extension that watches a service from a task,while w.next() is Some(e) { … }, sees its loop end when it stops. - What stays on its side crosses by its handle.
ctx.lend<any Cursor>(rows)lends an object the extension keeps, a resource, a shared object or what holds a function, and gives itsService<any Cursor>: a handle is data, which a service’s method returns or takes,fn open(self, shelf: String): Service<any Cursor>,fn subscribe(self, listener: Service<any Listener>), and its calls go through the lender’s barrier as any service’s. What is lent lasts until its holder releases it,handle.release(), its calls then returningMissing, or until the extension stops,Stopped. A service of the host’s, taken byctx.require<any C>(), is called as anyany C, without a barrier: what the extension hands it must be data or a handle, the host keeping it past the extension’s unload otherwise; what the host hands back is data, or anany Cof its own, which outlives the extension and is called through its table, C then held to the same. A contract that takes or gives anything else is refused where the host provides it and where the extension requires it. - A call across the boundary does not pass values as a program’s own calls do, which is the compiler’s to choose. An entry of a service’s table takes the service’s environment, then each argument, a scalar as it is, a number, a
Bool, aChar, and any other value by its address; a result that is not a scalar is written at an address passed last. A function of scalars and addresses is called the same way on a platform whatever splits a struct into registers: its C convention, where the platform alone decides. A service leaves in an environment of its own, which names the table of those entries, one per contract; where it arrives, a table of relays makes an ordinaryany Cof it. Anany Ca host hands back crosses the same way, as the address of its environment. A program’s own calls through anany Care as before. - A
task valis read by its binding in the task, which a call across the boundary carries with it, each way: an extension reads thedeadline, theclockand the log’shandlerits host set, a host’s service therequestIdan extension bound. It is one mechanism for the whole program, and nothing is to be handed over. The two sides find a binding under one key, made of thetask val’s name and of the shape of its type, as a service’s is of its contract’s: where one side holds another shape of the type, its bindings are not the other’s, and each reads its own, or the default. A binding of anany Ccrosses as a service does, in the environment it leaves in, called by addresses; any other crosses as it is. Athread varis each side’s own. A task launched under an extension’s binding is the extension’s, even from a host’s service: its stop waits for it. - The types that cross are data: the language’s own, numbers,
Bool,Char,String, arrays, tuples,Option,Resultand unions of them, and records and enums, a package’s or std’s alike. A host and an extension are held to one representation of those, whichwv abiwrites down, and to the same types: a service’s fingerprint describes what its contract exchanges down to the fields, so that two sides that do not have the same type, a package’s another version or std’s, do not name the same service, and the extension that requires it does not start. Nothing is listed that may cross: a program that hands over a type its other side has otherwise fails where the two meet, as it would not compile. What does not cross is what is not data: a function, a resource, a shared object, a pointer, but for aWaitSource, a deadline, whose object is the runtime’s, shared by both sides. - The constants of an extension do not outlive it: a string literal, a constant array, the place an
#Errorvalue was built at are addresses in the extension’s file, which its unload removes. What crosses its boundary is owned first: those constants alone are copied to the heap, what the heap already holds is shared, and a constant of the host’s, which never goes, is left as it is; an error forgets where it was built,e.location()then None. It is done where a value leaves: what a service returns, in its handle’s call; what an extension hands another’s service, where the call is written; and, in an extension’s code, the arguments of any call through anany C, which may be a service of its host. A constant is told at run time, a string or an array without a block whose bytes lie in a loaded extension’s image; an array or a box whose elements could hold one is copied, its elements owned in turn. Within an extension, a literal costs nothing more. - Several services of one contract are told apart by a name:
ctx.provide<any Greeter>(formal, named: "formal"), andnamed:likewise onrequire, onservice, and on a host’sext.provideandplugins.provide: a host’s datasources are as manyany DataSource, each under its name. A name is made of letters, digits,.,_and-; without one, the service is the contract’s unnamed one, as before. Astartdeclares them the same way,@provides(Greeter("formal"), Greeter("casual"))and@requires(Store("main")).plugins.services<any C>(), andctx.services<any C>()in an extension, give the handle of every service of the contract the registry’s started extensions provide, named or not, each extension in the order they were loaded and its services in the order it provided them;s.label()is a handle’s name, “” for the unnamed one. Each of those handles is of one extension’s service, and does not move to another provider. - Choosing among providers: several extensions of a registry may provide the same service, under the same name. Each says what it thinks of its own,
ctx.provide<any Store>(store, rank: 10, properties: ["region": "eu"]): a registry’s handle,plugins.service<any C>()orctx.service<any C>(), calls the one of highest rank among those started, the first loaded among equals, and moves to another when that one stops or a higher one starts, whichwatchgives as a departure then an arrival. A handle tells of the extension that answers now:s.provider(), its file’s name,s.rank(), ands.properties(), a map of texts. Choosing otherwise is filtering the handles of them all, each of one extension’s service:plugins.services<any Store>().filter(s => s.properties()["region"] == Some("eu")). A property’s key holds no=, and neither a key nor a value a line break. - Versions of a service: two versions of an API package whose contract differs name two services, which a registry holds side by side: an extension built with one finds the provider built with the same, whatever else is loaded, each carrying its own copy of the package. An extension’s file states, after each service its
startdeclares, the version its contract’s package declares,[package] version: it pairs nothing, the fingerprint does, and tells, when a service required is not there, which version is:it requires the service acme.greet.Greeter 1.0.0, and hello.wvx provides it as of 2.0.0. A range of versions a consumer accepts waits for the layout of what crosses to be frozen. - What an extension provides and requires: its
startdeclares them, by their contracts:@provides(Exporter)the services it provides,@requires(Store, Log)those it needs to start, from the host or from another extension. The compiler checks each name as it does the code’s, and the extension’s file carries them, with its capabilities: a registry knows them before any code of the extension runs.ext.start()fails withStart, the extension left as it was, while a service it requires is provided neither by the host nor by a started extension of its registry; and puts it in failure if its start did not provide a service it declares.plugins.start()starts the registry’s extensions that are loaded or stopped, each once what it requires is there, whatever the order they were loaded in, and fails withStartnaming those that did not start and why: a service nothing provides, one of an extension that did not start, or extensions that wait for each other.plugins.stop()stops them the last started first, one that calls another’s services before it, andplugins.stopBy(deadline)each with a deadline. An extension stopped or reloaded alone leaves those that require it started: their calls returnMissinguntil it is back, whichwatchtells them. A service taken without being required,ctx.service<any C>(), is one the extension does without. - The ABI is not the extensions’ own: it is what compiled code and a runtime share, whatever links them while the program runs, an extension today, a library loaded as a
.sois. A binary says which ABI it was built for, and a runtime that supports that ABI loads it. It guarantees one thing, that the code loaded is compatible: what is linked while the program runs is not built again for a minor or a patch of the compiler. That holds from Weaver 1.0.0: before it no version is stable, and any of them may break the ABI, its major raised. It does not say what the code does once loaded, nor when it may go. An extension states, in its descriptor, the version it was built for of each part of what it shares with a host, major then minor: the descriptor itself, which states its size and only grows at its end; the runtime it borrows from the host; the layout of the data that crosses. A minor adds, and an older extension still loads; a major breaks, and the host refuses the extension before running any of its code, naming the part and both versions.wv abiwrites down what those parts hold, a line each thing: the descriptor’s fields and what an extension’s file ends with, the types every program names and how a service is named, and the runtime’s entry points generated code calls, theextern "C"declarations of core and std with their types, each held to the function the runtime’s C defines; then what that code knows of the runtime’s objects, where a block’s count is and what its sign bit says, an environment’s first words, the size of a stack, held toruntime/weaver.h. A line says so where the two sides differ. The compiler’s tests compare it with the description committed: a line added owes its part a minor, a line removed or changed a major, and the description is only rewritten once the versions say so. Until what the description leaves out is frozen too, the layout of the data that crosses and what generated code knows of the runtime’s objects, an extension must still come from the very compiler that built its host. - Unloading rests on a contract between a host and its extensions, which the compiler does not prove. What an extension hands over through weaver.extension is guarded: a service or a handle is called under its extension’s barrier and says
Stoppedonce it is gone, what a call returns is owned before it leaves, a host’s contract takes no function. These are there so that a host does not write the guard itself; they are not a proof that nothing of an extension is left. What crosses otherwise, an event logged, atask val’s binding, is consumed where it arrives and dropped: a host that kept one past the unload of the extension it came from would hold its code. The one promise is that a host is not brought down where the runtime can help it. - A stop with a deadline:
ext.stopBy(clock.timer(5.seconds()))stops likestop, but waits for the calls in progress and for the tasks only until the deadline, and returns true if it passed. The stop then gives up on what still runs: the extension is in failure, itsstopfunction does not run, and its code stays loaded;reloadputs it back in service. A loop of an extension looks, at each turn, whether its host gave up on it, and then panics back to the host, its cleanups run: a call or a task that loops without a cancellation point ends there, the call returningFailed. The innermost loop over an array or a string does not look, which ends with its data. Only C code, which theunsafecapability grants, can run on past the deadline. - A panic in the extension, during a call from the host, its start or a task it launched, runs its own cleanups, then returns to the host: the extension goes into failure, its tasks are cancelled, the call returns
Failed, and the host carries on;reloadputs it back in service. The promise of a task that panicked has no result: awaiting it panics in turn, in the extension. The arguments of a service call are computed on the host’s side, before entering the extension: a panic while computing them is the host’s. - Capabilities: an extension only imports from std what touches the system,
weaver.io,weaver.fs,weaver.process,weaver.net,weaver.env,weaver.system,weaver.taskandweaver.c, if[extension] capabilities = ["weaver.io"]grants it, in its weaver.toml, and uses neitherunsafenorextern "C"without theunsafecapability; the rest comes from the host, through the services it provides. A hosttask val,clockfor instance, also holds in the extension. - The host’s policy:
load(path, Policy{capabilities: Some(["weaver.io"]), unsafe: .signed([key])})states the capabilities it grants, all of them by default, and what it allows ofunsafe:.deny, by default,.allow, or.signed(keys), an extension signed by one of its trusted keys, ed25519 public keys. An extension’s file ends with what it declares, its capabilities and its signature, which loading reads and checks before running anything of it; a byte changed after signing gets it refused.weaver key new acme.keycreates a key pair, the private one inacme.key, the public one, the one the host receives, inacme.key.pub;sign = "acme.key"in[build.<name>], orweaver build -sign acme.key, signs the extension. The signature authenticates the file; without it, the declared capabilities are only a promise of the compiler.
- Services are
- Libraries for C.
export "C" fn total(n: Int): Int { … }declares a function that C calls by its name: the same subset asextern "C"for its parameters and its result, integers, floats, booleans, pointers and C records by value, read-only. The header defines each record it exchanges,typedef struct { int64_t x; double y; } Point;, under the record’s name and with its fields’, which C must accept; a struct of a@cimportmodule keeps its C name, and the header includes the one it comes from. A shim, compiled by clang, receives the structs as the platform’s ABI passes them. It is neither generic, nor meta, nor async, nor a method; thewv_prefix is reserved for the runtime. A function that fails,export "C" fn parse(p: Ptr<UInt8>, n: Int): Int ! ParseError, its error declared, returns C anint32_tcode, 0 for success, its value through a last pointer,int64_t *out, which may be null; each variant of each error type the library’s functions return has its own code, which the header lists,CALC_PARSE_ERROR_EMPTY, andcalc_error_message()gives the message of the last error on the thread, until its next one. Two records of the same name exchanged with C are rejected: they would define the same type. Another record or enum of the program, not generic, a resource too, C only holds through a pointer, which the header declares,typedef struct Words Words;: returned, it is a block the library allocates; a parameter lends it,mutlends it to modify, andsinktakes it back, the function disposing of it, and frees its block. Each block carries its type’s mark, checked each time C hands it back: a pointer of another type, or one already given back, stops the program. A value C holds is used by one thread at a time.[build.calc] kind = "lib", orweaver build -lib, compiles the program as a library, withoutmain:bin/libcalc.a,bin/libcalc.so, andbin/calc.h, which declares each exported function with its doc. The Weaver code and the runtime form a single object in it, in which only the exported functions remain global: two Weaver libraries link into the same C program, each with its runtime.-sanitizecompiles it with AddressSanitizer, whose runtime the C program links,-fsanitize=address;-pgo geninstruments it, its profile runtime inside, and the C program writes the profile on exit, which-lib -pgothen reads.- The first call sets up the module constants, the task lets and the pools, once, the other threads waiting; they live until the end of the process. The program has no arguments.
- Each C thread has its own state, its pending
defers andusings, itstask valbindings, its thread vars, disposed of at the end of the thread: two threads call the library at the same time without sharing anything. An exported function can wait,await, the C thread waiting with it. - An exported function that reaches a recursion runs on a runtime stack, which the recursion knows how to grow: a C thread’s stack can be small. The others stay on the thread’s stack, without the cost of a switch.
- A panic writes its message, runs the thread’s cleanups, then stops the program with
abort(): it does not cross the C frames.
- C headers.
@cimport("zlib.h", link: "z") import zlibhas clang read the header, and the compiler writes a module from it, filed under the import’s name: its names are used qualified,zlib.deflate(&strm, zlib.Z_FINISH), and thelinklibrary is added to linking. Only the header and what it includes with quotes count; the system headers it includes with angle brackets only provide the types it uses.- Functions become
extern "C" fns, their integers translated faithfully:intto Int32,unsignedto UInt32,longto Int,unsigned longandsize_tto UInt64,charto Int8 (UInt8 on arm64). Achar *, achararray, avoid *or a function pointer become bytes,Ptr<UInt8>andUInt8[n], another pointerPtr<T>. A described struct is passed by value, and a variadic function ends its parameters with.... A function that exchanges by value an incomplete struct, an unnamed union or along doubleis left out, and the module says so. - A complete struct becomes a record with the same layout, with public fields. An incomplete struct, or one with a bit field, an unnamed union or a flexible array, becomes an opaque record, only handled through a pointer.
- A union becomes a record of its bytes, of the size and alignment clang gives it; a method reads each member,
n.i(), andfromfollowed by the member’s name builds one whose other bytes are zero,mini_num.fromI(7), whichC.bitsAsdoes. Its module’s functions exchange it by value, and a struct that holds one: through a C function that includes the header, which clang compiles knowing the union, whose members decide how the ABI passes it. C calling back and a library’s export do not exchange one by value: they pass its address. - A typedef becomes a type alias. An enum constant, and a
#definethat computes (an integer, a float or a string, literal or combined by C operators), becomepub meta vals. - A macro with parameters has no types for the compiler to read: the module only lists them. One is called through a declaration that gives its types,
@cmacro("sys/wait.h") extern "C" fn WEXITSTATUS(status: Int32): Int32,name:naming the macro when the function is named otherwise,@cmacro("sys/stat.h", name: "S_ISDIR") extern "C" fn isDir(mode: UInt32): Bool, andlink:a library its expansion calls. The compiler writes a C function that calls the macro with those types, in a unit that includes the header, which clang compiles: a type the macro’s body refuses, or a macro the header lacks, is reported at the declaration’s line. Its parameters and result are integers, floats, booleans and pointers, a pointer to a struct of a@cimportmodule written with its C name, for a body that reaches into it; nothing for a macro that is a statement,FD_SET(fd, set); a record goes by its address. It is called inunsafe, as any C function. - The module is kept in the cache, by the hash of the header and of clang.
[c] include = ["…"]andlib = ["…"], in weaver.toml, add the folders where headers and libraries are searched for.weaver.ccomplements this with its namespaceC:C.null(), a null pointer,C.zeroed(), a zeroed struct,C.string(s)andC.readString(p), a zero-terminated string in one direction and the other.
- Functions become