Resources
A Disposable value is linear: the compiler rejects any path on which it is neither disposed of nor passed on. using, defer, errdefer and arena are the four ways to organize that disposal.
7.1 The contract
Section titled “7.1 The contract”contract Disposable<E = Never> { fn dispose(sink self) ! E}
contract AsyncDisposable<E = Never> { async fn dispose(sink self) ! E}A resource whose disposal waits, closing a connection or flushing a remote buffer, adopts AsyncDisposable (weaver.task): leaving a using awaits its dispose on the spot, like await f(x), and its error follows the rules of a disposal (7.3).
On every execution path, a Disposable value must be:
- bound by
using, - passed as
sinkto a function, - returned,
- stored in a field, which makes the container
Disposablein turn, - or disposed of explicitly by
x.dispose()!.
val f = File.open(p)!if p.isEmpty() { return 0 // error: the resource `f` is neither disposed of nor passed on before this `return`}The error is placed at the exit that loses the resource: return, fail, break, continue or the ! that propagates an error, otherwise at the declaration.
Conformance. impl Disposable for File { fn dispose(sink self) ! IoError { … } }. dispose consumes its resource and returns nothing, or an error: E, which the impl can state, impl Disposable<IoError> for File, and which otherwise comes from the signature of dispose. Disposable is an ordinary contract: a bound T: Disposable asks for a resource that is disposed of without error, Disposable<Never>, and T: Disposable<IoError> for a resource whose disposal returns IoError; fn closeAll<T: Disposable>(sink xs: T[]) disposes of resources of any type. A Result that carries a resource is handled immediately, by !, catch or a match whose success arm adopts it, Ok(using f) => …, or is returned: it is not stored in a variable, nor in an arm’s name or _, which would lose it.
Propagation. A record with a Disposable field, an enum or a union with a variant that carries one, a tuple, an array or a fixed array of resources are Disposable in turn, without declaring anything: EntityManager{Connection{socket}}. Their disposal disposes of their parts in reverse declaration order: the active variant for an enum, the elements from last to first for File[]. An explicit impl Disposable runs first, then the parts are disposed of, as in Rust: it does not have to handle them, and does not give up self.
- A part stays with its container. It can be borrowed (
em.conn.query(),&em.conn,for c in pool,if o is Some(c)), but does not leave it:val c = em.connorkeep(pool[0])are rejected when the container has its ownimpl Disposable. Without one, its disposal being only that of its parts, an owned variable gives them up:val c = p.conn, thenp.log.dispose();val (a, b) = pair()receives those of a tuple,using (a, b) = …binds them withusing, anondisposefollowing it receiving the error of each, if they fail with the same type, and_does not abandon one...sourceandwithborrow their source and do not copy a resource: give the resource field a new value,conn.with(socket: other). Only the functions of its type can take a container apart: the one that receivessink selforusing selfgives up a part,return self.inner, or a variant’s payload,match self { Some(using x) => x … }, which consumes self without running itsdispose; its other parts, which are not resources, are disposed of with it.Buffered.finish()thus returns its writer. A container that gives up a part must give up its other resources, each once and on every path:fn split(sink self): (A, B) = (self.a, self.b), orself.b.dispose()thenreturn self.a. - Abandonment and adoption. A resource changes owner in a way that is written on both sides. Returning a value abandons it:
File.open(p),files.pop(),files.replace(i, f)which stores f in place of the element it returns, an iterator’snext(). Receiving it adopts it, and that is written:using x = …,Some(using f),for using f in …. Then giving it to someone else,runs.push(f), makes them responsible in turn.xs.drain()empties an array into an iterator that abandons its elements in order; abandoned midway, it disposes of those it has not returned. Nothing is abandoned or adopted silently: iterating an array,for f in files, borrows, and receives nothing. - Errors. A container’s disposal can fail with the error of each of its parts: with the union of their errors when they differ,
() ! NetError | IoError. A failing part does not stop the following ones, and the first error is returned.
Shared objects. A sync type (8.3) adopts Disposable, but is not a resource: its copies designate the same object, and its dispose runs when the last one disappears, and cannot fail. It does not make its containers Disposable.
Generics. A type parameter can receive a resource: fn count<R: Reader>(r: mut R) reads a ReadFile. The generic’s body is then checked with R treated as a resource: it passes it on (mut, sink, return) without copying or losing it. fn twice<T>(x: T): T[] = [x, x] is rejected at the call twice(file), with the line of the body that copies. A default method of a contract is checked the same way for each resource that adopts it.
7.2 using, defer, errdefer
Section titled “7.2 using, defer, errdefer”All three have block scope, not function scope.
| Form | Executed on block exit | Use |
|---|---|---|
using x = expr |
Always: x.dispose() |
Resource acquired and disposed of at the same level |
defer stmt |
Always | Cleanup action without a contract (log, metric, unlock) |
errdefer stmt |
Only if the block exits with an error (!, fail, panic) |
Multi-step construction |
Consuming a using binding. A using binding can be consumed: passed as sink, returned, or destroyed by a sink self method such as finish() or close(). It is then no longer disposed of at the end of the block; it is on the error paths that precede its consumption. This builds a resource under protection before returning it.
using parameter. fn merge(using runs: Run[]) receives the value as sink, and disposes of it at the end of the function if the body has not consumed it. Its disposal follows the rule of a binding (7.3): if it can fail, the function returns its error, or consumes it; to handle it on the spot, receive it as sink and bind it in the body, using c = c then ondispose e => …, since a parameter has no line to write it on.
Linearity rules. A resource is consumed on all paths that continue, or on none; it is not consumed in a loop if it was bound before, since it would be consumed on every iteration; a using binding is not reassigned. A defer or an errdefer cannot exit its block (return, break, !…), and errdefer only makes sense in a function that returns a Result.
fn copy(src: Path, dst: Path) ! IoError { using input = File.open(src)! using output = File.create(dst)! input.pipeTo(&output)!} // output then input closed; a flush error propagates
fn openDb(cfg: Config): Db ! DbError { val conn = Conn.open(cfg.url)! errdefer conn.dispose() val cache = Cache.load(cfg.cache)! errdefer cache.dispose() conn.migrate()! return Db{conn, cache} // success: responsibility passes to Db}
type Db = { conn: Conn, cache: Cache } // Disposable: its fields are
fn spill(lines: Bytes[], dir: Path): File ! IoError { using out = Buffered(File.anonymous(in: dir)!) for line in lines { out.write(line)!; out.writeByte(b'\n')! } using file = out.finish()! // consumes out file.rewind()! return file // consumes file: it is returned to the caller}7.3 Disposal errors
Section titled “7.3 Disposal errors”- Successful block, failed disposal. The error of
disposebecomes the function’s result. - Block already failing. The main error exits alone: a disposal that fails then goes to its
ondisposeif there is one, and is otherwise dropped. An error carries nothing but its value, which a list of attached errors, Java-style, would change for everyone. - Function that does not return
Result. The close error is handled byondispose, declared at acquisition and valid for all exits of the block. Without it, it is a compile error.catchandondisposecan start a line:using out = File.create(p)catch e => return Exit.code(1) // opening failedondispose e => log.warn(e) // closing failed, at the end of the block - Exiting from an
ondispose. Its body runs where the disposal fails, during the block’s exit, and can in turn exit from it.fail emakes the close error the function’s error, in place of the one already exiting;return,breakandcontinuereplace the exit in progress,breakandcontinuetargeting a loop that contains the binding. An exit diverted this way disposes of the value it carried, that of areturn:continueresumes the loop, and the function no longer returns what it was about to return. What remains to be disposed of, the bindings anddefers earlier in the block, then those of the enclosing blocks, is disposed of once, as on any exit; what an inner block already left held is not disposed of again. A body that completes lets the exit in progress continue.for p in paths {using out = File.create(p)catch e => continue // a file that does not open: on to the nextondispose e => {log.warn(e)continue // closing it failed: on to the next as well}out.write(data) catch _ => break} - Error type. The disposal error is imposed on the function’s error type only on the success paths where the value is actually disposed of. A value that is consumed or returned imposes nothing.
- Infallible disposal. A
disposedeclared! Nevercannot fail and imposes nothing. This is the case forTaskScope, since cancelling cannot fail. ReaderandWriter.weaver.iodeclares both contracts:read(mut self, buf: mut Bytes, from: Int = 0): Int ! IoError, withr.pipeTo(&w)by default, which writes to a Writer what remains to be read and returns the number of bytes, andwrite(mut self, data: Bytes) ! IoError, withwriteByteandwriteLineby default; a string is written as is, by its bytes.BufReaderreads up to a separator:readUntil(sep, into: &data)appends the bytes up to and includingsepand returns their count,readLine(sep)returns the next line orNone, andreadBlock(sep)the complete lines the buffer holds, a view without copying, the cut-off tail moving back to the front before the next read: line-by-line processing thus goes through a block in one sweep, without holding the file in memory. The buffer size is chosen according to the use,Buffered.new(r, capacity: 1 << 17),Stdin.lock(capacity: …).Buffered(r)puts a buffer in front of aReader, which it makes aBufReader, or in front of aWriter, whose buffer it flushes on disposal;close()flushes it and returns the error.ReadFileis aReader;FileandStdoutareWriters, andFilecan also be read back.Stdout.buffered()shares the buffer ofprint, which keeps writes in order, and its disposal flushes it:using out = Stdout.buffered()makes the flush error (disk full) the function’s result, whereprintignores it. In memory,Bytesis aWriterthat appends what it is given,var out = Bytes()thenrender(&out)!, andBytesReader.new(data)aReaderand aBufReader: a test gives code written for files a text, and reads back what it wrote.ReaderAtreads at any position without a cursor,readAt(&buf, at: n), withsize()andduplicate(), another reader of the same bytes of its own:ReadFile,FileandBytesReaderadopt it, what a format that keeps its index at its end reads, a zip archive’s directory.- Users and groups.
System.userName(uid)andSystem.groupName(gid), from weaver.system, give the name of an owner according to the system database (NSS: /etc/passwd or a directory service),Noneif it has none;System.userId(name)andSystem.groupId(name)go the other way. - Hashing.
weaver.cryptocomputes SHA-256 (FIPS 180-4), written in Weaver, without a system library:sha256(data)returns the 32 bytes of aBytes’s digest,Sha256.new(),h.update(data)thenh.finish()compute it in pieces, andHex.encode(bytes)writes bytes as hexadecimal digits, in lowercase.Sha512andsha512(data)do the same on 64 bytes;ed25519Sign(seed, message)signs with the private key whose 32-byte seed isseed(RFC 8032), the same signature for the same message, anded25519PublicKey(seed)returns its public key. SHA-1 and MD5 for what still names them, HMAC, X25519 exchanges and ChaCha20-Poly1305’s authenticated encryption beside them. It does not touch the system: an extension imports it without a capability. - Display width. Outside the standard library, the official package
weft/width:import weft.width.displayWidth, thendisplayWidth(ch)gives the width ofchin a terminal: 0 for a character that combines with the previous one, 2 for a wide character (CJK, emoji), 1 otherwise,Nonefor a control character. Its tables are those of glibc’swcwidth, so thatwc -Lcounts like GNU. - Whole files.
Fs.read(path)returns an entire file asBytes,Fs.readText(path)as a String, the errorInvalidUtf8(of kind.invalidData) signalling contents that are not UTF-8;Fs.write(path, data)andFs.writeText(path, text)write one, created or truncated, andFs.copy(from, to)copies one with its permissions, by the system, without the content passing through the program’s memory (copy_file_rangeon Linux,fcopyfileon macOS). - The environment.
Env.get(name)reads a variable,Env.set(name, value)andEnv.unset(name)change it for the whole process and the programs it launches afterwards, failing withInvalidNamefor a name that is empty or holds=;Env.vars()lists them,Env.homeDir()gives $HOME, or failing that the user database’s directory. The runtime guards the environment with a lock that every access takes, a child’s launch included: a task reads it while another changes it. The C library’s own reads, those of name resolution, do not take it. - Trees.
Fs.exists(p)says whether a path names a file, a symbolic link followed;Fs.createDirAll(p)creates a directory and its missing parents, an existing directory not being an error;Fs.removeAll(p)deletes a file or a whole tree, an absent path not being one either, and a symbolic link being deleted, not followed;Fs.walk(root, (path, md) => …)visits a tree depth first, in name order, the callback returning whether to enter a directory;Fs.symlink(target, link)creates a symbolic link,Fs.setCurrentDir(p)changes the current directory of the process. - Paths and standard input.
weaver.path.Pathis a path, kept as the system gives it (its bytes are not necessarily UTF-8):display,join,fileName,extension,parent. The functions ofweaver.ioandweaver.fsaccept aPathor a string, through theAsPathcontract. An open directory,weaver.fs.Dir, is a resource whose entries are read, described, opened and created by their name, relative to it (dir.openFile(name),dir.createFile(name),dir.createNewFile(name),dir.openDir(name),dir.createDir(name)), removed (dir.remove(name),dir.remove(name, dir: true)for an empty directory), and say what the user may do with them, like access(2) (dir.access(name, read: true)): a tree is walked without rebuilding each path from the root, however deep it is.Stdin.lock()returns aReaderof standard input, andStdin.metadata()says whether it is a redirected file, and its size. - Metadata.
Fs.metadata(p)gives a file’s kind, its size, its permissions (md.mode,0o644), the number of its names (md.links) and its last access and modification times, asTimestamps (md.accessed,md.modified).Fs.symlinkMetadata(p)describes a symbolic link instead of following it,isSymlink(), andf.metadata()the open file, even if renamed since.Fs.setMode(p, 0o600)andFs.setTimes(p, accessed: a, modified: m)change them,Fs.remove(p)removes a name. - Creation and errors.
File.create(p)creates a file or truncates it;File.createNew(p)creates a file that must not exist, and otherwise fails withAlreadyExistswithout touching it, the check and the creation happening atomically.e.reason()gives the system’s text, that of strerror in the C locale,No such file or directory.Stdin.isTerminal()says whether someone is typing;Stderr.lock()writes to the error output without a newline, andout.flush()flushes the standard output’s buffer. A disposed-ofReadFile.stdin()does not close standard input, which can be read again. - Current directory, renaming, temporary directories.
Fs.currentDir()returns the current directory, absolute;Fs.rename(from, to)renames a file within the same file system and replacestoatomically, so that whoever opens it sees it whole or not at all;Fs.createTempDir("build-")creates an empty directory with a unique name, readable only by its owner, in the temporary files directory or indir:, and nothing deletes it.Process.executable()returns the path of the running executable. ReadFileandFile.File.openreturns aReadFile, opened read-only: closing it cannot lose anything, itsdisposeis! Never.File.createandFile.anonymous(in: dir)return aFile, opened for reading and writing, whose closing can lose written data: itsdisposeis! IoError. An anonymous file has no name; the system deletes it when it is closed, andrewind()reads it again from the start.Env.tempDir()gives the temporary files directory,$TMPDIRor/tmp. Ausingon a file being read, in a loop, therefore does not force the function to acceptIoError.
7.4 Arena
Section titled “7.4 Arena”An arena block redirects all the heap allocations it contains to a region freed at once, in O(1). It is also a disposal scope: resources opened inside are disposed of before the memory.
fn handle(req: Request): Response ! HttpError { arena { val body = Json.decode<Command>(req.body)! // allocated in the arena using tx = db.begin()! val result = execute(body, &tx)! tx.commit()! return render(result).detach() // explicit copy out of the arena }}- No implicit escape. A value allocated in the arena leaves the block only through
.detach(), which copies it to the ordinary heap, its heap blocks shared rather than copied. The cost stays visible..detach()copies neither a function, nor a shared object, nor a resource, whose identity matters. - Dynamic. The arena is the task’s, for the duration of the block: the functions called allocate in it too, without knowing it. Freeing a block of the arena does nothing; its memory comes back on exit, all at once.
- What the compiler rejects. What would literally leave the block: a value returned by
return, or the block’s value, that holds a heap block without being a.detach(), a literal or a read of an outside variable; the modification of a variable declared outside the block, which would store some of its memory there; spawning a task, which would outlive the block. - What the run time rejects. An arena value passed to a shared object,
Mutex,Shared, a channel, stops the program. An error that leaves the block through!orfailis copied automatically. - Writes outside.
Mutex.with,Shared.write, thewithof athread varand the first computation of aLazysuspend the arena for the duration of their call: what they store outside the block goes on the heap. - Graphs.
Arena<T>(6.4) is a different thing: a table of nodes addressed by handles, long-lived, without anarenablock. - Integration. An HTTP framework can open an arena per request; application code then does not write
arena. The arena wins when the block allocates many small values and keeps few: where all the memory stays alive until the end, counting goes through it anyway, and the arena reuses nothing.
7.5 Order on block exit
Section titled “7.5 Order on block exit”- The
using,deferanderrdeferrun in reverse declaration order. Theerrdefers are skipped if the block succeeds. - Disposal errors are combined according to 7.3.
- If the block is an
arena, its memory is freed last.