# @sdxc/spec An executable specification runner. You describe behavior in `.spec` files — setup, action, expectation — and `spec` runs each one against your real app, in an isolated workspace, under permissions you grant explicitly. A suite is a directory of `.spec` files (conventionally `spec/`), each written in a deliberately tiny language — no `if`, no loops, no operators. Every test runs in its own fresh temporary directory, and every privileged act needs an explicit `--allow-*` grant. ## Installation ```bash npm add -D @sdxc/spec ``` The `spec` command runs on [Bun](https://bun.sh), so keep `bun` on your `PATH`. Then invoke the CLI through your package runner: ```bash npx spec run spec ``` `bunx @sdxc/spec run spec` runs it without installing first. The `browser` capability additionally needs the `agent-browser` CLI installed globally; every other capability is self-contained. ## Usage ### Write a suite Create `spec/greeting.spec`: ``` use fs use cli test "the script prints its greeting" { given { write "index.js" "console.log(\"hello from the workspace\")" } when { let result = run "node" "index.js" } then { expect result.exit_code 0 expect result.stdout "hello from the workspace\n" } } test "the generated config is on disk" { given { write "package.json" { name: "demo", type: "module" } } then { expect file "package.json" contains "\"type\": \"module\"" } } ``` ### Run it `write` creates files in the test's workspace and `run` starts processes there. Spawning `node` is a privileged act, so grant exactly that one executable: ```bash npx spec run spec --allow-run=node ``` ``` seed spec (replay with --seed=spec) run mtw62dtrkgga (replay with --run-id=mtw62dtrkgga) ✓ the script prints its greeting ✓ the generated config is on disk 2 passed, 0 failed (21ms) ``` The header comes before the first result, so a failure is already anchored to the inputs that produced it: both flags re-run this exact run, and a suite with configured bases prints each resolved address underneath them. `spec run` takes a directory (default `./spec`) and scans it recursively for `.spec` files. ### Read a denial Drop the grant and the process-spawning test fails before `node` is ever launched. Tests denied for the same missing grant collapse into one block naming the flag that would unblock them: ``` ✗ Permission denied: run (1 test) The spec attempted to reach: > cli.run Re-run with an appropriate permission, for example: > spec run --allow-run Affected tests: - the script prints its greeting (spec/greeting.spec:9) ``` Start with no flags and let the denials tell you the suite's true footprint. ## The language ### Tests and phases A `test` has up to three phase blocks, always in this order: `given` (arrange), `when` (act), `then` (assert). Each is optional. Lines end statements, and `#` starts a comment to end of line (a `#` inside a string is just text). ``` test "a write is read back verbatim" { given { write "notes.txt" "remember the milk" # arrange } when { let content = read "notes.txt" # act, capture a value } then { expect content "remember the milk" # assert } } ``` `skip` prefixes `test` and takes an optional reason, which the summary prints. The body never executes, and the test is counted apart from the passes and the failures: ``` skip "the staging database is down" test "the nightly report reconciles" { then { expect report.balance 0 } } ``` ### `expect` `expect` has three forms: ``` expect content "remember the milk" # two values: deep structural equality expect true # one value: it must be true expect file "notes.txt" exists # observable: assert straight from a capability ``` The observable form reads the world through a capability (`file … exists`, `file … contains`, `directory … exists`, `browser.heading …`) and passes when that observation holds. `contains` and `not` join either form. `contains` is a substring over a string and membership over an array, comparing members with the same structural equality the two-value form uses. `not` is valid as `expect`'s first argument and inverts whichever form follows it. ``` then { expect page contains "og:title" expect rows contains { id: 2, title: "Emma" } expect not page contains "twitter:card" expect not browser.cookie "session" exists } ``` A negative assertion holds when the thing is absent and equally when the page never rendered, so anchor it with a positive expectation in the same block. An observable denied a permission still fails the statement under `not`, so a missing grant is never read as the thing being absent. ### `let` and references `let` binds the result of a step, and a dotted reference reaches into a returned object: ``` when { let result = run "node" "build.js" } then { expect result.exit_code 0 expect result.stdout "built\n" } ``` A bare word in tool-argument position resolves against the tool's own declaration. It is a symbol when the tool declares a word of that spelling, or when the required parameter at that position is a word — which is what makes `expect file "notes.txt" exists` read `exists` as `fs.file`'s assertion word. Otherwise it reads the binding of that name, so `browser.open profile` hands the tool the URL `profile` holds. A spelling that is both a declared word and a live binding is an `ambiguous-name` error: rename the binding, or pass it as a dotted reference. A dotted path whose head names a binding is a reference. When the head names something callable instead — a tool that takes no arguments, or a command that takes no parameters — the path is that call, so `let current = browser.url` binds the tool's observed value and `let user = create_testing_account` binds what the command produced. Both are permission-gated exactly like a written call. A bound head is always a reference and an unbound one is always a call, so the two readings never overlap and the rule holds in every expression position — a right-hand side, an object entry, an array item, an argument. A tool's value therefore reaches the place it belongs with no binding in between: ``` when { let slug = format "${name}-${nonce}" { name: who.username, nonce: spec.nonce } } ``` Arrays are values like any other, writable as a literal, and a path segment spelled as digits indexes one **0-based**: ``` given { let titles = [ "Dune", "Emma" ] } then { expect titles.0 "Dune" expect titles contains "Emma" } ``` The addressing vocabulary's ordinals count from 1 instead (`nth 2`, `row 1 column 2`), and the two sit side by side in a real spec. ### `eventually` Wrap an observable assertion in `eventually` to retry it until it holds or the window ends — for anything that becomes true a moment later, such as a server coming up or an async write landing. A plain assertion checks exactly once. ``` then { eventually within 2s { expect file "ready.txt" exists } } ``` Only assertions may be retried; an action inside `eventually` is an error. `eventually` stands in a `then` block, in a command body, and in a `setup` or `teardown` hook — everywhere a body may wait for an effect to land. A command that waits for the effect it just caused is what lets a sequence of calls stand in for the loop the language does not have. ### `command` A command is a reusable step: it takes parameters, performs effects, and produces whatever it `return`s. Define one beside the test that uses it, or share it suite-wide by putting it under `spec/commands/`. Either way it resolves by name from anywhere — no import, no path — because every definition is registered before any test runs. ``` # spec/commands/catalog.spec use fs command book { return { title: "Dune", author: "Herbert", year: 1965 } } command seed_file(path) { write path "seeded" } # spec/tour.spec test "commands compose by name" { given { seed_file "out.txt" # run a command for its effect } when { let record = book # one with no parameters is called by its bare name } then { expect record.title "Dune" expect file "out.txt" contains "seeded" } } ``` A command runs on every call — no memoization, no lifecycle — so a value wanted once is bound once. Reusable data is a command that only returns, which is the whole of it: one definition form covers arrangement, effects, and data alike. ### `setup` and `teardown` A suite may declare one `setup` and one `teardown`, in whichever file suits them; a second of either kind is a load error naming both files. `setup` runs once before any test whatever the concurrency, in its own workspace and under the same grants, and its failure is fatal — the run reports a load error, exits `2`, and no test executes. `teardown` runs once after every test, failures included, which is where a suite removes the rows it left behind. ``` use db use spec setup { db.run_file "db/schema.sql" on "web" } teardown { db.query "delete from accounts where run_id = $1" params spec.run_id on "web" } ``` A hook body is an ordinary block with a fresh, empty scope; `return` inside one is a usage error. ### `use` and namespaces Capabilities live in twelve namespaces: `fs`, `cli`, `http`, `browser`, `html`, `db`, `url`, `jwt`, `env`, `sample`, `str`, and `spec`. Use a fully qualified name anywhere (`fs.write`, `cli.run`), or `use` a namespace at the top of a file to call its tools by their bare names (`write`, `run`). `use` applies per file, and a command or hook body resolves bare names against the imports of the file that declared it rather than the caller's. An ambiguous bare name is an error naming both candidates. ## Capabilities `fs`, `html`, `url`, `sample`, `str`, and `spec` need no grant; the rest are privileged and denied until you grant them. ### `fs` — the workspace filesystem Every path resolves inside the test's workspace. Tools: `write`, `read`, `mkdir`, `copy`, `remove`, and the observables `exists`, `file`, `directory`. Strings are written verbatim; objects and arrays are written as JSON. ``` use fs test "mkdir, copy, and remove move files around the workspace" { given { mkdir "src" write "src/index.ts" "export const answer = 42" } when { copy "src/index.ts" "dist/index.ts" remove "src/index.ts" } then { expect directory "src" exists expect file "dist/index.ts" contains "answer = 42" } } ``` ### `cli` — run processes · `--allow-run` `run` spawns a program in the workspace and returns `{ stdout, stderr, exit_code }`. The grant is scoped by executable basename. The child gets a minimal environment (`PATH`, `HOME`, `TMPDIR`, plus only the variables granted with `--allow-env`), so your host environment stays out of it. ``` use cli test "run captures stdout and the exit code" { when { let result = run "echo" "hello" } then { expect result.exit_code 0 expect result.stdout "hello\n" } } ``` ```bash npx spec run spec --allow-run=echo ``` ### `http` — call an HTTP API · `--allow-net` `get`, `post`, `put`, `patch`, and `delete`, each needing `--allow-net` for the URL's host (and port, if you scope one). A target is an absolute URL, or a `/path` resolved against a configured base. An optional bare body travels as `text/plain` when it is a string and as JSON otherwise. Each returns `{ status, ok, headers, text, json }`; an HTTP error status is a normal value, and only a network-level failure is an error. ``` use http test "creating a post returns 201" { when { let response = http.post "http://localhost:3000/api/posts" { title: "Hello" } } then { expect response.status 201 } } ``` ```bash npx spec run spec --allow-net=localhost:3000 ``` After the URL, a request takes word-tagged options in any order: - `headers { Name: "value", … }` — request headers. Numbers and booleans stringify, header names are case-insensitive, and an explicit `content-type` here overrides the body's. - `form { field: "value", … }` — a body sent as `application/x-www-form-urlencoded`. - `json ` — a body sent as `application/json`. - `text ""` — a body sent as `text/plain`. - `bearer ` — sets `Authorization: Bearer `. - `basic ` — sets `Authorization: Basic base64(user:pass)`. It is the one option that takes two values. The options combine, so an authenticated form post is one call: ``` when { let response = http.post "http://localhost:3000/oauth/token" form { grant_type: "authorization_code" code: "bogus" } basic "the-client-id" "the-secret" } ``` A request carries at most one body (the bare body, or one of `json`, `form`, `text`), at most one `headers` block, and at most one auth option. A second body, a body on a `GET`, both `bearer` and `basic`, or an unknown tag is an error, and an explicit `headers.authorization` wins over `bearer` and `basic`. A response body is read whole, which is what the CLI wants. An embedder running untrusted specs in a bounded runtime caps it instead — see [`createHttpPlugin`](#createhttpplugin-options). ### Named bases — where a relative target points `http`, `browser`, and `browser.fetch` address a running app by path, and `spec/config.jsonc` decides what those paths hang off. Each base reads an environment variable, falling back to a default, so one suite runs against a laptop, staging, and CI unchanged: ```jsonc // spec/config.jsonc { "bases": { "web": { "env": "FRONTEND_BASE_URL", "default": "http://localhost:4000" }, "work": { "env": "WORK_FRONTEND_BASE_URL", "default": "http://localhost:4020" }, }, } ``` ``` browser.open "/charities" browser.open "/dashboard" on "work" http.get "/portfolios" ``` Naming no base works when exactly one is configured; `on "name"` chooses otherwise, and naming none among several is an error listing what the config declares. A relative target starts with `/` — anything else is an error suggesting the slash, which keeps a statement from depending on the statement before it — and a target carrying a scheme, or `//host/path`, is absolute and ignores the base entirely. Substituting the environment into the config needs no grant: it is the runner's own settings, written by whoever runs the CLI, and could have held the literal URL. `env.get` inside a spec is a different act and keeps its grant. A base that resolves to nothing stops the run before any test, naming the variable to set. `--allow-net` keys on the **resolved** host, so a denial is copy-pasteable whatever the spec wrote: ``` ✗ Permission denied: net (1 test) The spec attempted to reach: > localhost:4020 Re-run with an appropriate permission, for example: > spec run --allow-net=localhost:4020 ``` The run header prints each resolved base, so a run says out loud which app it actually reached. ### `html` — read a served page's markup · no grant `html` addresses a string of HTML — typically the `text` of an `http` response — so a page's content, structure, and metadata are assertable with no browser, no sockets, and no grant. Every tool takes the source as its first argument. `title`, `meta`, and `rel` read what the document says about itself; `text`, `element`, `cell`, and `definition` read what a person would see. ``` use http use html test "the portfolios page describes itself" { when { let page = http.get "/portfolios" } then { expect page.status 200 expect html.title page.text "Invest your money for more charitable impact" expect html.meta page.text "description" "Give more, and give it sooner" expect html.rel page.text "canonical" "https://example.com/portfolios" expect html.meta page.text "og:image" containing "/build/_assets/og-image-" } } ``` `meta` matches a tag's `name` or its `property` and `rel` matches one token of a ``'s `rel`, which is why the tag is an argument rather than a field of a parsed object: `og:image` could never be a segment of a dotted path. A requested tag the document does not carry is an error naming the ones it does — `Present: description` — and `exists` is how a spec says the absence is the point. `html` and `browser` address elements under identical rules, so a document is addressed one way and the namespace choice is only about whether a live browser is needed. Both reach every role through `element`, and both carry `heading`, `link`, `button`, and `checkbox` as shorthands for the roles a suite reaches for most, so an assertion moved from one namespace to the other keeps its meaning — which is why a `` is read by `html.rel`, leaving the name `link` to the role it means on either side. `html` answers nothing that depends on layout: no viewport, no scrolling, no `in_viewport`. #### The addressing vocabulary Elements are addressed the way a person perceives them: by ARIA role (a bare word) and accessible name (a string). A control's label is its accessible name, so addressing by label needs no feature of its own. ``` then { expect html.element page.text heading "Portfolios" expect html.element page.text textbox "Tip" value "20" expect html.element page.text button "Save" enabled expect html.element page.text button "Save" attribute "type" "submit" } ``` - **Names match exactly**, on the normalized accessible name; `containing` opts into a substring. `text` is the mirror image — a substring assertion over the visible text, with `exactly` opting into equality. "Is there an element named X" and "does this string appear" are different questions, so they carry opposite defaults. - **Ambiguity is an error** listing every match with its position, because a nav link repeated in the footer would otherwise assert on whichever came first with no signal that a choice was made. `first`, `nth `, and `last` opt into one, counting from 1 — deliberately distinct from a 0-based array index. - **`field "name"`** resolves anything carrying a `name` attribute — input, textarea, select, button — and sits where a role word sits. `value "annual"` narrows a group sharing one name, which is how a radio group and a submit-intent button are addressed — in an action as much as in an assertion, so `browser.fill field "cadence" value "annual" with "12"` writes into the one member it names. - **Predicates** are `count `, `value `, `attribute `, `enabled` / `disabled`, and `in_viewport` (a browser only). `count` is the one set predicate and accepts any number of matches; the rest require exactly one. `attribute` reads the raw markup rather than the resolved property, and `value` compares as a string. - **`cell row 1 column 2`** is 1-based over body rows, with `including header` to count them, and **`definition "Total"`** reads the `definition` paired with a `term`. - **Roles are roles, not tags.** A `