Test Environments

Tests run on machines that differ in user, kernel, paths, network and load: these are the rules that keep a test correct on every one. Each comes from a test that passed on one machine and failed on another. A test or a fix that turns out to depend on the machine is fixed once, at its cause, and its rule is added here in the same pull request.

Check the selected environment

An environment has two axes: host capabilities and installed tooling. A test is eligible when both satisfy its requirements, as described by the workspace's build guide, its CI setup and the conditions below. Select tests for the available environment, then verify the operations they need. A pure unit test need not wait for unrelated infrastructure.

Host capabilities

The host determines the OS, architecture, kernel features, privileges, outer namespace and filesystem restrictions. Bubblewrap needs the host to permit its namespaces, and Landlock needs the relevant kernel operations. Root-only tests need their prescribed root environment. Installing a program does not grant any of those capabilities. Fixtures that fall back to --no-sandbox keep their ordinary launch assertions and gate checks for refused grants on the observed sandbox capability.

Proxy and Direct Connections

An HTTPS request through a configured HTTP proxy does not establish that the machine can resolve public names or open direct connections. Raw TCP and SOCKS checks need those operations independently. Keep the configured proxy and network policy intact, and report a missing route or resolver as an environment exclusion.

Installed tooling

Available executables, libraries and their versions determine which tools tests can use. Build the workspace programs the selection needs. Probe relevant installed tools: a database test needs working server and cluster-initialization programs, and an SSH test needs a client that accepts its configuration. Install a host package when it is needed for the component under test, rather than merely to run more tests.

Node's test runner

Node's --test-isolation option arrived in Node 24, and an older Node refuses it as a bad option before running anything. A test that runs Node with it probes the option itself, as the run passes it, and skips through its gate where Node lacks it.

Browser startup

An installed browser also needs a working headless sandbox. Probe a fresh profile independently of the Link tool before treating that capability as available; a version response or existing executable is insufficient.

Fixture state

Fixture homes, temporary folders and state locations must be writable and satisfy ownership and trust checks under the host's restrictions. Host settings and a read-only real home must not leak into fixtures. These are inputs to isolate, not reasons to install unrelated packages or change the user's machine configuration.

Missing prerequisites

Use an existing suitable environment or report the affected selection skipped or not run, with its reason. Unsupported tests need not run on every machine. Provision another environment only when the component under test needs it. CI requirements describe what its chosen profile must cover; they are not a demand to install every tool on each machine. Preserve that contract when claiming equivalent coverage.

Harness Program Allowlists

A check that an installed harness can read its own program while refusing the containing directory requires Landlock. Gate the confinement assertion before running it, because an unrestricted fallback can run the program but cannot prove the directory refusal.

Stop at an explicit skip

A gate reporting missing Landlock or another prerequisite completes the check for this host, as a skip. Never create a VM or change the host just to turn skips into passes, and fix assertion failures separately.

Required capabilities

A gate skips its test when the machine lacks what it needs, saying what and why, and a LNK_REQUIRE_* variable turns that skip into a failure. Set one only on an environment that provides every capability it names, as CI does, so that a gap there is a real regression. A session's or a person's machine leaves them unset, so a test it can't run is reported skipped, never failed. What each variable names is in Variables that require a capability.

A test that can't run here

A test that fails only because the machine lacks something needs a gate, not a change to the machine. What it lacks is on one axis: a host capability (OS, architecture, kernel feature, privilege), the host's configuration (its user, root, or where the checkout and temp folders are) or an installed program. Add the testing kit's gate for that axis, or a suite's own through gate::lacks with the variable that requires it, and say so on this page in the same pull request. When a skip is right, how a gate makes one and every variable that decides it: Skipping Tests.

Results and coverage

Report passed, failed and skipped from observed outcomes. Failures keep their original status when an environment error caused them. Ignored tests and explicit capability skips are not coverage, and a suite not run has no individual result counts. A capability gate can return early and still appear as passed to libtest, so unobserved gate output leaves coverage uncertain. Report that uncertainty rather than inventing counts.

Record a failure that depends on the machine

When a test passes on one machine and fails on another, record both before changing anything, then fix the cause:

  1. Record the machine: its OS and kernel, the user the tests ran as, whether bubblewrap and Landlock work, the checkout's path, TMPDIR and how many CPUs the build had.
  2. Record the run: the package and test, alone or in a full cargo test --workspace, and how many runs out of how many failed. To count, run the whole test program several times in a row on one build, not the one test.
  3. Reproduce the input that differs on a machine that passed, and fix the boundary that owns it, never the test's expectation. Never weaken a requirement or count a test that couldn't run as a pass.
  4. Add what differs to the section below, as a rule that keeps code and tests correct on every machine. Record only what a later session could run into again.

Inputs that differ between machines

Each rule below names an input that differs between machines, and how code and tests stay correct whatever its value.

Unix identity

Who the tests run as, and who else the machine has.

  • A test sets the child environment it needs, including HOME, USER, LOGNAME, XDG variables and PATH. The kit sets XDG_RUNTIME_DIR to the .run folder of the child’s fixture home (env::runtime), keeping runtime sockets out of the real user’s runtime folder and out of any folder a test grants in that home. One machine sets USER and another doesn't.
  • Root reads every file whatever its mode, so a permission a test checks as root proves nothing for an ordinary user. A root test checks as nobody, or as a user of its own, with setpriv.
  • A step run as another user must reach every folder above its own. A TempDir is made 0700, and a test that opens its own folder for that user opens each parent too, or works under /tmp, which lets anyone through.
  • GitHub's runners run as an ordinary user. A change to code that only root tests cover is run as root before it merges.
  • The system state parent-mode fixture needs root and trusted fixture ancestors. It checks directory modes inside a kit-owned temporary tree, creates no real service user and never changes /var/lib/lnk. Ordinary-user runs skip it through the root gate.
  • A system user a root test makes is the machine's, seen by every test running at once: each test names its users after itself, never a name another test could make, and removes them at the end, whatever happened.
  • Root inside a container may create network namespaces yet lack the sysfs mount that entering one requires. A suite that enters network namespaces probes the mount in a child mount/network namespace before changing the machine. Missing permission is an explicit capability skip through gate::lacks, required by LNK_REQUIRE_SANDBOX.
  • A concurrent fork can retain a socket until its child executes, even with close-on-exec set. After closing a listener, use the kit's bounded readiness wait for rebinding rather than assuming every inherited descriptor closed with the parent's.

Folders and paths

Where a test's files are, and what those places allow.

  • Keep checkouts outside /tmp, which sandbox specifications cannot read. A test that needs the sandbox to read its folder makes it outside /tmp, and out_of_tmp says when it can't.
  • Trusted configuration requires trusted ancestor ownership and modes. Agent state-export and team lifecycle CLI cases probe both spec and machine-settings readers and report refusals through gate::lacks. A namespace's untrusted root cannot be repaired by changing the fixture directory.
  • A Unix socket's path has a limit, 104 bytes on macOS and 108 on Linux. A Mac's TMPDIR and a long worktree's CARGO_TARGET_TMPDIR can pass it, so a socket goes under TempDir::short, which uses /tmp on Unix. Link's own runtime folder is in /tmp on a Mac for the same reason (Runtime Locations).
  • A Mac's temp folder is reached through a link, as /var is /private/var. Code that resolves a path's links compares the result only with paths resolved the same way, or a name made by removing a prefix keeps the whole path.
  • Give a short socket folder a brief label. The kit adds uniqueness; repeating resource IDs or counters can exceed its path budget as process numbers grow.
  • Check short fixture paths against the kit's path budget, never a particular process ID's digit count. Compact suite state uses counters instead of labels, so even a long label can produce a short path there.
  • An SSH home needs room for its control directory, hash and temporary suffix. Use TempDir::short_with_limit for that smaller total budget, so a longer suite-state path cannot turn connection sharing off.
  • A socket fixture whose paths must stay outside the sandbox's /tmp uses TempDir::short_out_of_tmp: a short LNK_TEST_STATE_DIR when set, else this user's cache folder. It checks that place; moving the fixture into the replaced /tmp would weaken its assertions.
  • A sandbox grants a fixture's folders, never the folder holding them. Under /tmp every parent is readable, so a program that opens a granted folder's parent passes there and is refused anywhere else. It opens the granted folder by its own path instead.
  • Filesystems can differ: a rename across mounts returns EXDEV.
  • A test's folder may be inside a checkout (CARGO_TARGET_TMPDIR, a checkout's target/), so a test of what's found up to a repository's root makes its own .git in its folder.
  • A fixture for a search outside any repository checks that none of its ancestors holds .git, even when made under /tmp. A command runner can put repository markers there too. If that prerequisite is absent, the test reports it through the kit's gate.

The account's own home

No test, and nothing a test runs, reads or writes the real ~/.lnk or ~/Link, or puts a socket in the machine-wide runtime folder: each test program starts in a home the testing kit makes for it (A home for each test program), and each child in its own (env::command). A test that reads HOME reads the kit's. One that needs the account's own home asks home::account().

Programs and processes

The programs a test writes and runs, and the processes it starts.

  • Relay installation fixtures need the host's install program. They probe its portable directory operation in an owned fixture through gate::program, so missing tooling reports a skip on either platform.
  • A program a test writes is written with write_program or copy_program, never std::fs::write or std::fs::copy before running it. Otherwise a child another test forks meanwhile can hold it open, and running it fails with "Text file busy", only when the machine is busy (why).
  • Modules are found at run time by file name, lnk-<module>. Renaming one breaks every caller that builds the name as a string, with no compile error, so search for the old name's strings.
  • An SSH fake keeps control-master state per destination, for checks and exits as well as opens. Short test paths can enable multiplexing: one host's connection must never make another host appear connected.
  • A fake that answers a question reads the question first, as the program it stands in for does. One that answers and exits without reading leaves the asker's write to a closed pipe, a failure only when the fake happens to end first.
  • A Mac has no setsid program, and its python3 can be older than 3.11, which brings tomllib. A fake writes the small program it needs on its own PATH, and a test that needs tomllib asks gate::program to import it, so an older Python skips the test.
  • A child needing its own session uses ChildGroup::spawn_session. Creating a process group first makes the child its leader, which prevents setsid from detaching it from the runner's terminal.
  • A Mac's /bin/sh keeps a variable assigned before a function's call after the call, where other shells drop it. A script that means the variable for one call unsets it afterwards.
  • A Mac's kernel differs at the edges of the system calls. A signal from kill has the si_code 0, not SI_USER. A socket from accept inherits its listener's non-blocking mode, so a server that polls its listener sets each accepted socket back to blocking before reading.
  • Rustup picks a toolchain by the folder a program runs in and by RUSTUP_TOOLCHAIN, which Cargo sets for every test. A program a test runs in a sandbox runs elsewhere and without it, so it can get the machine's default toolchain, and a probe for it runs as it will.
  • A language-server fixture selects the host toolchain before entering its sandbox and puts that sysroot's programs first on its own PATH. Rustup's cache variables need not reach the sandbox, so its proxies cannot choose the server's compiler there.
  • Few machines have the Java and Kotlin language servers, jdtls and kotlin-language-server, so their fixtures skip without them even where LNK_REQUIRE_PROGRAMS is set. Put each on PATH to run them.
  • A process inherits the signals its parent ignores. A run started under nohup ignores SIGHUP, and one started in the background by a script ignores SIGINT and SIGQUIT, so a test that signals a child it starts first sets those signals back to their defaults for the child.
  • A check that combines several conditions reports one cause for all of them. "bubblewrap can't run here" can mean that a program is missing, so test each condition alone before believing the message.
  • Process numbers are reused, and a busy machine has thousands. A fake that reports a process, such as a fake systemctl's MainPID, numbers it past the most a process can be (2^22), or code that waits for it to end waits on whatever of the machine's has that number.
  • A process reported by ps can already be a zombie: it has exited, waiting for its parent to reap it. Cleanup waits for live states and has a deadline. A container whose main process does not reap adopted children can retain zombies until it ends. Signal zero counts those too; process::left_under checks whether any owned process is still running.
  • process::left_under exists only on Linux. macOS shows a process's environment to no other process, even one of the same user, so a check there would miss every process named only by its HOME and pass whatever a stop left behind.
  • Tests run the programs in target/debug, and a build in the same checkout replaces them while a test runs one. Each concurrent task uses a worktree of its own.

Settings and network

What a child reads of the machine's settings, and what it can reach.

Fixture rebuilds

A test never downloads: the VM fixtures' committed manifests pin what a rebuild made, and their tests read only the checkout. Rebuilding a fixture fetches its recipe's archives from deb.debian.org, so a change to a fixture's recipe or guest source is rebuilt on a machine whose network reaches that host, and a manifest is never edited by hand.

JSON fixtures

Cargo feature unification can change JSON object key order in a full workspace build. Protocol fixtures accept keys in any order; a single package's serialization order is never part of the wire contract.

Child settings

Each child gets only the settings and network inputs its test owns.

  • A child given no HOME uses the real user's settings in ~/.lnk, shared with the machine and every other test. A test clears the child's environment (env_clear) and gives it a HOME of its own.
  • A bridge that a proxy starts gets a clean environment. In a debug build, it also gets the proxy's LNK_TEST_* variables, and nothing else of a test's, such as SSL_CERT_FILE.
  • Claude Code on the web reaches the internet only through an HTTPS proxy, so a test gives every name it reaches an address with LNK_TEST_RESOLVE and answers it itself (Write a test).

A browser on the web

A check that opens a web page in the agent's browser needs a Chromium that opens one through the machine's network. On Claude Code on the web it waits until it's stopped, even outside Link, while curl gets through the same proxy, so such a check skips there.

A launcher around the browser

A launcher the browser tool doesn't resolve to its engine, such as a container's wrapper around Playwright's headless shell, runs inside the sandbox, which refuses the programs it calls, so the check skips there too unless the spec's [tool.browser] program names the engine. The browser tool's own tests skip a launcher script the same way, unless $CHROMIUM names the engine, such as the wrapper's chrome-headless-shell.real.

Kernel confinement

What the kernel lets the sandbox do, which differs between machines.

  • A command runner can add its own user namespace around a test. Link refuses unsandboxed agents there because it cannot trust the machine's identity. Probe the UID map and bubblewrap in the selected execution mode. Use the runner's supported permission mechanism when its outer sandbox blocks the host capabilities the test needs.
  • bubblewrap needs unprivileged user namespaces. Ubuntu's AppArmor withholds them until kernel.apparmor_restrict_unprivileged_userns=0, which CI sets. With LNK_REQUIRE_SANDBOX=1, as in CI, a test that can't run it fails rather than skips.
  • Landlock depends on the host kernel and on what the container's runtime lets through. Without it, executable allowlist tests skip, saying why (Development Environment).
  • The Linux release links musl, whose libc differs from glibc where the sandbox leans on it (O_PATH). The sandbox's unit tests run on musl too.
  • A wrapped agent sandbox needs Landlock and bubblewrap. Use gate::sandbox_with_landlock. A namespace probe alone does not prove the wrapped program can start. QA uses native-sandbox for that combination on Linux and sandbox-exec on macOS. A test of refused reads uses that gate, since an unconfined fallback cannot prove refusal.
  • A sandbox check test uses the same gate because its probes enforce executable restrictions as well as file and network permissions.
  • A filesystem-only confinement test explicitly permits all commands. That selection proves file grants without requiring Landlock's executable allowlist. Tests of executable restrictions keep the Landlock gate and report its absence as a capability skip.

Load and timing

What changes when the machine is busy, or many tests run at once.

  • cargo test --workspace runs many tests at once, and a test that passes alone can fail there. Reproduce it by running its whole test program several times in a row, not the one test.
  • A failure that comes only under load is usually a race in the code under test, such as an answer dropped when the other side closes early. Fix the race. Never add retries, a longer timeout alone or one test thread to hide it.
  • A test waits through the testing kit, which polls with backoff and fails naming what it waited for and what it last saw. A machine slower than most sets LNK_TEST_TIME_SCALE (such as 3) once, and every deadline grows with it.
  • A 4-CPU machine is slower than most for the harness tests: Link Harness's test program starts several confined agents, each launching dozens of debug programs in turn. Its runtime tests take a permit for each, so at most two start at once there and their first connection comes within the unscaled 30 seconds. Its other tests take none yet, so such a machine still sets LNK_TEST_TIME_SCALE=3.
  • A test that needs one step done before the next starts waits until the first is seen (the stand-in asked, its file written), never for a fixed pause: on a loaded machine a task spawned first can run second. A PID file publishes its complete, newline-terminated number, not its existence: creation precedes the write.
  • A cancellation test observes the client's displayed output before interrupting it. A producer yielding a fragment does not establish that its consumer has written it. Use io::ObservedOutput with the kit's wait, so load cannot turn cancellation into an empty response.
  • Connection typing indicators run on the bridge child's own clock. Check the initial active indicator, any refreshes, one stop and one answer in order. A parent's paused clock does not pause that child.
  • A fake clock compared with a store's real timestamp starts from the observed stored value. Sampling wall time before asynchronous work can leave the fake clock behind it. Advance from that timestamp to test an expiry boundary, rather than shortening the configured age to zero.
  • In-process upload timers use a paused clock, with no network work on that clock. Admission and release are observed by polling the body and its permits, so progress depends on ownership rather than a pause.
  • A controller fixture that tests journals or cleanup supplies its own machine facts. Real host qualification belongs to the probe's tests; unrelated kernel and account probes must not consume fixture readiness.
  • A deadline of a second or so on a command also times that command's start, which a loaded Mac can pass. A test of a timeout makes only the step it times hang, and gives every other step the time any machine needs.
  • State the code under test keeps for a whole process, such as a static set of chats being worked on, is shared by every test of the program: one test's entry can make another's work skip. Such state belongs to the object that serves, a harness say, so each test's is its own.
  • A test that probes a port free and binds it later can lose it to another process under load ("Address already in use"): a server is given a listener the test already holds. Ports a program the test starts binds itself go to one test of the program only, and never among the ports the code's own lookup gives out, which every test's home starts from.
  • A number a program the test starts takes itself, such as an X display or an agent's block of ports, is held before it's probed free: another test program, or another run of the same one, can probe the same number before either program takes it. The kit's HeldNumber locks a file of the number's own with flock, which the kernel lets go however the test ends.
  • A single port such a program binds, an sshd's or a harness's, comes from HeldNumber::port(), 12000 to 15999: below the system's ports and Link's own.
  • A check that something was refused waits for the refusal itself, never a deadline: a busy machine can answer later than any limit, and a silence read as "let through" is then a leak that isn't. A limit may only shorten the wait for what's let through.
  • A test that checks an action left something alone compares only what that action could touch, such as the processes the code under test started and recorded. A user's other processes, a program's helpers and their children, start and end on their own under load.
  • A file another process appends to is read holding its lock, and only its whole lines: a read can end partway through a line still being written.
  • Each run of a test starts from files of its own. A file reused between two runs in one test can still hold the first run's lines when the second reads it.
  • A test that counts a stand-in provider's requests waits for the request it means, matched by content. A runtime can send several per turn (a title, a summary), and one sent just before the test kills the runtime can reach the stand-in after the kill, where it looks like the restarted runtime's.

Trusted Fixture Ancestors

Tests of protected settings need fixture files and all their ancestor directories to satisfy the production ownership checks. A container with an untrusted root, or fixtures under a writable /tmp, cannot supply that precondition. gate::settings and gate::trusted_file report explicit skips, required by LNK_REQUIRE_SANDBOX. These gates leave the production reader's path and ownership rules intact.

Box Identity Fixtures

Folder-backed environment and job tests that assert a box identity need a whole Linux UID map. Inside a user namespace the production reader deliberately identifies a computer, even with a fixture kind file. The environment fixture's box_identity gate reports this limitation before image, root-stage or disposable-box assertions run.

What local fakes do not establish

Local tests use stand-ins for the macOS Keychain, TPM, systemd and launchd, cloud accounts, paid models and payment providers. Passing a fake does not certify those systems: they are checked on real machines and accounts.