Test Environments

Host capabilities

The host determines the OS, architecture, kernel features, privileges, outer namespace and filesystem restrictions. Bubblewrap needs the host to permit its namespaces, and Landlock needs the relevant kernel operations. Root-only tests need their prescribed root environment. Installing a program does not grant any of those capabilities.

Installed tooling

Available executables, libraries and their versions determine which tools tests can use. Build the workspace programs the selection needs. Probe relevant installed tools: a database test needs working server and cluster-initialization programs, and an SSH test needs a client that accepts its configuration. Install a host package when it is needed for the component under test, rather than merely to run more tests.

Fixture state

Fixture homes, temporary folders and state locations must be writable and satisfy ownership and trust checks under the host's restrictions. Host settings and a read-only real home must not leak into fixtures. These are inputs to isolate, not reasons to install unrelated packages or change the user's machine configuration.

Missing prerequisites

Use an existing suitable environment or report the affected selection skipped or not run, with its reason. Unsupported tests need not run on every machine. Provision another environment only when the component under test needs it. CI requirements describe what its chosen profile must cover; they are not a demand to install every tool on each local machine. Preserve that contract when claiming equivalent coverage.

Required capabilities

A gate skips its test when the machine lacks what it needs, saying what and why, and a LNK_REQUIRE_* variable turns that skip into a failure. Set one only on an environment that provides every capability it names, as CI does, so that a gap there is a real regression. A session's or a person's machine leaves them unset, so a test it can't run is reported skipped, never failed.

• LNK_REQUIRE_SANDBOX: bubblewrap, sshd, a sandbox that the built lnk-sandbox runs, and a checkout outside the temp folders.
• LNK_REQUIRE_NATIVE: the kernel's Landlock.
• LNK_REQUIRE_PROGRAMS: the host programs tests run, such as python3, git, curl, ssh, zsh, Xvfb, Chromium and rust-analyzer.

A test that can't run here

A test that fails only because the machine lacks something needs a gate, not a change to the machine. What it lacks is on one axis: a host capability (OS, architecture, kernel feature, privilege), the host's configuration (its user, root, or where the checkout and temp folders are) or an installed program. Add the testing kit's gate for that axis, or a suite's own through gate::lacks with the variable that requires it, and say so on this page in the same pull request. When a skip is right, how a gate makes one and every variable that decides it: Skipping Tests.

Results and coverage

Report passed, failed and skipped from observed outcomes. Failures keep their original status when an environment error caused them. Ignored tests and explicit capability skips are not coverage, and a suite not run has no individual result counts. A capability gate can return early and still appear as passed to libtest, so unobserved gate output leaves coverage uncertain. Report that uncertainty rather than inventing counts.

Unix identity

Who the tests run as, and who else the machine has.

• A test sets the child environment it needs, including HOME, USER, LOGNAME, XDG variables and PATH. One machine sets USER and another doesn't.
• Root reads every file whatever its mode, so a permission a test checks as root proves nothing for an ordinary user. A root test checks as nobody, or as a user of its own, with setpriv.
• A step run as another user must reach every folder above its own. A TempDir is made 0700, and a test that opens its own folder for that user opens each parent too, or works under /tmp, which lets anyone through.
• GitHub's runners run as an ordinary user. A change to code that only root tests cover is run as root before it merges.
• A system user a root test makes is the machine's, seen by every test running at once: each test names its users after itself, never a name another test could make, and removes them at the end, whatever happened.

Folders and paths

Where a test's files are, and what those places allow.

• Keep checkouts outside /tmp, which sandbox specifications cannot read. A test that needs the sandbox to read its folder makes it outside /tmp, and out_of_tmp says when it can't.
• A Unix socket's path has a limit, 104 bytes on macOS and 108 on Linux. A Mac's TMPDIR and a long worktree's CARGO_TARGET_TMPDIR can pass it, so a socket goes under TempDir::short, which uses /tmp on Unix.
• A socket fixture whose paths must stay outside the sandbox's /tmp uses TempDir::short_out_of_tmp: a short LNK_TEST_STATE_DIR when set, else this user's cache folder. It checks that place; moving the fixture into the replaced /tmp would weaken its assertions.
• A sandbox grants a fixture's folders, never the folder holding them. Under /tmp every parent is readable, so a program that opens a granted folder's parent passes there and is refused anywhere else. It opens the granted folder by its own path instead.
• Filesystems can differ: a rename across mounts returns EXDEV.
• A test's folder may be inside a checkout (CARGO_TARGET_TMPDIR, a checkout's target/), so a test of what's found up to a repository's root makes its own .git in its folder.

Programs and processes

The programs a test writes and runs, and the processes it starts.

• A program a test writes is written with write_program or copy_program, never std::fs::write or std::fs::copy before running it. Otherwise a child another test forks meanwhile can hold it open, and running it fails with "Text file busy", only when the machine is busy (why).
• Plugins are found at run time by file name, lnk-<plugin>. Renaming one breaks every caller that builds the name as a string, with no compile error, so search for the old name's strings.
• An SSH fake keeps control-master state per destination, for checks and exits as well as opens. Short test paths can enable multiplexing: one host's connection must never make another host appear connected.
• A fake that answers a question reads the question first, as the program it stands in for does. One that answers and exits without reading leaves the asker's write to a closed pipe, a failure only when the fake happens to end first.
• Rustup picks a toolchain by the folder a program runs in and by RUSTUP_TOOLCHAIN, which Cargo sets for every test. A program a test runs in a sandbox runs elsewhere and without it, so it can get the machine's default toolchain, and a probe for it runs as it will.
• A process inherits the signals its parent ignores. A run started under nohup ignores SIGHUP, and one started in the background by a script ignores SIGINT and SIGQUIT, so a test that signals a child it starts first sets those signals back to their defaults for the child.
• A check that combines several conditions reports one cause for all of them. "bubblewrap can't run here" can mean that a program is missing, so test each condition alone before believing the message.
• Process numbers are reused, and a busy machine has thousands. A fake that reports a process, such as a fake systemctl's MainPID, numbers it past the most a process can be (2^22), or code that waits for it to end waits on whatever of the machine's has that number.
• A process reported by ps can already be a zombie: it has exited, waiting for its parent to reap it. Cleanup waits for live states and has a deadline. A container whose main process does not reap adopted children can retain zombies until it ends. Signal zero counts those too; process::left_under checks whether any owned process is still running.
• Tests run the programs in target/debug, and a build in the same checkout replaces them while a test runs one. Each concurrent task uses a worktree of its own.

Settings and network

What a child reads of the machine's settings, and what it can reach.

Kernel confinement

What the kernel lets the sandbox do, which differs between machines.

• bubblewrap needs unprivileged user namespaces. Ubuntu's AppArmor withholds them until kernel.apparmor_restrict_unprivileged_userns=0, which CI sets. With LNK_REQUIRE_SANDBOX=1, as in CI, a test that can't run it fails rather than skips.
• Landlock depends on the host kernel and on what the container's runtime lets through. Without it, executable allowlist tests skip, saying why (Agent Sessions).
• The Linux release links musl, whose libc differs from glibc where the sandbox leans on it (O_PATH). The sandbox's unit tests run on musl too.

Load and timing

What changes when the machine is busy, or many tests run at once.

• cargo test --workspace runs many tests at once, and a test that passes alone can fail there. Reproduce it by running its whole test program several times in a row, not the one test.
• A failure that comes only under load is usually a race in the code under test, such as an answer dropped when the other side closes early. Fix the race. Never add retries, a longer timeout alone or one test thread to hide it.
• A test waits through the testing kit, which polls with backoff and fails naming what it waited for and what it last saw. A machine slower than most sets LNK_TEST_TIME_SCALE (such as 3) once, and every deadline grows with it.
• A test that needs one step done before the next starts waits until the first is seen (the stand-in asked, its file written), never for a fixed pause: on a loaded machine a task spawned first can run second.
• State the code under test keeps for a whole process, such as a static set of chats being worked on, is shared by every test of the program: one test's entry can make another's work skip. Such state belongs to the object that serves, a harness say, so each test's is its own.
• A test that probes a port free and binds it later can lose it to another process under load ("Address already in use"): a server is given a listener the test already holds. Ports a program the test starts binds itself go to one test of the program only, and never among the ports the code's own lookup gives out, which every test's home starts from.
• A number a program the test starts takes itself, such as an X display or an agent's block of ports, is held before it's probed free: another test program, or another run of the same one, can probe the same number before either program takes it. The kit's HeldNumber locks a file of the number's own with flock, which the kernel lets go however the test ends.
• A single port such a program binds, an sshd's or a harness's, comes from HeldNumber::port(), 12000 to 15999: below the system's ports and Link's own.
• A check that something was refused waits for the refusal itself, never a deadline: a busy machine can answer later than any limit, and a silence read as "let through" is then a leak that isn't. A limit may only shorten the wait for what's let through.
• A test that checks an action left something alone compares only what that action could touch, such as the processes the code under test started and recorded. A user's other processes, a program's helpers and their children, start and end on their own under load.
• A file another process appends to is read holding its lock, and only its whole lines: a read can end partway through a line still being written.
• Each run of a test starts from files of its own. A file reused between two runs in one test can still hold the first run's lines when the second reads it.
• A test that counts a stand-in provider's requests waits for the request it means, matched by content. A runtime can send several per turn (a title, a summary), and one sent just before the test kills the runtime can reach the stand-in after the kill, where it looks like the restarted runtime's.