Test Troubleshooting

What a test run says when the machine, the checkout or a test itself is at fault, and how to tell which and fix it. Each symptom below has come up before, so a session can find its cause here rather than again.

A required capability is missing

A test panicked with LNK_REQUIRE_... is set, but it lacks .... The machine was declared to provide a capability it doesn't. Either provide it, or leave the variable unset so its tests skip, saying why (Skipping Tests).

lacks bubblewrap

The sandbox's tests need unprivileged user namespaces. Install bubblewrap, and on Ubuntu let AppArmor allow the namespaces with sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0. Check with bwrap --unshare-net --ro-bind / / true, the gate's own probe.

lacks Landlock

The kernel withholds Landlock, which no package installs: a container gets its host kernel's. Leave LNK_REQUIRE_NATIVE unset on that machine, or use one whose kernel has it, Linux 5.13 or newer.

lacks a test folder out of the temp folders

A fixture the sandbox reads is under /tmp, which the sandbox refuses a spec to read, since anyone can write there. Run from a checkout outside /tmp. A socket fixture uses TempDir::short_out_of_tmp, which needs a cache folder short enough, or a short LNK_TEST_STATE_DIR (LNK_TEST_STATE_DIR).

A program a test runs fails

A test ran a host program and it failed, or the test skipped saying the program is missing. Probe the program the way the gate does, from a folder outside the checkout, before installing anything.

Unknown binary 'rust-analyzer' in official toolchain

The code tool's test ran rust-analyzer in a sandbox, outside the checkout and without the RUSTUP_TOOLCHAIN Cargo sets, so rustup chose its default toolchain, which lacks it. Make the pinned toolchain the default and give it the component, as CI does: rustup default <pinned> and rustup component add rust-analyzer.

A test skips with a browser or display installed

The browser test finds Chromium on the PATH, in CHROMIUM or under PLAYWRIGHT_BROWSERS_PATH, and the desktop tests need Xvfb on the PATH. Set CHROMIUM to the browser's program when it is installed somewhere else, and install xvfb for the desktop tests.

Text file busy

A test ran a program it had just written with std::fs::write, and a child another test forked held the file open. Write it with the kit's write_program or copy_program (Programs and processes).

A test passes alone and fails in the full run

A test is green alone and red in cargo test --workspace, or only while something else builds. The cause is a race or an elapsed-time assertion, never chance. Run its whole test program several times on one build, under load, to count it, then fix the cause (Load and timing).

waited ... for ...; the last check saw

A wait ran past its deadline. Read what the last check saw: if the awaited event never came, the code under test lost it, which is the bug. A machine slower than most sets LNK_TEST_TIME_SCALE once. A longer deadline alone never fixes a lost event.

An assertion that something took less than some time

A test proved an order by elapsed time, such as an answer told within twice a pause. A loaded machine runs any step late, so the test proves the order by events the code reports instead.

Address already in use, or a port taken by another program

Two tests took the same port, one probed free and bound later, or one the code under test hands out itself. A test hands its server a listener it already holds (Held::loopback). A port a program the test starts takes itself is held across test programs with HeldNumber before it's probed, from a range the code's own lookup never gives out.

A port is the proxy's own, or a test server can't bind its port

A port probed free from the system's own range, bound to 0 and let go, can be any port there: one another listener bound to 0 is handed next, or one Link fixes for itself in that range, such as the sandbox proxy's own. A test's ports come from below that range, held with HeldNumber. A port reused from such a fixed range can still be in TIME_WAIT from a run before, so a server the test starts sets SO_REUSEADDR, as Rust's listeners and Python's http.server do.

A port held for a test is taken anyway

HeldNumber::port() holds one port. A program that binds the ports after its own too, such as a harness that serves its tools on the next one, takes a port another test holds. Such a program is given a block, held whole by its base, as the harness tests' free_block does.

a user ... is already on this machine in a root test

A root test that makes system users removes them when it ends, but a test killed mid-run never ends, and its user stays. The host then refuses to take over a user it didn't make. Remove the user a killed run left, with userdel -r lnk-<name>, and never kill a run of the root tests partway.

Broken pipe asking a fake

A fake answered without reading the question it was sent and exited, so the asker's write met a closed pipe whenever the fake ended first. A fake reads its whole input before it answers, as the program it stands in for does.

A test fails only with LNK_TEST_STATE_DIR set

A test passes with its folders in the temp folder and fails with them in the state folder, or leaves files there. The test or the code under it assumed something only /tmp gives.

A sandboxed program can't open a folder it was given

A program a sandbox runs may read and write the folders it was given, and never the folders that hold them. Under /tmp every parent is readable, so code that opens a folder through its parent passes there and fails with Permission denied elsewhere. It opens the folder by its own path. A test sees the difference with LNK_TEST_STATE_DIR set.

Folders left in LNK_TEST_STATE_DIR after a run

A turn that ends after its test can write a file again once the test's TempDir is gone, such as measures.jsonl. Folders in the state folder are named by a count, not the process, so nothing sweeps them later. Remove the state folder's contents between runs until that is fixed.

A check fails on files the change didn't touch

A check of the repository's files names files outside the change, or fails on a machine whose tools differ.

A check names files under .claude/worktrees

An agent session keeps worktrees inside the checkout, under .agents/worktrees, .claude/worktrees or .codex. A check that walks the folders reads each worktree as more of this checkout. It lists files from Git, or skips those folders.

ShellCheck disagrees between machines

ShellCheck 0.9, Ubuntu 24.04's, reports a function run only by a trap as SC2317, which 0.10 renamed SC2329. A disable comment names both, # shellcheck disable=SC2317,SC2329, so it passes either version.

The first build is slow

A session's first cargo installs the pinned toolchain, then builds the workspace from nothing. An environment's setup script installs the toolchain ahead, so the session's first check spends only the build.