Testing Kit keeps tests free of races

link-test-kit holds what keeps a test free of races and of the machine it runs on, written once. A test uses the kit's helper rather than the raw call it replaces, and rather than a helper of its own: a helper written twice is fixed in one place and not the other. A suite's own fixtures, such as a fake model or a spec it builds, stay with the suite.

Use it

The kit is for tests only. A package names it as a development dependency:

[dev-dependencies]
link-test-kit.workspace = true

In a crate only for tests

A crate that is itself only for tests, such as the harness conformance suite or a fixture, names it as an ordinary dependency. Code that ships never does. Each workspace reaches it through its workspace.dependencies, the public one by packages/test-kit.

What it holds

One module per concern. Each item is also at the crate's root, such as link_test_kit::TempDir.

temp

TempDir::new(name), instead of std::env::temp_dir(), tempfile, or a folder removed by hand: it is removed when the test ends, failed or not, and a killed test's folder goes on the next run. TempDir::under(parent, name) instead of a folder made in parent by hand, such as CARGO_TARGET_TMPDIR. TempDir::short(name) instead of a socket under CARGO_TARGET_TMPDIR or a Mac's TMPDIR, past the socket path limit: its path is at most SHORT (44) bytes, and a longer name fails the test.

Short folders out of /tmp

TempDir::short_out_of_tmp(name) is for a fixture a sandbox reads that holds sockets, such as an agent's home. It uses a short LNK_TEST_STATE_DIR when set, else this user's cache folder (XDG_CACHE_HOME, else ~/.cache) when a short folder fits there. A checkout's CARGO_TARGET_TMPDIR can be too long for a socket under it. Otherwise it falls back to TempDir::short, and gate::out_of_tmp reports that the folder is in /tmp.

programs

write_program(path, text) instead of std::fs::write and a mode change before running the file ("Text file busy"), and copy_program(from, to) instead of std::fs::copy before running the copy. cargo() instead of Command::new("cargo"), which carries the test's Cargo variables and rebuilds the workspace.

programs() and programs_in(root)

Instead of building lnk or its plugins from a test: programs() is the target/<profile> the test was built in, where cargo build --workspace --bins put them, and programs_in(root) the same for another workspace, the nearest target/<profile> above root.

process

ChildGroup::spawn(&mut command) owns a background process in a group of its own, after env::command or env::clean sets its environment. Its drop stops the group and reaps the direct child, including on panic. take_stdout and take_stderr give access to piped readiness signals and diagnostics; stop ends the fixture early. kill_group(child) stops an existing group; interrupt(child) sends SIGINT to one child.

Group ownership

The guard keeps the leader unreaped until cleanup so its group number cannot be reused. On Linux and macOS, try_status observes a natural exit without reaping it. The guard stops descendants that remain in the group; detached descendants need another owner. An orphaned grandchild is reaped by its adopter, so cleanup proves it stopped rather than claiming to reap another process's children. Use a domain fixture for its protocol, with the guard owning the process beneath it.

Dedicated supervisors

On Linux, process::Subreaper::for_fixture lets a dedicated fixture process adopt orphaned descendants when the container's main process does not reap them. After every ChildGroup drops, its reaped method waits without blocking and reports whether any children remain. It never signals them. Use it only in a dedicated supervisor, since ordinary test processes can have other tests' owned children. Drop reaps exited adoptees and restores the previous setting.

net

Held::loopback() instead of a port found free (bound to 0 and let go) and bound later by the server, which another program can take in between: the test holds the listener and gives it to the server, .listener() in its own process, .hand_to(command, variable) to a child. nodelay_of(group, port, peer) instead of reading a connection's TCP_NODELAY another process holds by hand.

HeldNumber

HeldNumber::first(kind, candidates, free) instead of a number probed free and taken later by a program the test starts, such as an X display or an agent's ports. It holds the first candidate whose lock no other test holds, across every test program on the machine, and that free says nothing else has taken, until it's dropped. HeldNumber::hold(kind, number) holds one number for a caller with a search of its own, such as the conformance kit's port blocks.

HeldNumber::port

HeldNumber::port() instead of a port bound to 0 and let go for a program the test starts that binds its port itself, such as an sshd or a harness given LNK_PORT: a loopback port from 12000 to 15999, below the system's ports and Link's own, read with .port_number().

release and gate

release::serve(name, versions) instead of a release server of a test's own, with the tests' signing key. gate::bubblewrap(), landlock(), sshd(), root() and out_of_tmp(dir) instead of a check of a test's own and a silent return (Gates); gate::program(name, args) instead of a have(program) of a test's own, for a host program such as python3 or git; and gate::sandbox(lnk_sandbox, home) for whether the workspace's built lnk-sandbox runs a sandbox here.

env and users

env::command(program, home) instead of Command::new with env_clear and a HOME of the test's, or the test's whole environment inherited (A child's environment), and env::clean(command, home) the same for a command made another way, such as cargo(). users::root(), me() and my_group() instead of geteuid() == 0, id -un, id -gn, or reading USER, and users::command_as(uid, gid, groups, program, home) instead of setpriv written by hand, to run a program as another user.

wait

eventually(what, deadline, check) instead of a loop of sleep and an Instant::now() deadline, or a wait_for, until or wait_until of a test's own, and within(what, deadline, check) the same loop where the deadline passing is the caller's to handle. eventually_async and within_async are the same loop with tokio::time::sleep, in an async test, and eventually_await and within_await the same for a check that is itself awaited, such as a request to a server, or a loop around one.

wait's polls

wait::exited(what, child, deadline) instead of polling try_wait, wait::accepting(what, address, deadline) and wait::accepting_unix instead of polling connect until a server listens, and wait::appears(what, path, deadline) instead of polling exists.

Where write_program comes from

write_program and copy_program are lnk's own (link_plugin::write_program), so a program the code under test writes and one a test writes are written alike (why).

Waiting

A test waits for the condition it needs, never for a fixed time. Its check returns a bool, an Option or a Result: done with true, Some or Ok, and otherwise what it saw. The kit polls it, from 5 ms doubling to 200 ms between checks, until the deadline. Past it, eventually fails with what was awaited, how long, and the last value the check saw:

waited 10.0s for the provider's port file; the last check saw: absent

within

within returns the same as an error instead: for a test that says more on failure, such as a server's log, or that expects the wait to run out.

LNK_TEST_TIME_SCALE

Every deadline is multiplied by LNK_TEST_TIME_SCALE, a positive number, 1 when unset. A loaded or slow machine sets it once, such as LNK_TEST_TIME_SCALE=3, and every wait gets more time, rather than each test adding slack of its own.

Gates

A gate answers whether a test can run on this machine, and the test returns where it can't. It is the only way a test skips: a test that prints its own skip and returns can't be required by any variable (Skipping Tests).

if !link_test_kit::gate::bubblewrap() {
    return;
}

What it prints

Where the capability is missing, the gate prints the test, what it lacked and why:

proxy::the_listening_proxy_refuses_this_machine: skipping, as it lacks bubblewrap: bwrap: No such file or directory

Under a variable that requires it

Under the variable that requires the capability, it fails the test instead, so a machine meant to run every test never passes by skipping one. LNK_REQUIRE_SANDBOX requires bubblewrap(), sshd(), sandbox(lnk_sandbox, home) and out_of_tmp(dir), LNK_REQUIRE_NATIVE requires landlock(), and LNK_REQUIRE_PROGRAMS requires program(name, args). A workspace program that isn't built is never a skip: sandbox() fails the test, as the run is missing a step.

root() and a suite's own gate

root() has none: CI runs as an ordinary user, and code only root may run is run as root before it merges (Test environments). A suite's own gate, for a capability only its code can probe, says so with gate::lacks(what, why, variable).

A child's environment

env::command gives a program the test runs nothing of the test's environment but what it needs to run here, so it passes or fails alike whoever runs it, from whatever shell:

  • HOME, the folder the test gives, and TMPDIR, that folder's .tmp, made for it.
  • PATH, where the host programs a gate found are.
  • The proxy and CA variables (HTTPS_PROXY, NO_PROXY, SSL_CERT_FILE and the like: env::PASSED), through which this machine reaches the network.
  • The test's own LNK_*, such as LNK_REQUIRE_SANDBOX or LNK_TEST_TIME_SCALE.

What it never passes

None that holds a secret: what link_secret_value::process::variables leaves out never reaches the child. Never USER, LOGNAME or any XDG_*: a program that needs its user asks the system. A variable the test sets on the command afterwards is added to these.

Clippy keeps tests on the kit

Each workspace's clippy.toml refuses, in every crate and its tests, the raw calls the kit replaces: std::thread::sleep, tokio::time::sleep and std::process::Command::new. A call that stays says why where it is:

#[expect(clippy::disallowed_methods, reason = "a fake model slow on purpose")]

What Clippy can't see

A pattern of calls, not one call, is read from the source by a repository check: a skip printed by hand, a port probed free and let go, an assertion on elapsed time, a wait made of a loop that sleeps, a test's change to its own process's environment, and the folders and programs a test makes by hand (the check). A raw call that stays says why; one that says it "moves onto the testing kit" is counted, and the count only goes down.

Starting a program

Link's own code starts a program from link_secret_value::process::command, and a test's from env::command.

Custom test programs

A test requiring harness = false uses harness::Counts::summary for its final libtest summary, with actual outcomes and elapsed time. harness::Selection handles include filters, --exact, --skip, --list, --ignored and --show-output, rejecting unsupported flags. It executes nothing. The custom main preserves its child modes and failure behavior, and explicitly reports cases not run after a failure as ignored, with their reason.

Platforms

Each helper has a Unix implementation. Where one has no Windows implementation yet, the kit fails to compile on Windows with an error naming it: process::kill_group, process::interrupt, net::Held::hand_to, users, and the check that a test process is gone, which TempDir uses to remove a killed test's folders. nodelay_of reads the flag on Linux and macOS and returns none elsewhere. gate::bubblewrap and gate::landlock say elsewhere that they are Linux's, and users::command_as is Linux's.