Security

What buckets protect: your files and their copies in your own backends, against losing the last good copy, and the keys to those backends. Files go straight between this machine and your backends, and buckets use your own cloud accounts and bills: Local Link holds no storage for you. lnk bucket talks to the relay only for share links.

Backend keys

Where the keys to your backends are kept, how they're asked for, and how each rclone run is handed the one it needs.

rclone.conf

Keys are in ~/.lnk/bucket/rclone.conf: mode 600, in a 700 folder. Link's own edits are atomic and made under a lock. rclone rewrites the file itself when a sign-in token refreshes.

Secrets at a prompt

Secrets are typed at a prompt with echo off. connect <type> key=value refuses any setting rclone marks secret or sensitive, such as passwords, keys, account ids and hosts; rclone then asks for them.

AWS and Google Cloud

On AWS and Google Cloud, a bucket signs in as its lnk cloud account and nothing else (how accounts are kept). rclone.conf holds none of that sign-in. Each rclone run on the bucket gets the account's from lnk cloud: its aws profile's name or its access key, or its Google credential file, as RCLONE_CONFIG_<SECTION>_<KEY> variables, or as GOOGLE_APPLICATION_CREDENTIALS for a person's Google login.

The shell's own keys

For those runs the shell's own AWS_* and GOOGLE_APPLICATION_CREDENTIALS are removed, so the account used is the one connected, never whatever the shell says at that moment.

Encryption passwords

Encryption passwords are kept in the same file, obscured. That is not encryption: the file's permissions protect it. It's the only stored copy. lnk uninstall keeps it unless you pick buckets, and says so.

On a Mac

On a Mac, the backend keys and encryption passwords Link writes are in the Keychain instead (secrets in the Keychain), and rclone-keychain.toml beside rclone.conf lists which.

How a run gets them

Each lnk bucket command reads them from the Keychain once, holds them in memory while it runs, and passes them to rclone in its environment (RCLONE_CONFIG_<SECTION>_<KEY>) for each run. A sign-in rclone saves itself, such as a Google Drive token, stays in the file.

Other storage

Other storage (connect b2, drive, sftp, ...) is set up by rclone's own questions; its keys or sign-in tokens go in the same config file. backend connect refuses rclone types that wrap other remotes (crypt, alias, union, ...) or aren't storage (local, http, memory).

A run's own config

For an agent's collections in its bucket, rclone is handed, for each run, a config file of its own (mode 600, ~/.lnk/bucket/runs/) holding only the sections that run reaches, its secrets from the Keychain too, never any of them in its environment. It's read back and removed once the run is done; one a killed run left is removed at the next start, once its process is gone (and any after a day).

All in ~/.lnk/bucket/, each named in Configuration:

  • clouds.toml: which backends are connected, and which bucket is which folder's.
  • rclone.conf: backend keys and encryption passwords (see above).
  • manifest-<personal or agent id>.json: what went where, each file's size and modification time, and when each copy was last checked.
  • engine: the pinned rclone.

Temporary files and folders

Each rclone run gets its file list in a temporary file, mode 600 and created new, deleted afterwards. A folder backend's backup folder is created 700.

The engine

rclone at a pinned version, downloaded over HTTPS from its GitHub releases and installed only if its SHA-256 matches the one in the plugin's source. Archives over 100 MB are refused.

Its pinned sums

The pinned sums come from rclone's SHA256SUMS, checked against its release key's PGP signature when pinned.

How it runs

It runs with its own config file and every RCLONE_* variable removed. File names are always passed to rclone after -- or in a file list, never where rclone would read them as options.

LNK_RCLONE

LNK_RCLONE points at another rclone, for tests. Whoever can set it can already run programs as the user.

Keeping a good copy

What clean proves before it deletes, and what push does and doesn't check.

What clean guarantees

At the moment it runs, every connected backend has a copy of each file that matches the file here, and every file was reported on. The check is MD5 on S3 and GCS, through the encryption for encrypted backends, or a byte-for-byte read where the copies have no checksum, encrypted or not. That catches missing, truncated and corrupted copies.

On S3, a checksum the uploader set is not taken

S3 computes an object's MD5 only for an upload in one part, in its ETag. For an upload in parts, rclone takes the MD5 from the object's md5chksum metadata, which whoever uploaded it sets and S3 never checks. Anyone who can write to the bucket could replace a file with junk carrying the original's MD5, an agent's box with the bucket's key included.

So it reads them back

On aws, s3 and rclone s3 backends, clean first asks for each object's metadata (one request per file) and reads back every copy that carries an MD5 there, through the encryption too.

A copy is proved once

On those backends, what a clean proved (its dry run included) is kept in the manifest: the file here, by its size, inode, and modification and change times to the nanosecond (a write here that puts the modification time back still moves the change time), and its copy, by its size and the time S3 wrote it, from a listing that reads no file.

Why S3's time

Only S3 sets that time, so a copy written since in one request shows another and is checked again. An upload in parts takes the second it began, so junk of the copy's size finished after a clean from an upload begun the second the copy was written would list as the copy does.

Uploads not finished

So before it checks, clean asks S3 for the uploads in parts begun and not finished (rclone backend list-multipart-uploads): a copy with one on its name, or written in the second the check began, or on a backend that can't say, is checked but not proved. One finished before the check is read back as the junk it is, and one begun after it begins in a later second than the copy's.

A later clean

A later clean finding both as they were takes the copy as proved, without asking for its metadata or reading it back. The first check of a copy still asks for its metadata, one request per file, 32 at a time: rclone shows an object's ETag only through it.

A copy written during the check isn't taken

The copies are listed again after a check that checked any; one written in between, to swap a checked copy for junk carrying its MD5, is reported as written while it was checked, and nothing is deleted.

A "copy" in a folder backend that is the file itself is refused: the backup folder being a link back into Link's folders, or a hard link. Every command refuses a folder backend whose backup folder has come to overlap Link's folders. A file changed or replaced after the check, or now reached through a link, is kept.

What reading back costs

Reading back costs what a download does, and takes as long, once per copy: a dry run then a clean reads it once. rclone sends a file over 200 MiB in parts, with its MD5 in that metadata, so those are the copies read back. An archived copy can't be read until it's restored, so clean keeps such a file here.

What push takes

push takes the checksum the backend reports, so it doesn't notice a copy replaced this way and doesn't send the file again, --replace included: reading every copy back on every push would cost a download each time. clean finds it and says it differs. To send the file again regardless of checksums:

lnk bucket push photos/2019 --recheck --replace -- --ignore-times

What no check protects against

A backend losing data later, a closed cloud account, lost keys or a lost encryption password, or a backend that sets out to deceive: the provider writes the ETag too, and MD5 is not collision-resistant against an attacker. On other storage rclone reaches, such as Backblaze B2 or Azure, the checksum of a large upload can also be the uploader's word, and clean takes it.

Local files

What keeps a bad file here from replacing a good copy, and a file from being taken for another.

Bad local files can't overwrite good copies

push refuses to replace a copy that differs unless --replace, so files corrupted or encrypted by ransomware here are reported instead of sent. Files never pushed are only as safe as this machine.

push trusts size and modification time

A file with the same size and modification time, to the nanosecond where the file system keeps them, as when its copy was checked is skipped unread. A file changed without either changing, or a copy damaged in the backend since, goes unnoticed by push; push --recheck and clean check every file.

Names backends store differently are skipped

A file whose name has control characters, or the look-alike characters rclone uses for them, is never pushed or cleaned, so two such files can't be taken for one another.

push and clean skip symlinks. pull refuses any file whose way from the folder goes through a link, which could lead outside it, for example into ~/Library/LaunchAgents.

The folders are kept apart

Link refuses to work when your files' folder and an agent's are one inside the other, or when either is or holds your home folder or Link's settings. When the harness plugin is installed but can't list the agents, it stops too, rather than take an agent's folder for yours.

Deleting from backends

remove counts a copy in another backend as kept only if a listing of that backend, made now, has it at its recorded size.

Who can read the files

Who reads your files in a backend depends on --encrypt, and what a bucket's name and labels say of you.

With and without --encrypt

Without --encrypt, the cloud provider, and anyone with access to the bucket. Buckets Link makes are private, GCS ones with uniform bucket-level access. With --encrypt, rclone's crypt encrypts contents (XSalsa20-Poly1305) and names before upload.

Bucket names say nothing about you

local-link- and 12 random characters, since bucket names are global and anyone can test whether one exists.

Labels do

An agent's bucket, in a backend from lnk cloud, is labelled lnk-agent (its ID), lnk-agent-name (its name) and lnk-owner (your account's owner ID, acct-<id>). Anyone who can list the account's buckets can read them, as with a box's tags.

lnk-layout

A backup adds lnk-layout (lnk bucket label), which a restore reads to know how the backup is laid out: only your account sets it, since the key a box gets for an agent's bucket reaches its objects alone (aws's s3:GetObject, PutObject and the like, gcp's roles/storage.objectAdmin), never its labels.

Agents' files

Each agent's files have their own bucket, always encrypted, apart from yours. Link backs both up with the key you connected, which reaches both.

While it runs

clean and pull refuse in an agent's folder while the agent runs, in the background or in a terminal, and when the harness plugin can't say whether it does. clean asks again right before deleting, and refuses if the harness plugin doesn't answer then.

Its password

Reconnecting an encrypted bucket, or taking an agent's in, needs its password: backend connect and adopt refuse one that doesn't decrypt a file already there. A file's sealed first block decides, not its name: about one name in 256 decrypts under a wrong password, by chance. A name decides alone only when every file it checks is empty.

A box's buckets

When an agent moves here from a box, the box names its buckets. Link takes only a name it gives an agent's bucket, in any backend: local-link- and 12 random characters, or local-link/<agent id> on a folder. In a backend from lnk cloud, the bucket must also be labelled as that agent's. So a box can't point an agent's backups at another bucket or folder of yours.

Found by their labels

Finding an agent's buckets by their labels takes, when you're logged in, only buckets labelled with your login's lnk-owner for lnk agent restore. A box's keys, revoking them and a move's buckets also take one with no owner, made before you logged in, and never one labelled as another login's.

Two buckets for one agent

Labels are only what anyone who can make buckets in the account says, so two buckets labelled as one agent's in one backend are refused, both named: delete the one Link didn't make.

Their collections in the bucket

Where an agent's spec prefers the bucket place for a collection and its machine supports it (Agent Spec), the bucket place's driver keeps it, through lnk bucket's engine, run by the agent's storage bridge outside every sandbox, only in the agent's own bucket under .link/agent/, which a push, a pull and a clean of its files never touch (Bucket Store). Names and contents are encrypted as the rest of the bucket is: a conversation's name, a file's path and what each holds are never readable there.

lnk bucket rclone is unguarded

It runs the pinned rclone with Link's config, so it can do anything the connected keys allow, including sync and purge, which delete. Link's refusals apply only to its own commands.

lnk bucket link create hands out files from your backends with a link, https://<user>.<domain>/b/<id>.

Each file's backend signs a URL for it on your machine, with your keys: S3 presigned URLs, through rclone. The relay keeps those URLs and gives them to whoever opens the link, who downloads from the backend. Your backend keys never leave the machine.

A signed URL lasts until it expires

Anyone holding a signed URL can download that file until it expires, whatever happens to the link. lnk bucket link delete ends the link at the relay, not URLs already handed out. A share lasts at most 7 days, S3's longest, and each URL expires with it.

Shorter credentials, shorter URLs

Credentials that expire sooner sign shorter URLs: URLs signed with an SSO or aws sts session stop working when that session ends. The relay's operator can read the URLs, as they can read tunnel traffic.

Only your files, unencrypted, in S3 storage

An agent's files are refused: their buckets are encrypted, and a link would hand out ciphertext or need the password. So are encrypted backends, archive tiers, and backends that can't sign expiring links (gcp, folders, other rclone storage).

24 random letters and digits, about 124 bits, on the sharer's own host, which gets a certificate only while they have a live share.

A share's page, password and account

What a visitor to a share's page sees and must give, and whose account a share is.

The page is Local Link's own and static

Names are escaped, a Content-Security-Policy lets nothing run or load, and it's sent no-store and no-referrer, so the URLs aren't cached or passed on. It says who shared it and that Local Link doesn't check the files.

Only URLs shaped like S3's

Over HTTPS the relay takes a file's URL only with a whole SigV4 presigned query: X-Amz-Algorithm of AWS4-HMAC-SHA256, X-Amz-Credential, X-Amz-Date, X-Amz-SignedHeaders and a 64-digit hex X-Amz-Signature. That keeps out plain links, but any host can be S3-compatible storage, so a sharer can still make a URL of that shape to any site.

The visitor warning

So an account's share page shows the warning its tunnels get before the page, once a week per browser. lnk bucket pull and programs asking for JSON pass. Share pages of the relay's users-file users show none.

A password

A password (--auth) is checked by the relay with HTTP Basic before anything is shown, and kept only as a salted PBKDF2-HMAC-SHA256 hash. Visits and password checks have the relay's limits.

Once given right

Once given right, it is also kept in memory as an HMAC of the link's salt and the password under a key the relay makes at each start, so a visitor coming back costs no PBKDF2; a wrong one is always checked in full, two at a time relay-wide and one at a time at one account's links.

Wrong guesses

Wrong guesses are limited per visitor and per link, as for password-protected tunnels, each counted before its check, so guesses sent at once count as many.

Your account, not your machine's key

The bucket plugin gets an access token from lnk auth token that works only for this relay's share API, scope share, and is short: the relay takes it for at most 15 minutes from its making (MAX_TOKEN_SECS), refuses it for tunnels, and no service takes it. A blocked user's links stop working, and their tokens are refused.

A share is its account's

The relay keeps the account that made each share, and only that account lists or removes it, so another account that takes a username after it changed gets none of the old holder's shares. Links made before a username change stay at the old name's host until they expire, or the old holder removes them (lnk bucket link delete, as their new name).

Pulling a share

lnk bucket pull <link> writes into the current folder only. Names are relative paths without .., a name reaching through a link here is refused, an existing file is kept unless --replace, and each file is written beside its place and moved there only once its size matches the share's.

Over https:// only

It reads the link only over https://, so a password never crosses the network in clear, and fetches each file only over https://, from a public address, without following redirects, and never through a system proxy (HTTPS_PROXY), which would resolve its name past that check: a share can't make your machine reach its own services or your network's. A share from a relay on your own machine (a dev relay) is the one exception.

Gaps

What buckets don't protect, and a command whose meaning changed.

Keys in a file

Keys and passwords are kept in a file, and one key reaches both your bucket and your agents'. See known limitations.

Archive tiers

Archive tiers exist only on aws and gcp. Other storage classes need lnk bucket rclone.