Reporting a security problem

What is in scope #

Two things: the desktop app and this site. The app is the interesting one.

Local Test runs a web server on your own machine, on 127.0.0.1 by default and, when you switch it on, on your local network as well. It reads folders you point it at, runs commands you configure, clones repositories you name, and stores API keys and access tokens you give it. Anything that crosses one of those lines is worth reporting:

• A way for a page, a device, or a repository to read a file outside the folder that was served, including through a symlink, a path that escapes, or a name the deny list does not recognise.
• A way for a website you happen to have open to reach a running project, for instance by rebinding a name it controls to your loopback address.
• A way for the contents of a project to cause something to run: a repository, a file name, a manifest, or a served page that ends up executing.
• Anything that exposes a stored secret: an API key, a GitHub token, a licence key, or a project's environment variables, including in a log, a backup export, or on screen.
• A way past the checks on an update or a downloaded server file: a checksum that is not enforced, a host that is not on the allowlist, a redirect that changes the origin.
• On this site: anything that gets script onto a page, changes what a download link points at, or would let a deploy serve something that was never committed.

Out of scope, and please do not spend your afternoon on them: the absence of a code-signing certificate, which is deliberate and documented; the fact that a served project is reachable from your own network once you turn that switch on, which is the switch working; missing security headers on a domain that only ever serves static files, unless you can show what the missing one would have stopped; and reports produced by a scanner with nothing behind them.

How to report #

Email hello@graysonanderson.com. That address is the one in security.txt and the one in the footer of every page here, and it reaches a person rather than a queue.

What helps, roughly in order of how much:

• The version, from the corner of the app window, and the operating system.
• The steps. A folder laid out a particular way, a setting turned on, a request sent. Something that can be repeated beats something that can be argued about.
• What you got, and what you expected instead.
• If it is easier to show than to describe, a small project folder or a request that demonstrates it.

Please do not test against anybody else's machine, and please do not need to: everything in this app runs on the computer in front of you, so the whole of it can be examined without touching a system you do not own. There is no production service to probe. The site is static files.

What to expect #

An acknowledgement that a human has read it, and then whatever the truth turns out to be. This is one person rather than a team, so the honest version is a few days rather than a number of hours picked to sound impressive.

If it is a real problem, it gets fixed in a release, and the changelog says what it was. That is not a formality: several entries there describe exactly what was wrong and what could have happened, because a changelog that only lists features is one nobody can use to decide whether to update.

There is no bug bounty, and no money. Saying so plainly is fairer than leaving it to be discovered after the work is done. Credit in the changelog is offered gladly, and declined gladly if you would rather not be named.

Nobody is going to be sent a legal letter for looking. Test against your own machine, do not go after anyone else's data, give a reasonable amount of time before publishing, and this side of it will not be a problem.

Encryption, and the PGP key that does not exist #

There is no PGP key. Not one that is lost or pending: there is not one, and security.txt therefore has no Encryption line rather than a line pointing at something that would never be read. Publishing a key nobody watches the mailbox of is worse than publishing none, because it invites a report that then sits encrypted and unopened.

So a report arrives as ordinary email, which is worth knowing before you write the interesting part of it. If a finding is sensitive enough that this bothers you, send a couple of lines saying roughly what area it is in and asking for another channel, and one will be arranged. That is a slower first step and a better one than encrypting to a key that has no reader.

What this project does not claim #

A security page that only lists what is protected is half a page. These are the boundaries, written down rather than implied:

• The builds are not code-signed. No Apple Developer ID, no Windows certificate. Both are paid yearly memberships and neither has been bought, so macOS and Windows both warn on first launch. That warning is accurate: nobody has paid to put their name on this file.
• A checksum is not a signature. Every artifact here publishes a SHA-256, and the app refuses to install an update whose checksum the manifest does not publish. That proves the bytes are the bytes that were published. It does not prove who published them, and if this site were taken over, the checksums on it would be taken over with it.
• The same is true of the Minecraft server files. Mojang publish a SHA-1 and PaperMC a SHA-256, and the app checks both against a fixed list of hosts over HTTPS. Neither publisher signs those files in a way an app can verify, so the trust boundary is the publisher's own HTTPS and nothing further. BuildTools has no published checksum at all, and the app says so out loud rather than implying a check it is not doing.
• Your keys are encrypted at rest, and that is not the same as safe. The GitHub token, the AI provider key and the licence key go through the operating system's own store, the Keychain on macOS and DPAPI on Windows, so the settings file on its own no longer contains them. What that protects against is a copy of the file: a backup, a synced folder, a disk image. What it does not protect against is anything running as you on your own machine while you are logged in, because that is precisely the condition under which the operating system will hand the key back. There is no protection against that available to an app in this position, and claiming one would be a lie.
• AI mode sends what you send it. If you use a hosted provider, the text of the prompt goes to that provider under your own key and their terms. The app shows you the whole prompt before it is sent and sends only what you press Send on, but it cannot make somebody else's service private. Ollama, running on your own machine, sends nothing anywhere.
• This site has no analytics and no logging you can audit. Nothing is collected here, which also means there is no access log to go back through if something does go wrong. That is the trade, and it was made on purpose.

Checking what you downloaded #

Every release publishes a SHA-256 for each file, on the download section and in SHA256SUMS.txt. That section carries a one-line command for macOS and for Windows that answers in a word, rather than a hash for you to compare against another hash by eye.

Every release also publishes a source archive beside the installers, with its own checksum. Local Test is not open source and the licence does not permit redistributing it, but the source is published so that anybody can read what the app does before running it, and run the test suites themselves. For a program that serves your folders and holds your keys, that is the part worth having.