Skip to content
Dust & Observer

The developer path

Run Observer from source.

The harness builds into two parts — a server with a database behind it, and a client that runs the agent. This is the path where you build both, in the order the repository documents.

Observer

Two halves, and they do not have to share a machine

The server is the API and its Postgres: authentication, durable memory, embeddings, traces. The client is the CLI and the agent that explores the application and verifies against your criteria.

  • The server belongs on a host that stays up, because that is where runs, stored facts and traces accumulate.
  • The client belongs on the machine doing the testing. It holds the tester’s own model key and the server’s key, and nothing else.
  • The same code runs in both places. The server’s address — QA_API_URL — is the difference.

Both are built on one machine in the steps below. Moving the server elsewhere later is a matter of the client being pointed at an address it can reach, not of building the code twice.

What you need before the first command

  • Node 22 or later, and a shell to build from: Git Bash or PowerShell.
  • Postgres 16 with the pgvector extension, running where the server can reach it.
  • A DeepSeek key. It belongs to the client, and it is what the agent’s model calls run on.
  • An embeddings key for the server, so the memory layer has something to store facts against.
  • An environment you nominate: a staging deployment of the application you want verified, reachable from the machine running the agent.
  • Optional: a TypeSafe key. With it the System One quality judge and the acceptance-criteria judgment run; without it that judge reports itself unscored and criteria fall back to a deterministic scan.

The embeddings key is what makes recall work. Without it the server stores no embeddings, so the agent still verifies and still files findings, but it stops carrying facts from one run to the next: recall is lost rather than the run failing.

Clone, install and build

  1. 1

    Clone the harness

    One repository holds the agent, the CLI and the server, and the commands below assume you are inside it.

  2. 2

    Install dependencies and build

    Dependencies install with lifecycle scripts switched off, then pi — the coding agent the harness extends — is built.

# 1. Clone and build
git clone <repo-url> qa-harness && cd qa-harness
npm install --ignore-scripts
npm run build

The clone line carries a placeholder for the repository’s address, which is yours to supply: a fork, an internal mirror, or wherever your team keeps it.

Configure the server’s keys

  1. 1

    Copy the example file

    The example file in the repository is the authority on the variable names. It ships with the code, so it is the place to check when a name has moved.

  2. 2

    Set the model key

    DEEPSEEK_API_KEY is the client’s. The agent’s model calls are made with it, on the machine running the tests.

  3. 3

    Set the embeddings key

    The server’s memory is pgvector plus embeddings, so the server holds that key. The client never needs it and never sees it.

  4. 4

    Set the optional keys

    TYPESAFE_API_KEY is optional and server-side. POSTGRES_PASSWORD matters when you bring the database up with the compose file, which has no committed default; on anything but localhost, set a real password first.

# 2. Configure secrets
cp .env.example .env

Keys stay on the side that uses them. The server’s secrets are never handed to the client, and the client’s model key is never handed to the server.

Start the server

The repository ships its own start script. It binds to 127.0.0.1 on port 7377 by default and applies the schema migrations on first boot, so an empty database is fine; an unextended one is not.

If the server and its database belong on a host of their own rather than on the machine you are building on, the compose file in the repository brings up Postgres with pgvector and the server together and applies the migrations on the way.

# 3. Start the server (PowerShell)
.\dev.ps1
# Or Bash: ./dev.sh

It answers on /health with its status and version, which is the first thing to check before anything is pointed at it.

Register a client key

  1. 1

    Register the first account

    The setup command registers an email against the server and writes the client’s config file, so the CLI knows which server to talk to and which key is its own.

  2. 2

    Know how registration closes

    The first user on an empty server registers freely. After that the route refuses until the server is started with QA_ALLOW_OPEN_REGISTRATION=1, so onboarding a team means setting the flag, letting each person register, then removing it and restarting.

  3. 3

    Treat the key as a secret

    It is shown once at registration and stored as a hash, so it cannot be read back out of the server. It can be rotated or revoked from the auth route if it is lost or leaked.

# 4. Set up your account (registers + writes ~/.qa/config.json)
npx tsx packages/qa-cli/src/cli.ts setup

Start the agent

The agent runs from the same repository, on the machine doing the testing, in a shell that can reach the server you just started.

# 5. Start the QA agent
.\pi-test.ps1 --env --approve
# Or: npx tsx packages/qa-cli/src/cli.ts --approve

Both lines start the same agent: the repository’s script, or the CLI entry point on its own.

First run against an environment you nominate

  1. 1

    Name the target

    Point the agent at the deployment you want verified. Naming the kind scopes the toolset to it, so a web target gets the browser and a target with an OpenAPI spec gets contract checks. Leave the kind off and the harness probes the address and maps whatever answers.

  2. 2

    Say what you want in a sentence, if that is easier

    The target can also be set in plain language. The repository’s own first command is a sentence that carries the kind, the address and the intent in one line.

/target web https://app.example.com

The address above is the documentation’s placeholder — substitute the environment you nominated. The agent treats it as a black box: it drives the browser and calls the API rather than reading the application’s source.

Before it is reachable from off the machine

  • The server speaks plain HTTP. Terminating TLS is the reverse proxy’s job — Caddy or nginx in front of it — and it stops being optional the moment anything other than localhost can reach the port, because the API key rides every request.
  • The database port stays unpublished. Keep Postgres on the internal network instead of adding a port mapping on a shared host, and remember the role’s password is only as strong as the password you set.
  • Point the client at it by address. The client takes the server’s address from its config file, or from QA_API_URL in its environment, so a team can move between a local server and a shared one without rebuilding anything.

Where your data lives, and what leaves it

If the people testing should not be building anything

There is a packaged path as well. The desktop app is a thin client for a server, so the database, the model keys and the agent stay where the server is, and the machines doing the testing carry none of them.

The two paths run the same server and the same agent, so a verification started from either one is the same verification.

The packaged app pathThe pack for your IT and security colleagues