The pack
Everything your IT and security colleagues will ask, in one file.
This is the page a QA lead forwards. Print it, or download it as markdown and put it in the ticket your security team is already watching.
Dust & Observer — the pack
Generated September 23, 2026 · https://dust.observer/pack/
Download as markdownPrint this page for the same document as a PDF.
1 · What runs where
There are two programs and one server. Dust is a tracker: a single deployment with Postgres, holding issues, acceptance criteria, comments and an append-only event log. The harness is the QA agent: it drives a browser, calls an API, checks a contract, captures what it saw. The desktop app is a client for that harness — it holds no database of its own and no model key.
If the harness runs on your infrastructure, the findings, the evidence and the runs stay there. If you use the instance run for early access, they live on that server instead, readable by whoever operates it. The same software runs in both places, so the choice is reversible and it costs nothing to change it later.
Dust calls no model provider. Its QA chat passes the question to the harness, so the model call is the harness’s, made with the harness’s key. The only external service Dust itself contacts is TypeSafe, and only when an operator configures a key for it.
Free during early access. The AI features run on a small per-user daily allowance that your administrator enables.
2 · What leaves the machine
When you run the harness yourself, the accounts and the keys are yours: the usage appears on your own bill, and you can see every call the agent made.
| Service | Job | What it receives |
|---|---|---|
| DeepSeek | The agent’s language model | Every inference the agent makes: planning a run, reading a page, writing up a finding. The calls are made by the harness with the harness’s key. Dust calls no model provider at all — its QA chat passes the question to the harness, so the call is the harness’s. |
| TypeSafe System One (Jev) | Typed decisions, for both products | The harness asks it to score a run, extract acceptance criteria, classify an HTTP failure or a console line, look up an element, and order the review queue. Dust asks it to rescue a severity word, cross-check a would-be all-clear verdict, and keep a polish edit from dropping a fact. Each call carries only the state its questions read, and each is fail-open: an answer that never arrives costs a judgment, never a result. |
| OpenAI | Text embeddings, for the harness’s memory only | Text the agent has chosen to remember, turned into vectors so it can find what it already learned. It is not used for inference, and the key lives on the server — the command line never sees it. |
3 · What you receive
- A verification report per issue: every acceptance criterion with its state, the checks that ran against it, and every finding with the steps and evidence that produced it.
- A leadership render of the same issue, for whoever signs it off, with criterion identifiers and raw evidence left out.
- The findings themselves, filed against the criterion each one violates, so a fix has a target and a retest has a definition of done.
- A verification matrix across runs, so a regression shows up as a change of state rather than as a new document nobody compares.
- The issue’s own event log: who moved it, when, and from what — the part that survives everyone who worked on it.
Read both renders of one issueHow a criterion becomes evidence
Both renders linked above are synthetic examples, not client engagements.
4 · How a verification runs
Black box. The agent is pointed at a running application — staging, a preview build, anything it can reach — and works on it the way a user or a client would: through a browser, or over HTTP. It does not read your source code, and it does not need to.
- The acceptance criteria come from the issue, where the team already agreed on them, rather than from a second list maintained for testing.
- Checks are planned per criterion, including the negative ones: what the application does when it is supposed to refuse.
- Every check records what produced it — the step, the response, the capture — and nothing is filed without evidence.
- Re-running is the ordinary case, not a special one: when the issue moves, the scenario that found the defect runs again against the new build.
5 · Who checks the checker?
The verification is run by the same person who builds the tooling, so treat that exactly as you would any supplier with a stake in the result: check the work.
Every finding carries the check that produced it, the steps to reproduce it, and the evidence captured at the time. The risk verdict is arithmetic over those checks, not an opinion. A finding that does not hold up gets dismissed, and it stays visible in the report as dismissed rather than quietly disappearing. The false-positive rate is published rather than asserted.
The tooling is source-available to a licensee, so your own engineers can follow the verification path — and they can run it themselves against the same build and get the same findings. Reproducibility, not neutrality, is what makes this checkable.
And if you would rather have no supplier in the loop at all, install it and run it yourself. It is the same engine.
6 · What your own people can check
- The licence. The software is proprietary, licensed to the organisation that runs it, and a licensee’s engineers may read the source under that agreement. It runs on your own infrastructure, so keeping it running does not depend on us. (The agent framework the harness builds on is MIT, and that notice ships with it.)
- The interface. What crosses between the tracker and the harness is a documented contract rather than a private convention, so your engineers can see exactly what is exchanged and what is not.
- The result. A verification is a set of checks against a build. Run the same checks against the same build and you should get the same findings — reproducibility is the property that makes a supplier checkable.
- The numbers. What the agent cost to run, how many findings were confirmed, and how many were explicitly marked wrong are published in one block with their provenance.