A test dashboard that isn’t tested is just a claim. testpulse is built test-first. Its own CI runs every suite on every pull request and reports the results here, alongside everything else.
Five rules from the specification. Each one exists because breaking it has cost real time on a real project.
Every phase starts with failing tests written from its acceptance criteria. A phase is done when each criterion has a passing automated test or a recorded manual check.
Parsers are tested against actual result files captured from Ostomate2, RouteServe and testpulse itself: Gradle JUnit XML, Jest JSON, Vitest and Playwright output, JaCoCo and istanbul coverage.
Projects send the files their tools already produce. testpulse does the parsing, so a project’s CI step stays a few lines long.
If testpulse is down, a project’s CI stays green. The reporting step always exits 0 and logs a warning.
In testpulse’s own pipeline, nothing is allowed to fail quietly. No || true, no continue-on-error on test steps.
Parsers are tested against result files captured from Ostomate2 and RouteServe, not hand-written samples. Integration tests run against a local Supabase, including the row-level security that hides private projects from the public.
| Layer | Tool | Scope |
|---|---|---|
| Unit | Vitest | Parsers against real fixtures, layer resolution, stat calculations, API key hashing. 90% line floor. |
| Contract | Vitest | The shared GitHub Action every project reports through, run against a local server. Runs in the unit job; counted as Unit. |
| Component | VitestTesting Library | Each component built from the design, rendered in jsdom: its words, states and links, and its stylesheet against the design’s values. |
| Integration | Vitestlocal Supabase | The ingestion endpoint end to end against local Supabase: idempotent retries, transactions and rollups, and row-level security checked with the anonymous client, so private projects stay hidden. |
| E2E | Playwright | A seeded database behind the landing, project, run and test-history pages; visibility rules in the rendered UI; every public page again with JavaScript turned off. |
| Accessibility | axePlaywrightcontrast script | axe on every public page, counted with E2E. A script in the unit job checks the contrast of the design’s token pairs in both themes. |
| Visual | Playwright | Snapshots of each page at desktop and phone widths, light and dark, compared inside a pinned container image so rendering can’t drift. Counted with E2E. |
| Leak sweep | Playwright | Walks every page that can show a private project as a visitor and fails if any private project’s failure text, stack trace or source link appears anywhere. Counted with E2E. |
|| true, no continue-on-error on test steps.Two incidents from Ostomate2’s post-mortems shaped how results are shown. A green dashboard means nothing if the pipeline behind it has stopped running, or if a suite passes without executing anything.
Since , testpulse’s CI posts its results to this site as the project testpulse, through the same shared GitHub Action Ostomate2 and RouteServe use.
Only spec §8 layers are counted. The axe, visual and leak-sweep checks run inside the Playwright suite and count as E2E; the Action’s contract tests run in the unit job and count as Unit. Each test counts once, in the layer testpulse’s layer rules give it, whichever report it arrived in.
Full project page →unit: 4,405 of 4,452 lines covered.