Test Output Parsers

Motivation

The test-status widget needs to decide pass/fail and show a meaningful summary. Relying on output regex alone for the pass/fail decision is brittle — different test frameworks, locales, custom reporters, or unusual test names can produce false positives or negatives. Using the process exit code as the canonical signal is universal and correct across all tools.

This document outlines a modular architecture where pass/fail is decided by exit code, and output parsing is purely cosmetic enrichment.

Architecture

┌─────────────────────────────┐
│  Decision (exit code)       │  ← single source of truth for pass/fail
│  - Always runs              │
│  - Produces: pass/fail bool │
└──────────┬──────────────────┘
           │ feeds status icon
           ▼
┌─────────────────────────────┐
│  Display                    │
│  - Icon from exit code      │
│  - Summary from parser      │
└──────────┬──────────────────┘
           │ enriched by
           ▼
┌─────────────────────────────┐
│  Parser Registry            │  ← extensible, tool-agnostic
│  - Tries parsers in order   │
│  - First match wins         │
│  - Each parser has:         │
│    • detect(output) → bool  │
│    • summarize(output)      │
│      → str | null           │
│  - Fallback: generic text   │
└─────────────────────────────┘

Parser Module Interface

Each parser is a self-contained module that exports two functions:

Function	Signature	Role
`detect`	`(output: string) => boolean`	Returns `true` if this tool's output is present. Checks for distinctive markers (e.g., vitest's "RUN" banner, pytest's `===` separator, cargo test's "running" lines).
`summarize`	`(output: string) => string \| null`	Extracts a one-line human-readable summary (e.g., "3 passed (242ms)", "2 failed, 1 passed"). Returns `null` if no summary can be extracted.

Lifecycle

A tool execution completes (any bash command).
Check if the command is a known test command (current behaviour).
Decision: Read event.details?.exitCode — 0 means pass, anything else means fail. This is the single source of truth.
Display — icon: Set widget icon (● green or ● red) based on exit code.
Display — summary: Walk the parser registry. For each parser, call detect(output). On first match, call summarize(output) for the text. If no parser matches or summarize returns null, fall back to a generic string ("All tests passed" / "Tests failed").

Adding a New Tool

To add support for a new test framework, create a new file in the parsers directory that implements the two-function interface. Register it in the registry list. Zero changes to the core decision or display logic.

Examples of tool-specific markers for detect:

Tool	Detect marker
Vitest	`" RUN "` banner
Jest	`"PASS "` / `"FAIL "` prefixes
pytest	`"=== test session starts ==="`
Go test	`"ok "` or `"FAIL "` line prefixes
cargo test	`"running "` followed by digit
Mocha	`"passing"` / `"failing"` in summary
RSpec	`"Finished in "`

Benefits Over Current Approach

Aspect	Current (inline regex)	Proposed (modular parsers)
Pass/fail signal	Regex on output — brittle	Exit code — universal & correct
Output parsing	Inline, single format	Registry of modular parsers
Adding new tools	Edit the one function, risk regressions	Add a new file, register it
False negatives	Possible (word "failed" in test name)	Eliminated (exit code is canonical)
False positives	Possible (weird output formatting)	Eliminated
Parsing failure impact	Can break pass/fail (wrong icon)	Only loses cosmetic detail (correct icon preserved)

4.2 KiB Raw Blame History