Reports

Source files: src/report/markdown.ts, src/report/json.ts

The report module produces persistent output after tests complete. Two formats serve different audiences. The Markdown report (latest.md) is for humans — it shows what failed, the expected versus observed behaviour, the file location, and a suggested resolution. The JSON report (ci-results.json) is for CI/CD pipelines — a machine-readable pass/fail signal with just enough data to identify failures and their locations.

The latest.md file is overwritten on each run, giving a stable path that bookmarks and scripts can rely on. With the --timestamp flag, timestamped copies are also generated for historical tracking without losing the convenience of the stable path.

Markdown report

Generated by generateMarkdownReport(). Written to {output}/latest.md (and optionally a timestamped copy).

Structure

# Semantic Test Report

Run Timestamp: 2025-01-15T10:30:00.000Z
Project: semtest-runner

## Summary

| Metric | Count |
|---|---|
| Total Tests | 12 |
| Passed | 10 |
| Failed | 2 |

---

## Test Results

### FAIL auth-check

Source: auth-middleware.md
Status: FAIL

Expectation: ...
Observed: ...
Location: ...
Resolution: ...

---

## Validation Issues

- **duplicate-id**: ...

---

Sections

Section	Always present	Content
Header	Yes	Timestamp and project name
Summary table	Yes	Counts for total, passed, failed (+ errored, skipped, invalid if > 0)
Test Results	Yes	One subsection per test (passing tests excluded unless --include-passing)
Validation Issues	Only if issues exist	Bulleted list of validation issues

Options

Option	Effect
includePassing	Passing tests appear in the Test Results section
validation	If provided and has issues, a Validation Issues section is appended

JSON report

Generated by generateJSONReport(). Written to {output}/ci-results.json.

CIResult shape

interface CIResult {
  status: "pass" | "fail" | "error";
  summary: {
    total: number;
    passed: number;
    failed: number;
    errored?: number;    // omitted if 0
    invalid?: number;    // omitted if 0
    skipped?: number;    // omitted if 0
  };
  tests: CITestEntry[];
  validation?: ValidationResult;  // omitted if not provided
}

CITestEntry shape

interface CITestEntry {
  id: string;
  sourceFile: string;
  status: TestStatus;
  location?: string;   // fail only
  error?: string;      // error only
}

The JSON report is intentionally minimal — it includes only the fields CI systems need to determine pass/fail and locate problems. Full diagnostic details are in the Markdown report.

Output paths

File	Path	When
Latest report	{output}/latest.md	Always
CI results	{output}/ci-results.json	Always
Timestamped report	{output}/{timestamp}.md	Only with --timestamp
Debug logs	{output}/debug/{testname}.json	Only with --debug