Discovery

Source file: src/discovery/tests.ts

Discovery is the first active step in the pipeline. Its job is to scan the configured test directory, find all files that represent test specifications (any file type — .md, .txt, .pdf, .json, etc.), read their full contents into memory, and return them as a SemanticTest[] array. This in-memory array is what drives everything downstream — the prompt builder reads test content from it, the executor iterates over it, and reports reference its file paths.

There are two modes: directory scan (the default, when you run semtest run with no arguments) and specific file resolution (when you pass file paths as CLI arguments, like semtest run auth.md api.md). Both modes produce the same SemanticTest[] output, so the rest of the pipeline doesn’t know or care which mode was used.

SemanticTest type

Every discovered test file is represented as:

interface SemanticTest {
  name: string;      // Filename (e.g. "auth-middleware.md", "api-routes.txt")
  filePath: string;  // Absolute path
  content: string;   // Full file content (read into memory)
}

Two discovery modes

discoverTests(testDir, extensions?)

Used when no specific files are passed on the CLI (the default). It:

1. Resolves testDir to an absolute path
2. Reads the directory entries
3. Filters to files only, excluding dotfiles
4. If extensions is provided, filters to matching extensions (e.g. [".md"])
5. Sorts alphabetically
6. Reads each file’s content into memory

Throws if the directory doesn’t exist or contains no matching files.

resolveTests(filePaths, testDir)

Used when specific file paths are passed as CLI arguments (e.g. semtest run auth.md api.md). For each path:

1. Tries to resolve the path as-is (absolute or relative to cwd)
2. Falls back to looking inside testDir
3. Throws if neither location exists

This allows users to pass either full paths or just filenames.

Error handling

Condition	Error
Test directory doesn’t exist	"Test directory not found: /absolute/path"
No test files found	"No test files found in /absolute/path"
Specific file not found	"Test file not found: filename"