RFC0013 REPORT CONTRACT PROPOSAL
Versioned draft snapshot
RFCs, reference catalogs, and evidence reflect Spec 3b2e3e7a114fb826b1445d39a48756b8fff629cd. Counts and pending/status notes belong to that snapshot. Read the dated project update for later progress.
Source repositories are private during staging. Links marked “repository access required” need access to the linked repository.
This is a trace and gap record from the pinned draft specification. It does not establish proof or implementation conformance.
Status: proposed, awaiting owner decision. Owners: DX, Compiler, CI. No normative RFC, RDF, or implementation behavior changed.
Single diagnostic document
The official TAP 14 specification defines one optional YAML diagnostic document per test point. Replace RFC-0013 §7.1’s multiple documents with one document holding all failure entries:
TAP version 14
1..1
not ok 1 - household constraints
---
failures:
- kind: assertion
message: status mismatch
assertion: Household.status == active
expected: active
actual: pending
- kind: assertion
message: too few members
assertion: Household.members.length >= 1
expected: 1
actual: 0
...
Keep one result line per selected Gnomon test. Assertion failures remain independently evaluated and ordered by source position. Preconditions and timeouts are failure entries with their diagnostic codes, not fake assertions. Expected/actual fields are present only when known; do not invent values. Escape test names for TAP control characters and serialize YAML through a real encoder, including multiline expressions and strings containing markers.
The proposed boundary schema is proto/proposals/test_report.proto; it compiles
with protoc. It is a proposal, not a released DTO or implemented renderer.
A validator must reject unspecified status/kind, null Value payloads, invalid
ordinal sequences, and inconsistent counts before emitting TAP.
Discovery, filtering, and plan cardinality
Compile all discovered declarations before execution, including filtered-out ones, as §6.1 currently requires. A compile failure exits 2 without executing actions or manufacturing passing TAP results.
After successful compilation, filtering selects tests in deterministic order. The TAP plan N is selected_count, not discovered_count. filtered_out_count plus selected_count equals discovered_count. Filtered tests do not produce result lines; tests skipped after fail-fast do, with an explicit SKIP directive. The results length equals selected_count, with contiguous ordinals 1 through N. A zero selection emits a valid zero-test plan with an explanatory comment. Required CI remains unfiltered and cannot silently substitute zero selected tests for conformance; zero discovery behavior needs an explicit governance rule.
Acceptance after decision
- Update §7.1 output rule and count wording, Lean/RDF facts, and fixture schema.
- Render two failing assertions as one YAML document under one result line.
- Parse the exact output through a pinned TAP14 consumer and check every failure survives; schema compilation alone does not establish parser compatibility.
- Verify filtering, fail-fast, Unicode/control-character names, multiline YAML, zero selection, compile failure, timeout, and precondition diagnostics.
- Keep report schema versioning and serialized artifacts bound to the same contract; do not report these fixtures as an implemented DSL test harness.
Proposal envelope verification
tools/check_proposed_test_report.py validates proposed protobuf text envelopes:
count reconciliation, contiguous ordinals, nonempty result identities, status
and failure consistency, skip reasons, precondition/timeout diagnostic codes,
and recursively non-null finite diagnostic values. Five focused tests pass,
including filtering with fail-fast and malformed nested values. This tool is
not wired into normative release gates while the contract decision is pending.
It does not render TAP, execute actions, or prove consumer compatibility.
Reviewable renderer prototype
tools/render_proposed_test_report.py renders validated proposal envelopes.
It emits one YAML document containing every failure, preserves diagnostic values
through YAML serialization, and escapes control characters and TAP directive
markers in names. Selected counts drive the plan; fail-fast skips remain result
lines. Zero selection has an explanatory zero-test plan.
Nine envelope/renderer tests pass, including multiline and marker-like values, Unicode names, injected SKIP text, filtering, skipped results, and invalid reports. These tests verify the prototype’s structure and YAML round-trip. A pinned TAP14 consumer test and an actual DSL harness remain required. No existing normative CLI output or RFC text was changed by this prototype.
Pinned consumer evidence
The renderer fixture tools/fixtures/proposed-test-report/multiple-failures.textproto
was parsed by tap-parser 18.3.4 in strict mode. The consumer preserved two
failure entries, multiline values and expressions, the M01319 code, one fail-fast
SKIP result, and the selected count of two. No non-TAP data was reported.
The failed test correctly makes the overall test outcome fail; the checker
separately verifies that no additional parser failure was introduced.
Package metadata retrieved from npm reports integrity
sha512-CiqzdpWn2CvONcWp7UNMF9/rCPJwCz0es+qykkgJruu1Y/rAS8A5MEQujmjx9NErfst3dGiZJU3lDS2jBsgbPA==.
The package was installed under /tmp/gnomon-tap-consumer with lifecycle scripts
disabled; no project dependency or normative build graph was changed.
The exact direct dependency and transitive dependency versions and integrity
hashes are saved in tools/fixtures/tap-consumer/package-lock.json. A clean
npm ci --ignore-scripts installation from this lockfile also passed the check.
Reproduce installation, rendering, and consumer checks with:
npm ci --prefix tools/fixtures/tap-consumer --ignore-scripts --no-audit --no-fund
python3 tools/render_proposed_test_report.py tools/fixtures/proposed-test-report/multiple-failures.textproto > /tmp/gnomon-proposed-multiple.tap
node tools/check_proposed_tap_consumer.cjs tools/fixtures/tap-consumer/node_modules/tap-parser /tmp/gnomon-proposed-multiple.tap
The checker requires exactly parser version 18.3.4. Its upstream API is documented in the tapjs parser. This single concrete consumer check establishes the fixture’s compatibility, not universal TAP consumer support, action execution, or an implemented DSL CLI.