gnomonlearn
Docs/Canonical AST fixture contract proposal

Canonical AST fixture contract proposal

Versioned draft snapshot

RFCs, reference catalogs, and evidence reflect Spec 3b2e3e7a114fb826b1445d39a48756b8fff629cd. Counts and pending/status notes belong to that snapshot. Read the dated project update for later progress.

Source repositories are private during staging. Links marked “repository access required” need access to the linked repository.

This is a trace and gap record from the pinned draft specification. It does not establish proof or implementation conformance.

Status: proposed first-release contract. Owners: Language, Compiler, DX. The owner approved AST expectations for successful fixtures and diagnostic expectations for failing fixtures. The AST stage still needs a decision.

Source evidence

RFC-0008 §8 requires parse fixtures with expected canonical ASTs. RFC-0012 §6.3 compares the extracted grammar and compiler against the same expectation. RFC-0012 §4 preserves sugar and works before desugaring; §5.1 also identifies post-desugar output as useful for generating conformance fixtures. RFC-0008 §8.21 expands query shorthand in compiled IR. These stages have different structures even when they represent the same program.

RFC-0010 §4 defines normalized semantic facts extracted from other codebases and explicitly distinguishes that IR from the authoring format. Its existing Rust serialization contract should remain with that IR. It does not establish a grammar AST schema. The legacy Lean AST is a permissive generic block model; it cannot establish RFC-0008 production conformance.

Use a typed parsed AST for grammar fixtures. Preserve declaration and body entry order, shorthand form, explicit versus omitted properties, and decorator form. Normalize incidental token whitespace and omit source locations from canonical comparisons. Keep locations in a separate diagnostic/source map. Decoded literal values belong in the AST; formatter trivia belongs in a separate concrete syntax representation when needed for comment preservation.

A separate desugar-fixture family should compare parsed input against the post-desugar AST, including query shorthand expansion and synthesized defaults. Grammar fixtures should stop at the compiler’s parse stage: a syntactically valid input can still require semantic diagnostics. Full compiler success belongs in semantic fixtures with enough module and symbol context.

Use concrete protobuf messages with typed declaration and expression variants for the AST interchange. Do not use untyped maps or a production-name tree as the canonical AST. The existing fixture envelope can carry the concrete root message in Any; descriptor resolution is mandatory. This proposal establishes the stage and preservation rules, not the final field-number assignments.

The independent framework frontend seam now has a reproducible pinned native probe. It exposes generic-tree losses, legacy rejection differences and absent source mappings. The division-token stall is fixed in the current producer pin, with the old evidence preserved; it does not supply a typed compiler adapter or change the pending stage choice.

Executable schema candidate

The parsed-stage recommendation now has a typed protobuf schema and actual generated-grammar lowering under gnomon.spec.proposal.syntax.v1; see the candidate inventory and handwritten fixtures. All 36 authored declaration kinds are implemented, with exact numbers, recursive types, ordered properties, explicit presence, and retained query/decorator sugar. Thirty-two AST expectations and three proposal rejections pass, retaining 175/180 productions. A separate source artifact binds exact comments, top-level UTF-8 ranges, and unambiguous leading doc owners to the source digest without changing AST comparisons or inventing node IDs. Remaining documentation ownership, compiled metadata, full formatter mappings, desugaring/semantics, and an independent compiler adapter remain open. Five catalog productions remain unreachable from the generated start rule. Domain bodies and rule statements also retain field sugar, optional blocks, ordered transitions, and an explicit authored expression chain without selecting operator precedence. The inventory records observed qualified-name/postfix spacing, double-newline precondition, unreachable client status-family, and repeated read-scope keyword gaps for grammar review. Operation bodies use typed assignments, effects, retries, failures, client returns/errors, and opaque foreign scripts, preserving order and optional presence. Typed control and composition forms preserve saga/workflow step keywords, handler terminal presence, aspect targets/contributions, policy slots, invariant parameters, precedence lists, and apply/extend roles. View/layout/application/agent/model structures also retain typed body variants, optional targets/widgets/scopes, show commas, ordered entries, and exact model parameter number/text alternatives. Manifest/test/annotation declaration candidates preserve current authored forms; the separate import, test, formal filename, and RDF identity decisions remain pending. Whole-form coverage does not establish whole-language conformance. This candidate makes the proposed stage reviewable; owner approval of that stage and the canonical interchange identity remains pending.

Alternative

Use post-desugar ASTs for every successful grammar fixture. The grammar runner would then need the same desugaring implementation as the compiler, and fixtures would intentionally collapse distinct shorthand forms. A separate parsed or concrete syntax representation would still be needed by the formatter. This would combine syntax and transformation checks in the grammar gate.

Acceptance

  1. Record the owner’s AST-stage decision in RFC-0012 Q&A and align §6.3, §5.1, RFC-0008’s fixture paragraph, Lean, RDF, and emitted references.
  2. Define concrete AST messages for every authored declaration kind, type, expression, annotation, and body entry. Document source-order and literal normalization rules and distinguish absent values from explicit defaults.
  3. Implement grammar and compiler parse-stage adapters with identical AST output. Reject unsupported variants rather than emitting generic fallback nodes. Maintain a separate concrete syntax/source map for formatting.
  4. Author positive and negative fixtures for every nontrivial production; validate coverage against accepted parse trees. Add separate desugar and semantic fixtures and run all required gates in both repositories.
  5. Keep the normative release gate red until full compiler conformance is implemented. Envelope validation and syntax probes are insufficient.