gnomonlearn
Docs/Executable parsed-AST proposal

Executable parsed-AST proposal

Versioned draft snapshot

RFCs, reference catalogs, and evidence reflect Spec 3b2e3e7a114fb826b1445d39a48756b8fff629cd. Counts and pending/status notes belong to that snapshot. Read the dated project update for later progress.

Source repositories are private during staging. Links marked “repository access required” need access to the linked repository.

This is a trace and gap record from the pinned draft specification. It does not establish proof or implementation conformance.

Status: proposal, not canonical language or compiler conformance. Owners: Language, Compiler, and DX. The comparison-stage decision remains pending in CANONICAL_AST_CONTRACT_PROPOSAL.md. This corpus makes that recommendation reviewable with concrete protobuf fields and actual generated-grammar lowering. It does not establish the proposed gnomon.lang.syntax.v1 identity.

Run and inspect

python3 tools/check_proposed_syntax.py
python3 -m unittest discover -s tools/tests -p 'test_proposed_syntax.py'
python3 tools/check_proposed_syntax.py --descriptor-out /tmp/gnomon-proposed-syntax.desc
node tools/parse_proposed_syntax.mjs grammar/proposals/parsed-ast-fixtures/query-forms.gnm

The last command emits one ProposalParseResult in protobuf text format. The schema is parsed_syntax.proto (repository access required). The descriptor is generated with protoc, not a handwritten descriptor or JSON schema. .expected.textproto files were authored by hand, not generated from the lowering implementation.

The checker compares decoded typed messages and validates UTF-8 byte spans. It reports 32 AST expectations, three proposal rejections, and 175/180 retained syntax productions. The production count is scoped to this corpus; it is not whole-language AST coverage. The positive fixtures exercise all 36 implemented declaration kinds. Eighty regressions verify distinct authored forms and refusal behavior. The checker and regressions run in spec CI as proposal checks, separate from the normative release gate.

Implemented declaration inventory

Authored productionCandidate messagePreserved distinctions
import-declImportDeclarationLiteral import text, Unicode, declaration order
types-declTypeDeclarationOrdered label/meaning/annotation properties, repeated properties, empty annotation blocks
config-declConfigDeclarationAll nine property forms in source order, explicit false, omitted properties, nested exact literals and types
config-set-declConfigSetDeclarationQualified name parts, ordered literal map entries, duplicate keys, empty lists/maps
query-declQueryDeclarationOrdered and repeated modifiers, shorthand/braced forms, parameter/field order, omitted/empty input, ordered properties
entity-declEntityDeclarationOrdered properties, bare fields versus field blocks, repeated/empty blocks, includes/examples/annotations
facet-declFacetDeclarationRequired meaning presence, optional requires/rules, authored rule order and model presence
relation-declRelationDeclarationOrdered endpoints, absent versus explicit cardinality, optional/empty qualifiers and constraints
flow-declFlowDeclarationOrdered/duplicate states and transitions, transition action names, absent/empty transitions and rules
claim-declClaimDeclarationOrdered/repeated properties, policies, exact policy numbers, evidence/status lists and annotations
enum-declEnumDeclarationOrdered/duplicate enum values, explicit empty meaning, ordered modifiers
action-declActionDeclarationOrdered properties, result/client/operation bindings, duplicate assignments, mutation references, absent/empty emit using blocks
mutation-declMutationDeclarationTyped effects in source order, optional state/binding/condition/member, link endpoints, recursive conditional blocks
actuator-declActuatorDeclarationAll 13 property forms, delivery/notification forms, exact retry numbers and ordered failure statements
procedure-declProcedureDeclarationOrdered properties, capability/target/binding/field preconditions, expression preconditions, runs assignments
event-declEventDeclarationOrdered properties/fields, duplicate and ordered emitted/subscribed identifiers
task-declTaskDeclarationOrdered input/effects/error/retry/after blocks, explicit empty effect and error blocks
compute-declComputeDeclarationAuthored expression chain, opaque language-tagged script text including Unicode, spacing and comments
client-declClientDeclarationOrdered operations/properties, all HTTP method forms, type/void/field return forms, exact error keys and classifications
saga-declSagaDeclarationOrdered properties and named expressions, step/compensate keyword retained in either authored container
workflow-declWorkflowDeclarationOrdered steps/compensations, typed calls/retries/handlers, optional terminal marker, matching container restrictions
policy-declPolicyDeclarationOrdered applies/rules/enforcement/annotation properties, repeated enforcement, absent versus empty blocks
invariant-declInvariantDeclarationOrdered and repeated parameters, recursive types, authored expression chain and parentheses
aspect-declAspectDeclarationAny-write/field-list/named targets, ordered emit/require/log/contribution statements, duplicate assignments and empty contributions
compose-declComposeDeclarationOrdered and repeated includes, exact precedence order and duplicates, singleton precedence list
apply-stmtApplyStatementDistinct applied/target qualified names, decorator syntax, statement order
extend-declExtendDeclarationAuthored unqualified name, ordered and duplicate include identifiers, decorator syntax
view-declViewDeclarationOptional for-type, ordered properties/layout/region entries, show comma presence, widget presence, typed navigation arguments/routes
app-declAppDeclarationOrdered properties, all five include categories, repeated groups and duplicate names, empty version text
agent-declAgentDeclarationOrdered inputs/outputs/rules/scopes, output state names, omitted versus empty scope fields, repeated names
model-declModelDeclarationOrdered properties/roles/parameters, numeric versus text values, exact decimals, duplicate names and empty blocks
project-declProjectDeclarationLiteral name/version presence, absent/empty imports/modules, optional string commas, ordered/repeated compiler properties
module-declModuleDeclarationDNS-style name labels, all three type choices, absent/empty blocks, duplicate imports/paths, string/structured operation sugar and rollback presence
test-declTestDeclarationRequired ordered given/when/expect blocks, optional state, operation/transition attempt roles, ordered verdict/reason alternatives
annotation-facet-declAnnotationFacetDeclarationOptional visibility, all target kinds, ordered constraints/fields, builtin/named types and absent/false/empty defaults
annotation-aspect-declAnnotationAspectDeclarationOptional visibility, all four target forms, ordered predicate atoms, recursive types and explicit annotation block

Declaration has these 36 explicit variants and no generic fallback. decorated-decl retains decorator order and distinguishes @Name from an @[A, B] group. This records syntax without deciding which annotations are valid on a declaration or synthesizing their semantic defaults.

This inventory accounts for all 36 alternatives in bare-top-level-decl. Every current declaration has an explicit typed candidate. The guard for any future unsupported declaration remains a whole-input refusal, and variants outside the current authored grammar produce SYNTAX_ERROR, never a generic successful AST. A separate digest-bound source artifact now preserves comments, top-level ranges, and unambiguous leading doc ownership. Full formatter mappings, compiled doc metadata, desugaring, symbol resolution, semantics, and an independent compiler remain open; see source preservation.

Shared typed structures

TypeExpression retains named, ordered union, recursive List/Claim, inline enum, and per-level required/optional/omitted presence. The parsed candidate does not resolve qualified names or apply type defaults. Literal uses typed alternatives for strings, Booleans, exact decimal spelling, ordered lists, and ordered map entries. Negative numeric syntax is normalized by removing spacing between the sign and digits; decimal digits, leading zeros, and trailing zeros remain intact. No runtime Number representation is chosen.

The authored STRING production has no escape-sequence rule. Its quoted contents are retained literally: a backslash followed by n remains those two characters, while Unicode code points remain Unicode. The prototype does not silently import JSON string-escape semantics into Gnomon source. Textproto escaping is only serialization of that retained value.

Annotation values retain string/number/Boolean/identifier variants. Ordinary comments and permitted spacing are incidental. Doc-comment attachment in Gnomon syntax is unimplemented, so DOC-COMMENT produces UNSUPPORTED_DOC_COMMENT with its byte span. Dropping an authored doc comment would violate the proposed preservation requirement. A separate source map/trivia artifact is still required for diagnostics and lossless formatting; the candidate AST has no source locations.

Domain bodies and authored rules

Entity properties retain their order and distinguish bare field sugar from an explicit fields block. The candidate does not move fields between those forms. Facet/relation/flow bodies retain required authored values and the presence of optional blocks. An omitted relation cardinality receives no default. Flow states and transitions, enum values, claim properties, and claim policy entries retain duplicates and order. Those syntax distinctions do not decide whether a duplicate is semantically valid or how a lifecycle or claim is evaluated.

Rule blocks have explicit variants for compiled natural-language rules, authored expressions, runtime prompts, and documentation notes. Optional model names are not filled in. Expression retains the authored unary-expr { binary-op unary-expr } chain as a first operand and ordered typed operator/operand tails. It does not assign precedence or association. Unary negation, parentheses, postfix member/invocation forms, argument order, literals, qualified/context names, and every builtin form in primary-expr have explicit typed variants. Quantifier parameters and predicates remain distinct from their collection expressions. No expression or rule is executed by this candidate.

The current generated PEG also exposes a lexical/ordered-choice gap that still needs grammar and AST review: record.field becomes a qualified-name primary, while record . field becomes a primary plus a postfix member. A direct probe accepts record . method() but rejects record.method(). The prototype retains the selected parse and does not redefine those source forms or claim broader whitespace equivalence. Its spaced-postfix fixture makes the actual typed shape reviewable. Resolving this gap is required before canonical AST promotion; passing the scoped fixtures is not proof that the generated PEG implements all intended expression syntax.

Operation bodies

Bindings keep result, client, operation, mutation, and assignment names distinct. Assignment and emit lists retain order and duplicates. An omitted emit using block differs from an explicit empty block. Effects have ten typed alternatives, with source-order preservation and nested conditional effect blocks; state, binding, condition, and member presence remain explicit. Retry properties retain order, repeated entries, exact numeric spelling, and each authored backoff form. No retry defaults, validity policy, or failure behavior are synthesized. Failure blocks distinguish ordered emissions from log expressions. Session preconditions retain capability, optional target/binding/field, and expression alternatives without evaluating or resolving them.

Client operations retain all five HTTP methods and distinguish a named type, void, and a field-block return, including an explicitly empty field block. Error entries preserve exact numeric spelling, duplicates, timeout, and each classification. The schema also has the authored status-family alternative; the current generated parser rejects 4xx: transient because NUMBER wins before NUMBER "xx" in the ordered choice. No positive status-family fixture is claimed. Procedure preconditions have another observed gap: both precond-stmt and its enclosing repetition require NEWLINE, so the current parser accepts ready followed by a blank line but rejects a single newline. Focused regressions record these behaviors as grammar-review evidence, not new normative restrictions. Resolving both gaps is required before promotion.

Foreign scripts use the dedicated authored script IDENT { ... } variant. The language tag and exact opaque body text, including CRLF, Unicode, comments, and spacing, are retained. Script text is neither Gnomon documentation nor a generic fallback for an unsupported declaration. Translator spacing/comment matches inside that opaque body are excluded from Gnomon production coverage; actual header provenance remains retained. Embedded-language parsing and execution are outside this candidate. The current grammar does not model balanced braces or embedded-language string syntax; script delimiter behavior still needs review before canonical promotion.

Control and composition bodies

Workflow and saga properties and steps retain source order and duplicates. Saga’s authored grammar permits both step and compensate in either the steps or compensations container; their keywords remain explicit variants. Workflow’s narrower grammar requires the matching keyword for each container. No container, dependency, or compensation meaning is inferred. Named saga expressions use the explicit authored IDENT ":" expr alternative, including when its name matches a fixed slot but its expression selects that alternative. They are not an unknown-property fallback for other bodies.

Step properties keep calls, required-name lists, retries, success handlers, and failure handlers distinct. The presence of (terminal) is retained as a marker, with no implied value when omitted. Handler statements have typed mutation, transition, emit, and compensation alternatives, preserving assignment order, empty blocks, and emit-using absence. Policy enforcement remains an authored hard/soft variant; no default or effective rule is evaluated. Invariants retain nonempty ordered parameter lists and the same recursive authored expression representation as other bodies, without choosing operator precedence.

Aspects distinguish any write, the authored parenthesized field list, and a named target. Emissions, requirements, logs, and contribution clauses remain ordered; an empty contribution differs from a contribution with an empty properties block. Composition includes and precedence lists retain order and duplicates without conflict resolution. Apply statements keep their two qualified-name roles separate. Extend statements retain the authored simple name and include list. Apply/extend have no declaration modifiers in the authored grammar; the proposal rejects modifier-prefixed variants rather than inventing that syntax. Semantic target validity, execution, authorization, compensation, composition, and desugaring remain compiler responsibilities.

Views, applications, agents, and models

Views retain an omitted for target, every ordered property, and optional comma markers on each show entry, including the final entry. Explanations and actions retain duplicate names and explicit empty labels. Layout sections, regions, extension points, sublayouts, and hints have distinct typed alternatives. Mixed region items and sublayouts retain order. Field claim-widget presence and trigger/navigation button forms remain distinct. Navigation arguments preserve their ordered assignments and authored expression types; the grammar requires at least one argument. Exact column spelling and section/extension sequences are retained without imposing layout behavior. The current grammar limits region contents to items and sublayouts; the candidate rejects nested sections rather than adding a recursive layout syntax by assumption. Routes preserve ordered path, condition, and fallback variants without deciding route behavior.

Applications retain all five authored include categories and repeated groups. Agents retain their purpose/model syntax, input/output order, state names, and separate permission/prohibition/requirement rule blocks. Scope entries keep resource order and distinguish omitted field restriction from an explicit empty field list, including repeated field names. A concrete authored/generated grammar gap remains: agent-prop requires "readScope" read-scope-block, and the latter begins with "readScope" again. The current parser accepts readScope readScope {} and rejects readScope {}. The fixture and regression record that behavior as review evidence; the prototype does not resolve this contract inconsistency. Language/Compiler review is required before promotion.

Model parameters retain the authored number/text alternatives, exact decimal spelling, string contents, duplicate keys, order, and empty blocks. Model property name stays separate from the declaration name. Roles and annotations retain their authored forms. No provider resolution, model defaults, agent capability evaluation, claim wrapping, or application activation is performed. Diagnostic locations retain UTF-8 byte spans. A separate protobuf source artifact binds exact comments and top-level declaration ranges to the source digest; AST comparisons remain location-free. Complete formatter source maps and the canonical comparison-stage decision remain pending.

Manifests, tests, and annotation declarations

Project and module manifests retain fixed authored field order and optional block presence. Project names/versions remain literal strings, including explicit empty text accepted by the grammar. String-sequence comma markers, duplicates, and order remain visible. Compiler properties retain repeated entries and order; a present compiler block must be nonempty. Module names and import keys use DNS-style labels, distinct from DSL QUALIFIED identifiers. Import constraint strings are not resolved or interpreted. String operation entries differ from structured entries with the same path, and rollback checkpoint false differs from omission. Dependencies without a checkpoint and explicit empty dependency lists remain authored syntax without an inferred rollback meaning.

The test candidate retains RFC-0008’s required ordered given/when/expect blocks, binding/state presence, operation versus transition attempts, and ordered allowed/denied/reason statements. Empty blocks remain present. It does not execute tests, interpret a combination of verdicts, or resolve action versus procedure authorization. RFC-0013’s competing then/mock contract remains pending in its proposal. Likewise, the import syntax conflict, formal manifest filename, and module-choice RDF identities remain owner decisions; supporting current authored syntax approves none of those alternatives.

Annotation facets retain optional single visibility, all 18 authored target kinds, constraint order, builtin versus named field types, and optional values without synthesizing defaults. False and empty text remain explicit values. Annotation aspects distinguish entity/facet/action filters from a named target, preserving the ordered and predicate atoms and recursive type syntax. Their inner annotations block remains explicit. Target applicability, default/type compatibility, propagation, merge rules, and materialized effects are not evaluated. The target-kind list includes api, serve, variant, and integration; it does not introduce additional top-level declaration syntax.

Uncovered retained productions

The five productions absent from accepted proposal fixtures are WHITESPACE, nat-lang-block, nat-lang-stmt, qualified-set, and query-shorthand. All five are unreferenced from the generated start rule: whitespace uses translator _/__, query sugar is reached through query-decl, and the legacy natural-language and qualified-set rules have no root-reachable caller. Positive handwritten fixtures now establish ordinary line/block and leading doc-comment provenance. Foreign script comments cannot establish Gnomon comment coverage. Language/Compiler must reconcile coverage scope or grammar reachability before the conservative full-production gate can pass. No annotations manufacture missing counts.

Separate source preservation

The source artifact proposal records exact UTF-8 ranges and comment spelling in a separate digest-bound protobuf. Six independently authored source expectations and twenty source regressions verify text, Unicode boundaries, order, explicit ordinal presence, digests, opaque script exclusion, and refusals. Consecutive leading /// lines directly above a bare unmodified declaration have an unambiguous owner under RFC-0008 §8.1 and RFC-0012 §4.3. Their source/comment ordinals are recorded separately from the location-free AST. Decorator/modifier adjacency, blank-line or member ownership, compiled bundle doc metadata, CommonMark diagnostics, and full formatter mappings remain open. The existing documentation refusal fixture now uses a separating blank line. The candidate invents no canonical node IDs.

The adapter lowers internal retained grammar nodes into these typed messages; the nodes and Peggy arrays never cross the serialized AST boundary. Accepted production names come from the existing backtracking-aware instrumentation. Optional runtime locations support diagnostics, and UTF-16 parser offsets are converted to UTF-8 byte offsets. Rejected inputs contribute no successful AST coverage. Proposal-local codes are not normative compiler M-codes.

Exact decision and remaining gates

The existing owner decision request is whether grammar fixtures compare the authored parsed AST before desugaring, with locations/trivia separated, or the post-desugar AST. This candidate makes the recommended parsed choice concrete: the two query forms remain different trees, explicit false differs from omission, and property order remains visible. Approving that stage does not automatically approve every field number or establish whole-language coverage.

Before promotion, Language/Compiler/DX must review all production-to-message assignments and remaining documentation ownership/metadata, establish the canonical namespace and descriptor publication, resolve documented grammar gaps and uncovered rules, and implement an independent compiler frontend adapter. The parser and compiler must then run against the same handwritten canonical fixtures and normative diagnostics through run_conformance_fixtures.py, with full accepted-production coverage. Shared expected-output generation or passing this parser as both adapters would not satisfy that requirement.

grammar/fixtures/ intentionally still contains no canonical corpus. check_fixture_envelopes.py --require-corpus still fails, and check_release.py still reports the compiler conformance gap. No RFC, Lean, RDF, public language identity, or compiler behavior changes in this proposal.