gnomonlearn

RFC-0021 · Transform Expression Language

Versioned draft snapshot

RFCs, reference catalogs, and evidence reflect Spec 3b2e3e7a114fb826b1445d39a48756b8fff629cd. Counts and pending/status notes belong to that snapshot. Read the dated project update for later progress.

Source repositories are private during staging. Links marked “repository access required” need access to the linked repository.

Status Draft corpus. Reference catalogs and RDF projections have generation drift checks; Gnomon proofs and projection semantics remain unverified for release.

View the generated reference catalog →

Read design questions and decisions →


Reference catalog: the diagnostics, linked-RFC graph, defined terms, and normative-rule index for this RFC are in the reference catalog, generated from the structured Lean module. This document carries the prose: design rationale, entity-shape definitions in context, and migration notes.


1. Scope

This RFC defines the Transform Expression Language (TEL): the pure, total, terminating sublanguage used to express field-level data transformations in migration declarations (RFC-0016).

This RFC covers:

  • Formal EBNF grammar for transform expressions
  • Allowed operations: field access, arithmetic, string operations, conditional expressions, and type coercions
  • Explicitly prohibited operations: I/O, external references, mutation, and recursion
  • The totality-checking algorithm: how the compiler proves a transform terminates
  • Type inference rules for transform expressions: input type derived from the old field type; output type must satisfy the new field type
  • Diagnostic codes in the M21xxx range
  • Integration with RFC-0016: once this RFC is ratified, M01603 (catch-all malformed transform) is superseded by the specific M21xxx codes

This RFC does not cover:

  • General rule expressions in RFC-0008 §11 — the TEL is a strict subset and does not replace the broader expression language
  • AI-assisted transform authoring — that is an application concern
  • Runtime evaluation of transforms — the compiler produces a validated transform IR; runtime execution is an implementation detail
  • Transforms over collection-typed fields (deferred; collections introduce termination proof complexity)

2. Goals

  1. Every valid transform expression MUST be provably total: the compiler MUST statically prove that the expression terminates and produces a value of the declared output type for every possible input value.
  2. Transform expressions MUST be pure: they MUST NOT cause I/O, mutation, or observable side effects.
  3. The grammar MUST be a strict subset of the RFC-0008 §11 expression syntax, so that existing authoring tooling (LSP, formatter) requires only incremental extension.
  4. Type inference MUST be flow-sensitive within a single expression: the inferred type of a conditional branch MAY be narrower than the declared field type, provided both branches together cover the full declared type.
  5. The compiler MUST emit specific diagnostic codes (M21xxx) for each class of violation, replacing the catch-all M01603 from RFC-0016.

Linked References

The cross-RFC dependency graph is rendered in reference catalog § Linked References. Out-of-KG pointers to legacy code or external resources, when present, remain in this document under “Source-tree references” below.

3. Grammar (EBNF)

The following EBNF defines the Transform Expression Language. Terminals from RFC-0008 §11 are referenced by name without redefinition.

transform-expr   ::= conditional-expr

conditional-expr ::= "if" bool-expr "then" value-expr "else" value-expr
                   | value-expr

value-expr       ::= arithmetic-expr
                   | string-expr
                   | coerce-expr
                   | field-access
                   | literal
                   | "(" value-expr ")"

arithmetic-expr  ::= value-expr ( "+" | "-" | "*" | "/" | "%" ) value-expr
                   | "-" value-expr                             (* unary negation *)

string-expr      ::= value-expr "++" value-expr                (* string concat *)
                   | "uppercase" "(" value-expr ")"
                   | "lowercase" "(" value-expr ")"
                   | "trim" "(" value-expr ")"
                   | "substring" "(" value-expr "," int-literal "," int-literal ")"
                   | "length" "(" value-expr ")"

bool-expr        ::= value-expr ( "==" | "!=" | "<" | "<=" | ">" | ">=" ) value-expr
                   | "not" bool-expr
                   | bool-expr ( "and" | "or" ) bool-expr
                   | "isNull" "(" value-expr ")"
                   | "(" bool-expr ")"

coerce-expr      ::= "as" type-name "(" value-expr ")"
                   | "toString" "(" value-expr ")"
                   | "toInt" "(" value-expr ")"
                   | "toFloat" "(" value-expr ")"

field-access     ::= "input" "." IDENT ( "." IDENT )*
                   | "input"

literal          ::= STRING | INT | FLOAT | BOOL | "null"
type-name        ::= IDENT

3.1 Notes

  • input is the sole implicit binding in a transform expression. It refers to the current value of the field being transformed (the old value in the migration context).
  • No other variable bindings are permitted. There are no let expressions, no lambdas, and no named function definitions.
  • The grammar does not include loops, recursion, or list comprehensions. This restriction is a core design choice enabling static totality proofs.
  • as TypeName(expr) coercion succeeds when the runtime value is compatible with TypeName as defined in RFC-0006. An incompatible coercion at runtime is not possible: the type inference rules (§5) guarantee type-safety statically.

4. Allowed and Prohibited Operations

4.1 Allowed

CategoryOperations
Field accessinput, input.fieldName, nested field access via .
Arithmetic+, -, *, /, %, unary negation; operands MUST be numeric types
String++ (concat), uppercase, lowercase, trim, substring, length
Comparison==, !=, <, <=, >, >=
Booleanand, or, not, isNull
Conditionalif … then … else … (both branches MUST be present)
Type coercionas TypeName(…), toString, toInt, toFloat
LiteralsString, integer, float, boolean, null

4.2 Prohibited

The following are prohibited and produce compile errors:

ProhibitedReasonDiagnostic
External function callsI/O or non-determinism riskM21101
References to declarations other than inputPurity violationM21102
Mutation operators (=, +=, etc.)Purity violationM21103
Recursion (self-reference in any form)Termination cannot be provedM21104
Loop constructs (for, while)Termination cannot be provedM21105
Lambda or closure expressionsNot part of the TEL sublanguageM21106
null coercion without a conditional guardMay produce runtime null propagationM21107
Collection operations (map, filter, reduce)Deferred; not in this RFCM21108

5. Type Inference Rules

The compiler infers types for transform expressions using the following rules. All type names refer to RFC-0006 semantic types.

5.1 input Binding

input has the type of the field’s old (pre-migration) declared type. This type is always known at compile time from the migration declaration’s field reference.

5.2 Literal Types

Literal formInferred type
"..."String
42Int
3.14Float
true / falseBool
nullNull (bottom type; widened by context)

5.3 Arithmetic Expressions

Both operands MUST be numeric (Int or Float). If either operand is Float, the result is Float. If both are Int, the result is Int. Division (/) always produces Float. Modulo (%) requires both operands to be Int and produces Int.

5.4 Conditional Expressions

The type of if C then A else B is the least-upper-bound (LUB) of the types of A and B, as defined by the RFC-0006 type lattice. Both branches MUST be present; a missing else branch is M21110.

The inferred type MUST be a subtype of (or equal to) the field’s new (post-migration) declared type. If the LUB is wider than the new field type, the compiler emits M21111.

5.5 Coercion Expressions

as TypeName(expr) infers to TypeName. The compiler MUST verify that there exists a valid coercion path from the type of expr to TypeName in the RFC-0006 coercion table. An invalid coercion path produces M21112.

5.6 Output Type Check

After inferring the type of the complete transform expression, the compiler MUST verify that the inferred type is assignable to the new field type. Assignability follows RFC-0006 subtyping rules. A mismatch produces M21113.


6. Totality-Checking Algorithm

A transform expression is total if it is guaranteed to produce a non-error value of the declared output type for every possible input value. The compiler proves totality via the following checks:

6.1 Structural Totality

The grammar (§3) already excludes loops and recursion. Every grammatically valid expression is structurally finite. The compiler verifies grammar compliance first.

6.2 Null Safety

null is a valid value for nullable field types (RFC-0006). The compiler tracks nullability as part of the type:

  • If input has a nullable type, the expression MUST handle the null case. A conditional if isNull(input) then … else … satisfies this requirement.
  • Using a nullable input in an arithmetic or string operation without a null guard produces M21107.

6.3 Division-by-Zero

The compiler emits M21114 (warning) when a division expression has a denominator that is not statically provably non-zero. The transform is still valid, but the runtime will propagate an error value if the denominator is zero at execution time.

6.4 Coercion Safety

All type coercions are checked statically per §5.5. No coercion can fail at runtime if the static check passes.


7. Integration with RFC-0016

RFC-0016 §2.4 deferred the specification of transform expressions to this RFC and used diagnostic code M01603 as a catch-all for malformed transforms.

Upon ratification of this RFC:

  • M01603 is deprecated. Its meaning is narrowed to: “transform expression is present but the expression language version cannot be determined.” All other transform violations use M21xxx codes.
  • The RFC-0016 compiler MUST recognise M21xxx codes and incorporate them into migration validation output.
  • Tooling that displays M01603 MUST also display the deprecation notice and the superseding M21xxx code.
  • The (v) => expr lambda syntax defined in RFC-0016 §2.4.1 is deprecated as of this RFC’s ratification. TEL expressions (this RFC) are the normative replacement. The lambda form MUST NOT be used in new declarations.

8. Failure Modes

ScenarioOutcomeDiagnostic
Transform expression calls an external functionCompile errorM21101
Expression references a declaration other than inputCompile errorM21102
Mutation operator used in expressionCompile errorM21103
Recursive reference detectedCompile errorM21104
Loop construct used in expressionCompile errorM21105
Lambda or closure in expressionCompile errorM21106
Nullable input used without null guardCompile errorM21107
Collection operation attemptedCompile errorM21108
else branch missing from conditionalCompile errorM21110
Conditional LUB type wider than output field typeCompile errorM21111
Invalid coercion pathCompile errorM21112
Inferred type not assignable to new field typeCompile errorM21113
Division by potentially-zero denominatorWarningM21114

9. Invariants

  1. Totality. Every transform expression that compiles successfully is provably total: it terminates and produces a value for every possible non-error input.
  2. Purity. Transform expressions have no observable side effects. They are referentially transparent given the same input.
  3. Grammar subset. The TEL grammar is a strict syntactic subset of RFC-0008 §11. Every valid TEL expression is a valid RFC-0008 expression, but not vice versa.
  4. Static type safety. The output type of a compiled transform is statically guaranteed to satisfy the new field’s declared type. No type error can occur at transform execution time for a statically type-checked transform.
  5. M01603 supersession. Once this RFC is ratified, M21101–M21114 replace M01603 for all transform-specific violations. M01603 is retained only for version-indeterminate malformed transforms.
  6. No runtime failures from null. Any nullable input that is not statically guarded will produce a compile error (M21107), not a runtime null-pointer error.

10. Diagnostic Codes (M21xxx)

All diagnostics introduced by this RFC occupy the M21100–M21199 range.

CodeSeveritySummary
M21101ErrorExternal function call in transform expression; only built-in operations are permitted.
M21102ErrorReference to a declaration other than input in transform expression; purity violation.
M21103ErrorMutation operator in transform expression; transforms must be pure.
M21104ErrorRecursive reference detected in transform expression; recursion is not permitted.
M21105ErrorLoop construct in transform expression; loops are not permitted.
M21106ErrorLambda or closure expression in transform; not part of the TEL sublanguage.
M21107ErrorNullable input used in operation without null guard; may produce null propagation.
M21108ErrorCollection operation in transform expression; collection transforms are deferred.
M21109—Reserved.
M21110ErrorConditional expression is missing the else branch; both branches are required for totality.
M21111ErrorInferred LUB type of conditional branches is wider than the output field type.
M21112ErrorInvalid coercion path: no coercion exists from source type to target type.
M21113ErrorInferred expression type is not assignable to the new field’s declared type.
M21114WarningDivision by a denominator that is not statically proved non-zero.