RFC-0020 · Reactive Query and Subscription Model
Versioned draft snapshot
RFCs, reference catalogs, and evidence reflect Spec 3b2e3e7a114fb826b1445d39a48756b8fff629cd. Counts and pending/status notes belong to that snapshot. Read the dated project update for later progress.
Source repositories are private during staging. Links marked “repository access required” need access to the linked repository.
| Status | Draft corpus. Reference catalogs and RDF projections have generation drift checks; Gnomon proofs and projection semantics remain unverified for release. |
|---|
View the generated reference catalog →
Read design questions and decisions →
Reference catalog: the diagnostics, linked-RFC graph, defined terms, and normative-rule index for this RFC are in the reference catalog, generated from the structured Lean module. This document carries the prose: design rationale, entity-shape definitions in context, and migration notes.
1. Scope
This RFC defines the reactive subscription layer on top of the Gnomon query API. It specifies:
- The
gnomon.subscribe(query, callback)API — a reactive wrapper over the RFC-0007 point-in-time query API - Delivery semantics: ordering, deduplication, and at-least-once guarantees
- Subscription lifecycle: open, pause, close transitions
- Back-pressure handling: max pending events and overflow policy
- Integration with the RFC-0010 event log (subscriptions tail the event log)
- Security: read-scope enforcement for subscriptions
This RFC does not cover:
- The point-in-time query API itself — that is defined in RFC-0007
- The event log storage format or commit protocol — defined in RFC-0010
- DSL-level subscription declaration syntax — this RFC is runtime-only; no
.gnmauthoring surface is introduced (deferred to a future RFC) - WebSocket or SSE transport details — those are implementation concerns; this RFC specifies the logical delivery contract only
- Cross-module subscription: a subscriber cannot observe events from modules it does not have readScope access to
2. Goals
- The subscription API MUST be a pure overlay on the RFC-0007 query model; it does not introduce new storage concepts.
- Events MUST be delivered in emission order within a single entity. Cross-entity ordering is best-effort.
- Every committed event matching the subscription query and effective read scope MUST eventually be delivered to an open, non-overflowed subscription. An overflow may drop an event only with M20003 or M20004 as specified in §6.1.
- The API MUST enforce
readScopeconstraints from RFC-0008 §8.10 at subscription open time and at each delivery boundary. Events that fall outside the subscriber’s read scope MUST be silently filtered, not produce an error. - Back-pressure MUST be explicit: a subscriber that cannot consume events fast enough MUST receive a defined overflow signal rather than silent event loss.
- Subscription lifecycle transitions MUST be deterministic and observable for debugging.
Linked References
The cross-RFC dependency graph is rendered in reference catalog § Linked References. Out-of-KG pointers to legacy code or external resources, when present, remain in this document under “Source-tree references” below.
3. Subscription API
3.1 gnomon.subscribe()
type SubscriptionHandle = {
pause(): void;
resume(): void;
close(): void;
readonly state: "open" | "paused" | "closed";
};
type SubscriptionCallback<T> = (event: SubscriptionEvent<T>) => void | Promise<void>;
function subscribe<T>(
query: GnomonQuery<T>,
callback: SubscriptionCallback<T>,
options?: SubscriptionOptions,
): SubscriptionHandle;
The GnomonQuery<T> type is the same query descriptor used by the RFC-0007 point-in-time read API. A subscription over query Q delivers events whenever the result set of Q changes.
3.2 Subscription Event Shape
type SubscriptionEvent<T> = {
eventId: string; // Globally unique event ID from the event log
entityId: string; // The entity that changed
entityKind: string; // The declared kind of the entity
timestamp: string; // ISO-8601 UTC timestamp of event commitment
changeType: "created" | "updated" | "deleted" | "transition";
previousValue: T | null; // null for "created"
currentValue: T | null; // null for "deleted"
sequenceNumber: number; // Monotonically increasing within this entity
};
eventId is the canonical identifier from the RFC-0010 event log. It is used for deduplication (§4.3).
3.3 Subscription Options
type SubscriptionOptions = {
maxPending?: number; // Max queued events before overflow (default: 1000)
overflowPolicy?: "drop-oldest" | "drop-newest" | "error"; // default: "drop-oldest"
startFrom?: "now" | "beginning" | { eventId: string }; // default: "now"
readScope?: ReadScopeDescriptor; // Explicit scope override; subject to authorization
};
startFrom: "beginning" causes the subscription to replay from the beginning of the event log for the matched entities before switching to live delivery. This is useful for hydrating read-model caches.
4. Delivery Semantics
4.1 At-Least-Once Delivery
The runtime MUST deliver every event that matches the subscription query and effective read scope at least once while the subscription is open and non-overflowed. An event is considered matched if it affects an entity whose current state would be returned by the subscription’s underlying query. Queue overflow is the explicit exception: a drop emits M20003, or closure emits M20004, before any event is lost (§6.1).
At-least-once means duplicate delivery is possible, most likely during reconnection after a transient failure. Consumers MUST be idempotent with respect to eventId.
4.2 Intra-Entity Ordering
Events for the same entityId MUST be delivered in sequenceNumber order. The runtime MUST NOT deliver event N+1 for an entity until event N has been acknowledged by the callback (i.e., its returned Promise has resolved, or the synchronous callback has returned without throwing).
Cross-entity ordering is not guaranteed. Two entities updated in the same kernel batch may deliver their events in any order across separate subscription callbacks.
4.3 Deduplication
The runtime tracks acknowledged eventId values per subscription handle using a sliding window of the last 10,000 event IDs. An ID enters this completed window only after callback acknowledgement. An event with an ID already in the completed window is silently discarded before delivery. The runtime tracks in-flight IDs separately; callback rejection removes the in-flight marker and re-queues the event, so its retry is not discarded as a duplicate. Consumers SHOULD still implement their own idempotency checks for events older than the deduplication window.
4.4 Acknowledgement Model
The callback’s return value serves as the acknowledgement:
- Synchronous callback: acknowledgement occurs when the function returns without throwing.
- Async callback: acknowledgement occurs when the returned
Promiseresolves. If thePromiserejects, the event is re-queued at the front of the pending queue and redelivered after a brief backoff. - A callback that throws synchronously is treated the same as a rejected
Promise.
5. Subscription Lifecycle
A subscription is always in one of three states:
open()
┌─────────────────────────────┐
│ ▼
[closed] [open] ──── pause() ──► [paused]
▲ │ │
└──── close() ─────────┘ resume() ───────┘
- open: Events are delivered to the callback as they arrive. Created by
gnomon.subscribe(). - paused: Events are buffered in the pending queue (up to
maxPending). No callback invocations occur. Useful during UI background transitions. - closed: The subscription is terminated. No further events are delivered. The pending queue is discarded.
close()is idempotent.
Transitions:
pause()on an open subscription: transitions to paused.resume()on a paused subscription: transitions to open and drains the pending queue.close()on any state: transitions to closed.pause()on a closed subscription: no-op, emits runtime warning.resume()on an open subscription: no-op.
6. Back-Pressure
6.1 Max Pending Queue
When the runtime has events ready to deliver but the subscription is paused or the callback has not yet acknowledged the previous event, events accumulate in the pending queue.
If the pending queue reaches maxPending:
drop-oldest(default): Emit M20003, then drop the oldest undelivered event. The dropped event is not recoverable via this subscription; consumers that need full history should replay from the event log.drop-newest: Emit M20003, then discard the incoming event.error: Emit M20004, then transition the subscription to closed.
6.2 Recommended Defaults
For UI subscribers (short-lived, tolerant of minor gaps): drop-oldest, maxPending: 100.
For audit or data-pipeline subscribers (must not miss events): error, maxPending: 10000, with reconnect-and-replay logic.
7. Integration with the RFC-0010 Event Log
Subscriptions are implemented as a streaming tail of the RFC-0010 event log. The subscription runtime:
- Registers a query filter against the kernel’s event emission stream.
- Each committed event batch is evaluated against all active subscription filters.
- Matching events are enqueued into the subscription’s pending queue.
- The delivery loop dequeues and invokes callbacks in intra-entity order.
The startFrom option controls the log position at which the subscription begins consuming:
"now": Subscribe from the next committed event after subscription open. No historical replay."beginning": Replay from log position 0. All historical events are delivered before live events.{ eventId: "<id>" }: Resume from the event immediately after the specified event ID. Used for crash recovery.
8. Security: Read-Scope Enforcement
At subscription open time, the runtime MUST verify that the subscribing principal has read access to the entity kinds included in the subscription query. This check uses the same readScope mechanism as RFC-0008 §8.10.
Enforcement rules:
- If the subscription query references entity kinds not covered by the principal’s read scope, the subscription MUST be rejected at open time with M20005.
- At delivery time, each event is checked against the intersection of the subscription’s scope fixed at open and the principal’s current grants. An event outside this effective scope is silently filtered and not delivered. A revocation narrows delivery immediately; a later grant cannot widen an existing subscription, so the principal must open a new one to use it.
- An agent subscription MUST NOT widen its read scope beyond what is declared in the agent’s
permissionsblock. Attempts to pass areadScopeoverride that exceeds declared permissions produce M20006.
9. Failure Modes
| Scenario | Outcome | Diagnostic |
|---|---|---|
| Subscription query references entity kind outside read scope | Subscription rejected at open | M20005 |
Agent subscription exceeds declared permissions read scope | Subscription rejected at open | M20006 |
Pending queue reaches maxPending, policy drop-oldest | Oldest event dropped, subscription continues | M20003 (warning) |
Pending queue reaches maxPending, policy error | Subscription closed | M20004 (error) |
| Callback throws synchronously | Event re-queued, backoff, redelivered | — |
Callback Promise rejects | Event re-queued, backoff, redelivered | — |
| Event log cursor lost (e.g., runtime restart) | Runtime SHOULD resume from last acknowledged eventId if startFrom is provided; otherwise reopen from “now” | M20007 (warning) |
pause() called on a closed subscription | No-op | M20008 (warning) |
startFrom: { eventId } references an event not in the log | Subscription rejected at open | M20009 (error) |
10. Invariants
- At-least-once delivery. Every event committed to the RFC-0010 event log that matches the subscription query and effective read scope MUST be delivered at least once to an open, non-overflowed subscription. Queue overflow is subject to the diagnostic rule in §6.1.
- Intra-entity ordering. Events for a single entity are always delivered in
sequenceNumberorder. The runtime MUST enforce this regardless of event log batch ordering. - Deduplication window. The runtime tracks the last 10,000 acknowledged
eventIdvalues per subscription. Events within the completed window are not redelivered; rejected callback attempts remain eligible for retry. - Read-scope immutability. Effective delivery scope is the intersection of the scope fixed at open and the principal’s current grants. Revocations narrow it immediately; later grants do not widen the opened subscription, so a new subscription must be opened.
- No silent overflow. An event lost due to queue overflow MUST produce a diagnostic. Silent event loss is not permitted.
- Closed is terminal. Once a subscription is closed, it cannot be reopened. A new subscription must be created.
- Query immutability. The subscription query cannot be changed after the subscription is opened. Changing the query requires closing and reopening the subscription.
11. Diagnostic Codes (M20xxx)
All diagnostics introduced by this RFC occupy the M20000–M20999 range.
| Code | Severity | Summary |
|---|---|---|
| M20000 | — | Reserved. |
| M20001 | — | Reserved. |
| M20002 | — | Reserved. |
| M20003 | Warning | Event dropped due to pending queue overflow (drop-oldest or drop-newest policy). |
| M20004 | Error | Subscription closed due to pending queue overflow (error policy). |
| M20005 | Error | Subscription query references entity kind outside principal’s read scope; subscription rejected. |
| M20006 | Error | Agent subscription read scope exceeds declared permissions; subscription rejected. |
| M20007 | Warning | Event log cursor was lost; subscription resumed from approximate position. |
| M20008 | Warning | pause() called on a closed subscription; ignored. |
| M20009 | Error | startFrom event ID not found in event log; subscription cannot resume from that position. |