Acknowledged delivery: Finished, seq/acked_seq, cancel resend, DB backups (proto 2) #21
Loading…
Reference in a new issue
No description provided.
Delete branch "lu/acked-delivery"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
docs/live-upgrades.md §4.2 (durable, acknowledged event delivery) and §4.5 (schema and rollback). Builds on #15 (handshake) and #12 (CP restarts) — until those land, this diff includes them; review the commits unique to this branch.
What
InstanceEvent::Finished { state, exit, outputs, expansion, usage }, applied by the CP in one transaction (usage, outputs/expansion, state/exit, result-cache entry). A success whose declared outputs are missing, or whose expansion doesn't parse/validate/materialize, is recordedfailed(outputs-invalid/expansion-invalid), keeping usage, with the reason in the log. Closes the TODO "Outputs lost in transit → green but uncached".Hello.boot(random per agent process) +Event.seqfrom 1. The agent outbox keeps events after writing them and drops them only whenLeaseAck.acked_seqcovers them; unacked ones are re-sent in order on the next connection (the "never drop non-log" bound is kept). The CP'sDeliveredtracks the highest applied seq per (node, boot); each seq is claimed before applying, so re-sent events apply once (fixes duplicated log bytes after the write timeout). The mark survives CP restarts vianodes.agent_boot/nodes.applied_seq(persisted per heartbeat and at disconnect; a CP crash may repeat ≤ one heartbeat of log). An event whose apply hits a DB error is un-claimed and the connection closed, so the agent re-sends it — a verdict is never lost to a transient DB error.cancelledgetsCancelagain — covers cancels issued while the node was disconnected.metatable; on start, ifmeta.last_build≠kb_core::build(),VACUUM INTO <db dir>/backups/<old build|unknown>-<ts>.sqlitebefore touching the schema; newest 3 kept. A failed copy is logged as an error and startup continues.db.rs→db/{mod,backup}.rs.meta, newnodescolumns, the expand/contract migration rule, backups); forgejo-setup.md (backups, rollback/restore); TODO and live-upgrades rollout line.Compatibility (
PROTO= 2; CP admits 0, 1, 2)Finished, seq/acked_seq, dedupe; a Welcome(2) arriving after the 3 s settle switches to acked modeAdditive fields (
boot,seq,acked_seq) are skipped when zero and ignored by older peers. The agent sends nothing from its outbox until it knows what the CP speaks, and translates at send time, so an event queued under one CP and re-sent to another (rollback) is still correct. Schema changes are additive only.Tests
Finished.db::backup(copy on build change, none on same build/fresh DB,unknown, keep 3, failed copy doesn't stop startup);it/acked_delivery.rs: Finished atomic, rejection classes, re-sent seqs applied once within a connection / across reconnect / across a graceful CP restart, new boot resets, cancel re-sent on Hello, an event failing to apply is not acked and is re-sent.🤖 Generated with Claude Code
Version skew becomes explicit and safe for the mixed CP/agent versions a staggered nuxbox/ares upgrade produces (docs/live-upgrades.md §4.4, and the `Hello.leases` part of §4.1). - `Hello` carries `proto` (absent = 0, legacy), `build` (`kb_core::BUILD`: `<version>+<rev>` from the flake, else the crate version), `caps`, and `leases`, the instances the agent is running. All additive: today's control plane ignores them. - The control plane answers every admitted Hello with `Welcome { proto, build }` (not sent to legacy agents, which could only log it as undecodable) and an immediate `LeaseAck` that renews the claimed leases exactly as a heartbeat does. It admits proto cp-1..=cp plus legacy 0 and refuses others with a 1008 close naming what to upgrade; refusals are logged and shown on /ui/nodes, /api/nodes and `kb nodes`, which also gain the agent build, proto and caps. - `kb_core::caps::required_caps(def, sched)`: an exhaustive map from every payload/input/isolation/effect/resolved-ref kind and SchedMeta enum value to a cap, empty for everything at proto 1, plus `constraints.caps`. The matchmaker requires them (legacy agents have exactly the frozen legacy set) and checks resolved refs against the chosen node. The skew rule is written into kb-core's docs and SPEC. - The bootstrap eval script is `KB_EVAL_FRONTEND=1 exec kb-eval-nix ...`: the frontend generation is identity (every eval's def_hash changes once; no kbN- bump), the eval requires `eval-nix:1`, agents with the nix feature advertise it, and kb-eval-nix (and `kb` as kb-eval-nix) refuses another generation. An environment variable rather than a flag, because today's frontend ignores it where it would reject an unknown flag - so a legacy agent is safely treated as having `eval-nix:1`. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>@krisbuild r+
Merged as
8bf274cdc4.