kris/krisbuild
Watch
1
0
Fork
You've already forked krisbuild
 0
Code Issues Pull requests 2 Projects Releases Packages Wiki Activity Actions

tests: fix pr_events race between the eval result and the closed webhook #19

Closed
kris wants to merge 1 commit from fix-flaky-pr-closed into main
pull from: fix-flaky-pr-closed
merge into: kris:main
Conversation 4 Commits 1 Files changed 2 +18
kris commented 2026-09-27 19:51:36 +02:00
Owner
Copy link

pr_events::closed_cancels_the_graph_but_lets_an_effectful_task_finish failed in 3 of 8 local runs once all the control-plane suites share one test process (#18). It passed 10/10 when run alone.

The extra Cancel it saw was for the eval instance, not the effectful deploy. The eval's Succeeded event travels over the agent WebSocket, while the closed webhook arrives over HTTP and can overtake it. The control plane then correctly cancels an eval it still sees as running. So this is a race in the test, not a control-plane bug.

The fix adds common::wait_task_state and waits for the eval to be terminal before posting closed.

🤖 Generated with Claude Code

`pr_events::closed_cancels_the_graph_but_lets_an_effectful_task_finish` failed in 3 of 8 local runs once all the control-plane suites share one test process (#18). It passed 10/10 when run alone. The extra Cancel it saw was for the **eval** instance, not the effectful `deploy`. The eval's `Succeeded` event travels over the agent WebSocket, while the `closed` webhook arrives over HTTP and can overtake it. The control plane then correctly cancels an eval it still sees as running. So this is a race in the test, not a control-plane bug. The fix adds `common::wait_task_state` and waits for the eval to be terminal before posting `closed`. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
tests: wait for the eval before closing the PR in pr_events
Some checks failed
krisbuild/kris/krisbuild/nix/workspace-deps cached
Details
krisbuild/kris/krisbuild/nix/hello cached
Details
krisbuild/kris/krisbuild/nix/world cached
Details
krisbuild/kris/krisbuild/nix/build cached
Details
krisbuild/kris/krisbuild/nix/clippy cached
Details
krisbuild/kris/krisbuild/nix/kb-check cached
Details
krisbuild/kris/krisbuild/nix/test failed on nuxbox (timeout)
Details
krisbuild/kris/krisbuild krisbuild kris/krisbuild: one or more tasks failed
Details
krisbuild/queue the pull request was closed
70c9a8c3b3 
The agent's eval `Succeeded` travels over the WebSocket while the
`closed` webhook arrives over HTTP; under load the webhook overtakes it,
the control plane correctly cancels the still-running eval, and the
"never yanked" assertion sees that extra Cancel. Failed 3 of 8 runs
with all control-plane suites in one process; wait for the eval to be
terminal first.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
kris commented 2026-09-27 19:52:05 +02:00
Author
Owner
Copy link

@krisbuild r+

@krisbuild r+
krisbuild commented 2026-09-27 20:20:27 +02:00
Collaborator
Copy link

The pull request head's own build failed (graph 600).

  • nix/test: exit-code on nuxbox, exit 1 — log

Push a fix, or comment @krisbuild retry to rerun the failed tasks and requeue.

The pull request head's own build failed ([graph 600](http://192.168.0.2:1337/ui/graphs/600)). - `nix/test`: exit-code on nuxbox, exit 1 — [log](http://192.168.0.2:1337/ui/instances/17689) Push a fix, or comment `@krisbuild retry` to rerun the failed tasks and requeue.
kris commented 2026-09-27 22:00:13 +02:00
Author
Owner
Copy link

Superseded by #20, which landed the same pr_events eval wait (plus common::wait_task_state). Closing.

Superseded by #20, which landed the same `pr_events` eval wait (plus `common::wait_task_state`). Closing.
kris closed this pull request 2026-09-27 22:00:13 +02:00
krisbuild commented 2026-09-27 22:00:15 +02:00
Collaborator
Copy link

Removed from the merge queue: the pull request was closed.

Removed from the merge queue: the pull request was closed.
Some checks failed
krisbuild/kris/krisbuild/nix/workspace-deps cached
Details
krisbuild/kris/krisbuild/nix/hello cached
Details
krisbuild/kris/krisbuild/nix/world cached
Details
krisbuild/kris/krisbuild/nix/build cached
Details
krisbuild/kris/krisbuild/nix/clippy cached
Details
krisbuild/kris/krisbuild/nix/kb-check cached
Details
krisbuild/kris/krisbuild/nix/test failed on nuxbox (timeout)
Details
krisbuild/kris/krisbuild krisbuild kris/krisbuild: one or more tasks failed
Details
krisbuild/queue the pull request was closed

Pull request closed

Please reopen this pull request to perform a merge.
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
kris/krisbuild!19
No description provided.