Paperclip
Paperclip exposes queue starvation and runtime recovery controls
A Paperclip report says an in-review wake waited 12 hours 48 minutes behind newer in-progress runs; related work adds a watchdog to the sandbox callback bridge and surfaces runtime state and recovery controls. The cluster turns hidden delay into an operator-visible scheduling fault.
paperclipai/paperclip issue #11348 is the inspected primary source: “No aging in heartbeat queue delayed an in_review comment wake 12h48m behind newer in_progress runs.” The related records below were inspected as supporting context rather than independent confirmation.
The facts
- paperclipai/paperclip issue #11348 was created 2026-08-13T23:49:32Z and was open at inspection; its title is “No aging in heartbeat queue delayed an in_review comment wake 12h48m behind newer in_progress runs.” - paperclipai/paperclip PR #11341 was created 2026-08-13T20:25:53Z and was open at inspection; its title is “fix(adapter-utils): add per-iteration timeout and watchdog to the sandbox callback bridge poll loop.” - paperclipai/paperclip PR #11337 was created 2026-08-13T19:46:07Z and was open at inspection; its title is “Surface runtime state and recovery controls.” - The repository reported 78046 stars, 5047 open issues, default branch master, and last push 2026-08-14T01:24:16Z. - The latest tagged-release baseline checked was v2026.722.0 published 2026-07-22T23:05:41Z.
What changed
paperclipai/paperclip issue #11348 reports: Pre-submission checklist [x] Searched open and closed paperclipai/paperclip issues; no duplicate found. [x] Reproduced on deployed Paperclip build v2026.722.0-214-gaf6b32d82 (commit af6b32d82a30dcdf001dd96c406d3981382d5d89). [x] Confirmed the same queue ordering on upstream master a8d118a779ac1befa33660b25ef14ef0821a9d6c. [x] Core server bug; not adapter-specific. What happened? With maxConcurrentRuns=1, an older queued wake attached to an in_review or blocked issue is assigned a lower dispatch rank than every dependency-ready wake for an in_progress issue assigned to the same agent. With no aging or fairness term, the parked-issue wake is passed over for as long as the agent has newer in_progress work. Its delay is bounded only by when the agent next has no rank-0 work; under sustained active work, that delay is unbounded in principle. The wake is accepted normally: enqueueWakeup() inserts agent_wakeup_requests(status="queued") and heartbeat_runs(status="queued"), and calls startNextQueuedRunForAgent(). The claim-time staleness gate permits comment wakes on in_review, and blocked issues can be auto-checked out. However, startNextQueuedRunForAgent() sorts every dependency-ready in_progress issue run at rank 0 and every other ready issue status at rank 1: Only availableSlots are then claimed. Because the queue is re-sorted on every dispatch and there is no aging/fairness bound, newer in_progress work repeatedly passes
paperclipai/paperclip PR #11341 reports: Thinking Path - Paperclip runs AI agents through adapters and sandboxed execution paths - The sandbox callback bridge carries file requests between the host and a sandbox - The poll loop waited forever when a sandbox call stopped responding - A permanent wait stranded queued requests and hid the run failure - This pull request adds per-call timeouts and an independent progress watchdog - The benefit is prompt request failure, run-level error reporting, and trace visibility Linked Issues or Issue Description What happened?** The sandbox callback bridge could wait forever when a client call stopped responding without a rejection. Expected behavior** The bridge should fail queued requests and report a run-level error when the sandbox channel stops responding. Steps to reproduce** 1. Start a sandbox callback bridge. 2. Queue a request. 3. Make the sandbox call stop responding. 4. Observe that the request does not receive a failure response. Paperclip version or commit** Commit 122669084a123add66980dfc67d16bfb798e67ad. Deployment mode** Built from source. Installation method** Built from source with pnpm. Agent adapter(s) involved** Custom or external sandbox callback bridge. Database mode** Not database-related. What Changed Add a per-iteration timeout for listJsonFiles and processRequestFile.
paperclipai/paperclip PR #11337 reports: Thinking Path - Paperclip is the open source app people use to manage AI agents for work. - Managed execution workspaces run development services for agents and operators. - Operators need one accurate view of each service state and its latest failure. - The prior read model could lose configured service identity, desired state, and recovery evidence. - Sibling workspaces could also select the same service port during concurrent allocation. - The user interface did not show all stopped or failed services or give clear recovery actions. - This pull request aligns runtime persistence, API projection, port allocation, and recovery controls. - The benefit is a stable runtime view with clear and safe service recovery. Linked Issues or Issue Description Refs #9807 Refs #9808 What existing behavior does this improve?** This change improves managed execution workspace runtime state and recovery. Subsystem affected** This change affects workspace runtime services, execution workspace APIs, shared runtime contracts, and workspace service controls. Current behavior** Runtime projections can omit configured service identity, desired state, and the latest failure evidence. Sibling workspaces can select the same service port. The user interface can hide stopped or failed
Why it matters
A queue can be technically alive while old work never runs. Without aging, watchdogs and visible recovery state, operators see silence rather than a scheduling decision they can inspect. The operator test is whether visible state, retained state and authority still describe the same event after retries, switching or restart.
Current
The primary record was open when captured on 2026-08-14. The tagged-release baseline was v2026.722.0 published 2026-07-22T23:05:41Z. Repository metadata, full source bodies, current pull requests and issues, releases, Google News, Hacker News, Lobsters, Metamesh and the rendered ClawCharts row were inspected. ClawCharts selected the subject; it did not prove the claim.
Evidence
The primary URL is https://github.com/paperclipai/paperclip/issues/11348. Supporting records are https://github.com/paperclipai/paperclip/pull/11341 and https://github.com/paperclipai/paperclip/pull/11337. Source bodies, timestamps and states are preserved in the daily evidence bundle. Test counts and reproductions remain attributed to their authors unless identified as independently rerun.
Source boundary
Open work is described as open, closed work as closed, and operator reports as reports. A pull request is evidence of proposed or reviewed direction, not proof of a shipped release. Search residue, package mirrors and historically published source spines were excluded.
Operator take
Add bounded aging to priority decisions, report oldest-wait age by class and make recovery controls emit durable receipts instead of silently resetting state. Preserve a before-state receipt, make the smallest reversible change, and verify the original failure independently.
Caveat
Public project records are mutable. Status, scope and evidence can change after publication. This brief records the inspected state and does not authorize changes to a reader’s deployment.
Source inspected; source state, environment and release boundary remain explicit.