wazionapps/nexo

NEXO 7.11.2 — Watchdog reaper + Enforcer respects restart_required

Two reliability fixes in the same family — components that were ignoring signals already on disk and burning cycles or holding slots open as a result. v7.11.0 added the runtime fingerprint precisely to avoid forcing restarts when not needed; v7.11.2 makes the rest of NEXO actually respect the restart marker when one exists.

Fix 1 — Watchdog stuck-cron reaper

The v5.8.1 in-flight detection stopped the watchdog from killing running jobs (deep-sleep loop incident 2026-04-14..17). The same restraint silently let truly hung wrappers — e.g. headless claude --bare blocked on an MCP that flagged mcp_restart_required — block their own next tick for days. morning-agent, followup-runner and orchestrator-v2 went silent 2026-04-24..27.

New run_stuck_reaper() sweep at the top of every watchdog tick:

• Reads every cron_runs row with ended_at IS NULL and reaps anything past per-cron stuck_after_seconds (per-cron from manifest.json, fallback STUCK_DEFAULT_SECONDS=43200 = 12h global).
• Live wrapper → SIGTERM to wrapper PID via pgrep -f "nexo-cron-wrapper.sh CRON_ID ". The wrapper's existing trap forwards to the child and runs finalize_row, closing the row at exit_code=143. After 10s grace, any survivor (wrapper + descendants) gets SIGKILL via pkill -KILL -P.
• Orphan zombi row (wrapper PID gone, row still NULL) → cleaned in-band: ended_at=now, exit_code=137, summary='stuck row reaped by watchdog: wrapper PID gone'.
• cron_id='watchdog' is hard-coded into STUCK_REAPER_SKIP so the watchdog never reaps itself mid-tick.

Per-cron overrides protect the v5.8.1 case: deep-sleep: 28800 (8h), sleep: 14400 (4h), evolution: 14400 (4h, weekly heavy run). Short-runners get tighter bounds: morning-agent: 1800, followup-runner: 1800, email-monitor: 600. New observability counter TOTAL_REAPED in watchdog-status.json (summary.reaped), the human report header (REAPED:), and the final log line (REAPED=N).

Fix 2 — Enforcer respects mcp-restart-required marker

The Guardian/Enforcer (HeadlessEnforcer in src/enforcement_engine.py) periodically injects <system-reminder> blocks asking the agent to call nexo_* tools (heartbeat, session_diary_write, smart_startup, guard_check, etc). When the MCP server has the ~/.nexo/runtime/operations/mcp-restart-required.json marker on disk (written by plugins/update.py after a nexo update that actually changes runtime .py bytes — see v7.11.0 fingerprint gating), every one of those reminders triggered a tool call that immediately failed with mcp_restart_required. The agent burned cycles on guaranteed no-ops until the operator restarted the client.

New gate at the top of HeadlessEnforcer._enqueue(): if the prompt mentions nexo_ and the marker file exists, skip + log SKIP: ... mcp_restart_required marker present. Reminders that don't reference nexo_* (R23 deploy guards, R25 nora/maria read-only, etc.) still fire — they don't depend on the MCP being live. New helpers _mcp_restart_marker_path() and _mcp_restart_pending() (cached per-instance with 30s TTL). Conservative: any path/IO error returns False so the gate never blocks legitimate enforcement.

Tests

• tests/test_watchdog_stuck_reaper.py — 6 new tests: fresh in-flight row left alone (v5.8.1 regression guard), per-cron threshold respected, orphan zombi cleanup, real wrapper killed via SIGTERM, self-immolation guard, default fallback.
• tests/test_enforcer_restart_required_gate.py — 6 new tests: nexo_* prompt enqueues normally with no marker, nexo_* prompt skipped with marker present, non-nexo_* prompt still enqueues with marker, 30s TTL cache behavior, pre-F0.6 legacy operations/ path detected, missing NEXO_HOME returns False instead of raising.
• 3 existing watchdog tests + 52 existing enforcer tests stay green.

Live triggering

3 zombi wrappers (morning-agent 5h29m, followup-runner 21h, orchestrator-v2 ~20h) reaped manually with the same kill -TERM path the reaper now automates; cron_runs rows closed with exit_code=143 as expected. Enforcer gate validated by inspecting the same session that built this release: it received ~6 mcp_restart_required no-op pings before this release would have suppressed all of them.

What's Changed

• fix(tests): repair two pre-existing failures on main by @wazionapps in https://github.com/wazionapps/nexo/pull/309
• Release 7.11.2: watchdog reaper + enforcer respects restart_required by @wazionapps in https://github.com/wazionapps/nexo/pull/310

Full Changelog: https://github.com/wazionapps/nexo/compare/v7.11.1...v7.11.2