bug(shutdown): 'Stop server' button no-ops on ctl.sh-managed daemons (SIGINT inherited SIG_IGN from non-interactive bash '&')
#7,078 opened on Aug 16, 2026
Repository metrics
- Stars
- (17,426 stars)
- PR merge metrics
- (Avg merge 14h 31m) (314 merged PRs in 30d)
Description
Symptom
The Stop server button in Settings (POST /api/shutdown) returns 200 {"status": "shutting_down"} but the WebUI process keeps running — for daemons launched via ./ctl.sh start (including the launchd-managed setup where the plist calls ctl.sh start).
Downstream consequences for the user:
- Every subsequent
./ctl.sh startspawns a new instance that dies with[!!] FATAL: Another server is already responding on 127.0.0.1:8787. Stop the existing instance first.(_abort_if_already_serving, server.py:529) — the old process still owns the port. - The pid/state files (
~/.hermes/webui.pid,~/.hermes/webui.ctl.env) keep pointing at the just-died new PIDs, soctl.sh stopalso cannot reach the real process. - Result: the only way to actually stop the daemon is
kill <pid>from a terminal.
Root cause
Three facts combine:
-
ctl.sh startspawns the server from a non-interactive bash background job (( cd ...; exec nohup python bootstrap.py ... ) >> log 2>&1 &). POSIX/bash rule: "If job control is not in effect, asynchronous commands ignore SIGINT and SIGQUIT in addition to SIGHUP" — the child inherits SIGINT = SIG_IGN.Minimal proof:
$ bash -c 'python3 -c "import signal; print(repr(signal.getsignal(signal.SIGINT)))" & wait' <Handlers.SIG_IGN: 1> -
CPython preserves an inherited SIG_IGN for SIGINT — it does not install its KeyboardInterrupt handler when SIGINT was ignored at interpreter startup.
-
_handle_shutdown(api/routes.py) commits suicide with SIGINT:def _do_shutdown(): import time time.sleep(0.3) os.kill(os.getpid(), signal.SIGINT)For a ctl.sh-managed daemon this signal is a silent no-op at the OS level.
Meanwhile server.py only registers a graceful handler for SIGTERM (server.py:717
signal.signal(signal.SIGTERM, _request_shutdown)) — which is exactly whykill -TERMstops the daemon cleanly while the button's SIGINT does nothing.
Reproduction
./ctl.sh start(any non-interactive shell).curl -X POST http://127.0.0.1:8787/api/shutdown(authenticated session) →200 {"status":"shutting_down"}.- Process still alive:
$ kill -0 <pid> && echo alive alive - Signal-level equivalent:
Note:$ kill -INT <pid> # no effect, process stays alive $ kill -TERM <pid> # graceful shutdown (shutdown-audit + drain in the serve_forever() finally block)_log_shutdown_audit()(server.py:492) has never fired in my environment's webui.log despite many stop attempts — consistent with no shutdown signal ever being processed.
Suggested fix
In server.py main(), register the existing _request_shutdown handler for SIGINT as well as SIGTERM:
try:
signal.signal(signal.SIGTERM, _request_shutdown)
signal.signal(signal.SIGINT, _request_shutdown)
except (ValueError, OSError):
...
This matches the handler's own docstring intent ("The handler is idempotent and guards against double-shutdown (e.g. repeated SIGTERM/SIGINT)") and makes the Settings stop button, external kill -INT, and foreground Ctrl-C all take the same graceful path (the finally block around serve_forever() already drains sessions either way).
Alternative (narrower): have _do_shutdown send SIGTERM instead of SIGINT. This fixes the button but leaves process-level SIGINT ignored for ctl.sh daemons.
Verification performed
signal.getsignal(SIGINT)in a bash-&-spawned child →SIG_IGN(mechanism).- Isolated instance (separate
HERMES_HOME/port 8789, launched viactl.sh):kill -INT→ alive;kill -TERM→ clean exit with[crash-visibility] process exitlogged. - Production daemon (PID up 19 days, launched via launchd→ctl.sh):
/api/shutdownreturned 200 at 20:39:51, process still alive 12+ minutes later; allctl.sh startattempts in between died with the port-in-use FATAL. - Confirmed against current upstream master (2026-08-15): server.py still registers only SIGTERM; routes.py
_do_shutdownstill uses SIGINT.
Related: #6396 assumes the Control Center stop works for ctl.sh-managed daemons — this bug is why the restart half of that story matters.