Browse Source

G-036 round 2: wedge mechanism proven - tcludp global event handles closed mid-session

Instrumented traffic-enabled specimen: wedged workers blocked (not spinning)
in WaitForSingleObject(sockListLock, INFINITE) inside tcludp event-source
procs (disassembly maps both wait sites to the sockListLock static; the
SocketThread released properly and parked on waitForSock). Handle-table data
from the full dump proves both tcludp global event objects were CloseHandle`d
mid-session: the statics still hold handle values now recycled to an
IoCompletion object and a manual-reset event (tcludp only creates auto-reset
events). Waiters starve forever on unsignalable objects. Only ExitSockets
closes these handles; the sole udp-loaded thread that died mid-session (run-1
shellfilter-run worker, terminated in the run transition) is the prime
trigger suspect. Console dependence dissolved - the CONIN$ restart was simply
the only mid-session udp-thread-death scenario. Minimal-repro die-variants
(release / terminate-mimic / full punk env) do not yet reproduce; next
candidates recorded (resultvar-terminate, thread::cancel, live CloseHandle
breakpoint).

Assisted-by: harness=claude; primary-model=claude-fable-5; api-location=anthropic.com
master
Julian Noble 2 weeks ago
parent
commit
281357dd89
  1. 46
      goals/G-036-tcl9-udp-console-worker-wedge.md

46
goals/G-036-tcl9-udp-console-worker-wedge.md

@ -169,6 +169,52 @@ udp-specific by construction (matching "only udp-using workers wedge"):
Dump artifact: %TEMP%/punk_wedge.dmp (244MB, round-1 specimen) + %TEMP%/punk_wedge_stacks.txt.
## Findings round 2: object-level proof of the wedge mechanism (2026-07-08)
Instrumented traffic-enabled specimen (syslog force-toggle in shellfilter::run to make the
runtag log worker send datagrams again): run-2's three syslog workers completed init then
wedged in <1s, blocked not spinning (~30-47ms CPU, Wait state). Their dump stacks all show
**WaitForSingleObjectEx inside tcl9udp1012 called from the Tcl event loop** - the
`WaitForSingleObject(sockListLock, INFINITE)` in tcludp's per-thread event-source procs
(disassembly confirms both worker wait sites load the sockListLock static; run-1's two
surviving syslog workers block at the sibling call site). The SocketThread's park site loads
the waitForSock static immediately after a SetEvent(sockListLock) - it released properly.
**Handle-table proof**: the sockListLock static holds handle 0x38c which the dump's handle
data shows is now an **IoCompletion object**, and waitForSock (0x37c) shows as a
**manual-reset event, set** - tcludp only ever creates auto-reset events. Both tcludp global
event objects were **CloseHandle'd mid-session** (waiters keep an object alive but no handle
remains to ever signal it; the freed handle VALUES were recycled by unrelated code). Every
udp-loaded thread that touches the event-source procs after that blocks forever. 16-thread
census: no thread holds the lock doing work; the previously-terminated run-1 shellfilter-run
worker (the only udp thread that died mid-session, in the run-1->run-2 transition) is the
prime closer suspect - `ExitSockets` (the only code closing these handles, registered via
Tcl_CreateExitHandler in every Udp_Init) apparently runs during that thread's teardown.
Downstream picture is now complete: closed-lock wedge -> udp-loaded workers freeze ->
punkshell's old synchronous teardown send to a frozen worker -> the original exit/quit hang
(mitigated). "Console dependence" dissolved: the CONIN$ restart was simply the only scenario
where a udp-loaded worker died mid-session. Tcl 8.6 immunity presumably = its
finalize-at-thread-death path not running the process exit handler (to verify; also check
which tcludp version the 8.6 kit bundles).
Minimal-repro attempts that did NOT reproduce (probe 'die' variants, consoleless script
mode): a released udp-loaded worker; a terminate-mimic (open socket + datagrams + close +
self thread::release -wait, verified thread death); same with the full punk package env
(packagepreference + shellthread). So the ExitSockets trigger needs something more from the
live shell. Next candidates:
1. terminate via `thread::send -async <tid> <script> <resultvar>` (the real
shutdown_free_threads form - result delivery during thread death may take a different
teardown path)
2. `thread::cancel` (the codethread is cancelled in the same transition window)
3. Decisive regardless: live-attach WinDbgX during the run transition with a conditional
breakpoint on KERNELBASE!CloseHandle for the lock handle value (read the statics early via
the running process) - catches the closer with a full stack. Or a debug tcludp build with
logging in ExitSockets (needs a compiler).
Round-2 artifacts: %TEMP%/punk_wedge2.dmp + _stacks/_disasm/_h2 logs.
## Alternatives considered
- Mitigate-only (make the sync send robust, default syslog off) - rejected as the *whole*

Loading…
Cancel
Save