Commit Graph
1 Commits
Author SHA1 Message Date
Matt Van Horn 065b1b1a1f fix(scrape): add per-worker timeout to prevent indefinite hangs
Closes #7.

A stuck worker subprocess in `obscura scrape` could hold its semaphore
permit forever and hang the entire job. The two `BufReader::read_line`
calls and the final `child.wait()` were unbounded; a worker that never
wrote a response line, or a `--eval` expression that never terminated,
would block indefinitely.

This wraps the per-worker spawn->navigate->evaluate->shutdown sequence
in `tokio::time::timeout(worker_timeout, ...)` and adds tighter
`tokio::time::timeout(read_timeout, ...)` wrappers around each
`read_line` call. Shutdown waits and the kill-on-timeout path are
bounded with a 5s `shutdown_timeout`, so a misbehaving worker can't
block cleanup either.

Adds a `--timeout <SECS>` flag on the `scrape` subcommand (default 60,
clap-validated `>= 1`). On timeout, the child is killed and the result
JSON entry becomes `{"url": ..., "error": "timeout", "time_ms": ...}`,
keeping the output stream well-formed.

Per-worker max time is now bounded by `worker_timeout` (outer) plus the
shutdown_timeout cleanup window, regardless of worker behavior.
2026-04-24 16:00:48 -07:00