Commit Graph
56 Commits
Author SHA1 Message Date
F0Rextasy 6f150065bc Reject scrape with empty URL list (#93) 2026-05-03 16:46:13 +02:00
F0Rextasy 986bcf5b8e Reject zero scrape concurrency (#96) 2026-05-03 16:46:01 +02:00
F0Rextasy 0e84f15711 fix(cli): remove unused AsyncReadExt import (#98) 2026-05-03 16:45:36 +02:00
h4ckf0r0day 99e75f1e62 fix: load balancer panics, stealth flag, shadow DOM polyfill
Fixes #1: replace unwrap() panics in run_multi_worker_serve with match on the Result.
Fixes #2: route /json round-robin via worker_port instead of hardcoded port+1.
Fixes #3: return 502 Bad Gateway when a worker is unreachable instead of silently dropping the connection.
Fixes #5: thread --stealth through start_with_options -> CdpContext -> BrowserContext::with_options so the flag actually has an effect in serve mode (was let _ before). Log message now reflects the actual capability based on the stealth cargo feature being compiled in.
Fixes #15: try/catch the parentNode assignment in the attachShadow polyfill so it doesn't throw a TypeError against the read-only Node.parentNode getter (Cloudflare Turnstile triggers this path).
2026-04-25 14:27:52 +02:00
Matt Van Horn 065b1b1a1f fix(scrape): add per-worker timeout to prevent indefinite hangs
Closes #7.

A stuck worker subprocess in `obscura scrape` could hold its semaphore
permit forever and hang the entire job. The two `BufReader::read_line`
calls and the final `child.wait()` were unbounded; a worker that never
wrote a response line, or a `--eval` expression that never terminated,
would block indefinitely.

This wraps the per-worker spawn->navigate->evaluate->shutdown sequence
in `tokio::time::timeout(worker_timeout, ...)` and adds tighter
`tokio::time::timeout(read_timeout, ...)` wrappers around each
`read_line` call. Shutdown waits and the kill-on-timeout path are
bounded with a 5s `shutdown_timeout`, so a misbehaving worker can't
block cleanup either.

Adds a `--timeout <SECS>` flag on the `scrape` subcommand (default 60,
clap-validated `>= 1`). On timeout, the child is killed and the result
JSON entry becomes `{"url": ..., "error": "timeout", "time_ms": ...}`,
keeping the output stream well-formed.

Per-worker max time is now bounded by `worker_timeout` (outer) plus the
shutdown_timeout cleanup window, regardless of worker behavior.
2026-04-24 16:00:48 -07:00
h4ckf0r0day 8749725b45 Initial Release 2026-04-13 12:30:28 +02:00