Each connection runs on its own OS thread with its own V8 isolates, so
connection count is what drives the server's thread footprint.
MAX_PENDING_WS_HANDOFFS bounds only the handoff queue: once a stream is handed
off, nothing limited how many could be live at once, and a client could grow
the server without limit.
Reserve a slot in the accept loop and release it from a guard that runs when
the connection thread unwinds, on every path including panics and the early
returns before the runtime is built -- a plain decrement at the end of the
closure would leak slots until the cap wedged the server shut. Connections
past the limit get an explicit 503 with X-Obscura-Reason: max-connections
rather than a bare reset, so a client can tell a refusal from a crash. The
refusal happens in the WS dispatch loop, so the HTTP control plane (#62) stays
reachable at the cap.
Default 128, matching the handoff bound and well above any real client fan-out
(Playwright/Puppeteer use one connection per browser). At the cap, with arenas
capped by the previous commit: 146 threads, 33.2 GiB of reserved address space
and 51 MiB resident -- nearly all of that reservation being V8's process-wide
sandbox, which is present at zero connections.
Threads are the honest cost here, not memory: at 100 connections each running
JS this server reserves 83.5 GiB against `main`'s 83.6 GiB, because `main`
pays the same V8 sandbox as soon as it runs any JS. What it does not pay is
118 threads against 17.
The test covers all three properties, and specifically that a closed
connection frees its slot: a leak there would only surface as a server that
silently stops accepting after N lifetime connections.
Refs #430.
Picks up @Timtech4u's PR #33, which has been waiting since April. The
design (CLI flag + threading through BrowserContext + http client) is
theirs and is correct; this is a clean rewrite on top of current main
because repeated rebase rounds had been reverting unrelated work that
landed in the meantime.
Issue #4 blocked all loopback / RFC1918 / link-local fetches to prevent
SSRF. The only escape was OBSCURA_ALLOW_PRIVATE_NETWORK=1, which is
process-wide and clunky in pipelines. Multiple users (@Timtech4u in
the original PR, then @duartemvix and @necrogami in the comment
thread) reported they cannot use obscura for development against
http://localhost:N without it.
Adds --allow-private-network as a global CLI flag, works on fetch,
serve, and scrape. Threads through BrowserContext::with_storage_and_network
and the http client's validate_url. CdpContext gets a new_full
builder and obscura_cdp::start_with_full_serve_options entry point so
'obscura serve --allow-private-network' lets Puppeteer/Playwright
navigate to localhost too. Default remains blocked.
Backward compat: the env var still works (OR'd with the flag inside
validate_url). All existing constructors and entry points unchanged.
Verified end-to-end against a local http.server on 127.0.0.1: fetch
and CDP Page.navigate both blocked by default, both succeed with the
flag. Public URLs unaffected. Integration suite 28/28, full workspace
cargo test all green.
* feat(cdp): implement Network.setCookie + deleteCookies, full field fidelity
Closes#168.
Adds the missing Network.setCookie (singular) and Network.deleteCookies
methods and upgrades the existing Network.setCookies/getCookies handlers
so sameSite, expires, httpOnly, secure, and the url-fallback parameter
all round-trip correctly.
Network and Storage cookie handlers share a single parse_cdp_cookie /
parse_delete_cookies_params helper to avoid drift.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
* fix(cdp): fall back to default cookie jar, add Network.getAllCookies
Addresses @SGavrl review on PR #170:
- Network.{set,get,delete}Cookie(s)/clearBrowserCookies now resolve the
cookie jar via a shared helper that prefers the session page's jar and
falls back to ctx.default_context.cookie_jar when no target has been
attached. Mirrors how Storage.setCookies already routes when there is
no session, so Puppeteer/Playwright flows that issue cookie methods
before Target.attachToTarget no longer hit "No page".
- Add Network.getAllCookies (separate CDP method per spec); aliases the
existing getCookies handler so both arms return the full jar.
Tests: 6 new #[tokio::test] cases in domains::network::tests cover the
no-session fallback for set/get/setMany/delete/clear and the new
getAllCookies arm. RED on prod-reverted file (6/6 fail with "No page" /
"Unknown Network method: getAllCookies"), GREEN on patched
(22 -> 30 obscura-cdp lib tests, obscura-net 22/22 baseline preserved).
GHSA-q55h-vfv9-qcr5 patched Page.navigate to refuse file:// URLs
unless --allow-file-access was set, but Target.createTarget accepted
the same scheme and called page.navigate(url) directly, bypassing
the policy. A CDP client could call Target.createTarget {url: file://...}
then Runtime.evaluate to read document.body.textContent and recover
arbitrary local files.
Apply the same gate at the Target.createTarget entrypoint. Pull
the file-scheme detector into a shared util module so both domains
stay in lock-step instead of one drifting from the other.
Verified blocked vs the unpatched build (pr69):
- Patched: Target.createTarget {url: file:///tmp/secret} errors
"Target.createTarget to file:// is disabled. ..." before any
navigation runs.
- Unpatched: same call returns the file contents via
Runtime.evaluate {expression: "document.body.textContent"}.
GHSA-q55h-vfv9-qcr5. Any process able to reach the CDP port could
send Page.navigate {"url":"file:///etc/shadow"} and read arbitrary
local files (the process's UID's worth of read access). When obscura
runs as root inside Docker or as a systemd service that translates
straight into /etc/shadow, /root/.ssh/id_rsa, /proc/1/environ, etc.
Validate at the CDP boundary, not in obscura-net, so direct CLI
callers ("obscura fetch file://...") keep working. A new
BrowserContext.allow_file_access flag defaults to false; Page.navigate
returns an error when the requested URL is file:// and the flag is
off. CdpContext::new_with_security threads the flag from the server
entry point, and "obscura serve --allow-file-access" opts in for
local-HTML testing workflows.
Tested:
- Default obscura serve: CDP Page.navigate {file:///etc/passwd} ->
refused with descriptive error.
- obscura serve --allow-file-access: same call succeeds.
- obscura fetch file://... CLI path: unchanged, still reads the file.
- All earlier PoCs (GHSA-6pj9 JS->file SOP, GHSA-4789 sub-resource
SSRF, GHSA-4789 CSS template injection) still pass.
- cargo test --workspace clean.
Adds a multi-stage Dockerfile (rust:1-slim-bookworm builder → distroless/cc
runtime) plus a --host flag for obscura serve that defaults to 127.0.0.1
for safety but can be set to 0.0.0.0 so the container's port is reachable
via docker run -p mapping.
Dockerfile cherry-picked from #84. The SPA chunk loading commit from that
PR was left out — verified it didn't actually fix the Next.js webpack
errors on kiwi.com (#86).
obscura serve --user-agent <ua> only logged the value via tracing::info!
and never passed it to the CDP server, so every navigation kept using
the default Chrome UA string and httpbin.org/user-agent reported the
default rather than the override.
Plumb the option end-to-end:
- BrowserContext::with_full_options(id, proxy, stealth, user_agent)
initializes both BrowserContext::user_agent and the underlying
ObscuraHttpClient::user_agent RwLock at construction time so the
first navigation already sees the override.
- CdpContext::new_with_full_options forwards the UA to the default
BrowserContext.
- obscura_cdp::start_with_full_options accepts the UA and passes it
through cdp_processor.
- main.rs Serve arm calls start_with_full_options with the parsed UA.
- run_multi_worker_serve forwards --user-agent <ua> when spawning
worker subprocesses so multi-worker mode honors the flag too.
Existing two-arg start_with_options and three-arg with_options stay
as thin shims (call the full variant with None) so older callers and
the public crate API remain source-compatible.
Three new unit tests in obscura-browser cover the propagation, the
Chrome default fallback, and the legacy with_options path.
Generated by Ora Studio
Vibe coded by ousamabenyounes
Co-authored-by: Ora Studio <[email protected]>
sycatle