* fix(#19): serialize V8 work across pages with process-wide async lock Concurrent CDP clients (≥5) trip V8's `heap->isolate() == Isolate::TryGetCurrent()` invariant and abort the process via V8_Fatal — no Rust panic, just abort(3). Each Page owns its own JsRuntime/Isolate and they all run on a single OS thread via LocalSet+spawn_local; when two pages' V8-touching futures interleave across an .await, V8 trips the check. This adds a process-wide tokio::sync::Mutex (obscura_js::v8_lock) that is acquired: - At the top of obscura_cdp::dispatch::dispatch (covers every CDP handler that may call into a JsRuntime). - In server.rs's process_with_interception spawn_local that runs page.navigate_with_wait + execute_preload_script (this path bypasses dispatch). The guard keeps each V8-touching block contiguous on the thread: V8 fully exits one Isolate before another page is allowed in. Trade-off: non-V8 handlers (Browser.*, Storage.*, Emulation.*) also serialize, but those are cheap and the safety win dominates. The properly concurrent fix is to pin each JsRuntime to its own OS thread and message-pass; tracked as the larger #19 follow-up. Adds an opt-in smoke test (`--ignored`) that boots the CDP server and runs 5 parallel clients through createTarget+navigate. Note: the deterministic abort needs subresource-fetch yields (page.rs:285 join_all) which data: URLs skip — full repro requires JS-heavy real traffic (issue author used scrapegraphai.com). * fix(#19): suspend other Isolates and defer foreign CDP during interception nav Address maintainer feedback on PR #36 (comment 4328238087): the V8 fatal still reproduces with `Fetch.enable patterns:["*"]` + 5 concurrent `Page.navigate`s. The `obscura_js::v8_lock` mutex was insufficient because V8's actual invariant is "only one Isolate may be entered per OS thread", and the mutex can't enforce that across `.await` points inside V8 ops. Two distinct paths violated the invariant: 1. Foreign CDP messages dispatched while a nav task was running. `process_with_interception`'s `select!` routed every non-Fetch message through `process_cdp_message → dispatch → page::handle::*`. `get_session_page_mut` then `suspend_js`'es other pages (drops their `JsRuntime` → `Isolate::Exit`) while the spawned nav task is parked at `op_fetch_url`'s `resolve_rx.await` with its Isolate still entered. Drop trips `heap->isolate() == Isolate::TryGetCurrent()`. 2. Subsequent navs through `process_with_interception` did not suspend prior pages. After nav-1 completes (page-1's JsRuntime stays alive in `ctx.pages`), nav-2's `init_js` constructs Isolate-2 while Isolate-1 is still current. The next V8 scope unwind aborts via `Context::Exit`. Fix (3 changes in `crates/obscura-cdp/src/server.rs`): - `cdp_processor`: add a `VecDeque<ServerMessage> deferred` drained before each new `rx.recv()`, so deferred messages are processed with no nav task in flight. - `process_with_interception` `select!`: only handle `NewConnection` and `Fetch.{continue,fulfill,fail}Request` inline (safe — no V8 enter on this side); push everything else into `deferred`. - `process_with_interception` entry: explicitly `suspend_js()` every other page in `ctx.pages` before spawning the nav task, mirroring the invariant the regular dispatch path already enforces via `get_session_page_mut`. Verification: - New test `crates/obscura-cdp/tests/concurrent_navigations_with_fetch.rs` is the maintainer's exact repro (5 clients, `Fetch.enable patterns:["*"]`, concurrent `Page.navigate` to https://example.com, auto-reply `Fetch.continueRequest`). Aborts on PR #36 head, passes after fix (5/5 stable runs, ~1.2s each). - Existing data-URL smoke test (`concurrent_navigations`) still passes. - Full workspace test suite (90 tests) still passes. Run with: cargo test -p obscura-cdp --test concurrent_navigations_with_fetch \ -- --nocapture --ignored Honest caveat. This is still the lock/serialize approach — it converts the abort into latency. The properly concurrent fix (Option 2 in issue #19: pin each `JsRuntime` to its own OS thread and message-pass) remains the right long-term direction. * fix(cdp): bound deferred message queue to prevent OOM during stalled navigation PR #36 comment 4341743194: process_with_interception deferred foreign CDP messages into an unbounded VecDeque while a navigation task was in flight. A stalled navigation (e.g. waiting for Fetch interception resolution) could accumulate arbitrary queued messages and OOM the process. Add MAX_DEFERRED_MESSAGES = 256. When the cap is reached, return an explicit CDP error (-32000) to the client instead of silently dropping or growing without bound. All 90 workspace tests pass.
Obscura
The open-source headless browser for AI agents and web scraping.
Lightweight, stealthy, and built in Rust.
Obscura is a headless browser engine written in Rust, built for web scraping and AI agent automation. It runs real JavaScript via V8, supports the Chrome DevTools Protocol, and acts as a drop-in replacement for headless Chrome with Puppeteer and Playwright.
Why Obscura over headless Chrome?
Designed for automation at scale, not desktop browsing.
| Metric | Obscura | Headless Chrome |
|---|---|---|
| Memory | 30 MB | 200+ MB |
| Binary size | 70 MB | 300+ MB |
| Anti-detect | Built-in | None |
| Page load | 85 ms | ~500 ms |
| Startup | Instant | ~2s |
| Puppeteer | Yes | Yes |
| Playwright | Yes | Yes |
🎉 10,000 stars and what's next
I'm working on Obscura Cloud the hosted version, with managed infrastructure, residential proxies, and dedicated support. For people who want the engine without operating it themselves.
The open-source engine stays Apache-2.0, fully featured. No feature gating, ever.
Install
Download
Grab the latest binary from Releases:
# Linux x86_64
curl -LO https://github.com/h4ckf0r0day/obscura/releases/latest/download/obscura-x86_64-linux.tar.gz
tar xzf obscura-x86_64-linux.tar.gz
./obscura fetch https://example.com --eval "document.title"
# macOS Apple Silicon
curl -LO https://github.com/h4ckf0r0day/obscura/releases/latest/download/obscura-aarch64-macos.tar.gz
tar xzf obscura-aarch64-macos.tar.gz
# macOS Intel
curl -LO https://github.com/h4ckf0r0day/obscura/releases/latest/download/obscura-x86_64-macos.tar.gz
tar xzf obscura-x86_64-macos.tar.gz
# Windows
Download the `.zip` from the releases page and extract it manually.
No Chrome, no Node.js, no dependencies. Release archives include both
obscura and obscura-worker; keep them in the same directory for the
parallel scrape command.
Linux release builds target Ubuntu 22.04 so the downloaded binary remains usable on common LTS servers with glibc 2.35+.
Build from source
git clone https://github.com/h4ckf0r0day/obscura.git
cd obscura
cargo build --release
# With stealth mode (anti-detection + tracker blocking)
cargo build --release --features stealth
Requires Rust 1.75+ (rustup.rs). First build takes ~5 min (V8 compiles from source, cached after).
Quick Start
Fetch a page
# Get the page title
obscura fetch https://example.com --eval "document.title"
# Extract all links
obscura fetch https://example.com --dump links
# Render JavaScript and dump HTML
obscura fetch https://news.ycombinator.com --dump html
# Write dump or eval output to a file
obscura fetch https://example.com --dump text --output page.txt
# Wait for dynamic content
obscura fetch https://example.com --wait-until networkidle0
# Bound navigation time for slow or broken pages
obscura fetch https://example.com --timeout 10
Start the CDP server
obscura serve --port 9222
# With stealth mode (anti-detection + tracker blocking)
obscura serve --port 9222 --stealth
Scrape in parallel
obscura scrape url1 url2 url3 ... \
--concurrency 25 \
--eval "document.querySelector('h1').textContent" \
--format json
# Suppress scrape progress on stderr for script-friendly output
obscura scrape https://example.com --quiet --format json
Puppeteer / Playwright
Puppeteer
npm install puppeteer-core
import puppeteer from 'puppeteer-core';
const browser = await puppeteer.connect({
browserWSEndpoint: 'ws://127.0.0.1:9222/devtools/browser',
});
const page = await browser.newPage();
await page.goto('https://news.ycombinator.com');
const stories = await page.evaluate(() =>
Array.from(document.querySelectorAll('.titleline > a'))
.map(a => ({ title: a.textContent, url: a.href }))
);
console.log(stories);
await browser.disconnect();
Playwright
npm install playwright-core
import { chromium } from 'playwright-core';
const browser = await chromium.connectOverCDP({
endpointURL: 'ws://127.0.0.1:9222',
});
const page = await browser.newContext().then(ctx => ctx.newPage());
await page.goto('https://en.wikipedia.org/wiki/Web_scraping');
console.log(await page.title());
await browser.close();
Form submission & login
await page.goto('https://quotes.toscrape.com/login');
await page.evaluate(() => {
document.querySelector('#username').value = 'admin';
document.querySelector('#password').value = 'admin';
document.querySelector('form').submit();
});
// Obscura handles the POST, follows the 302 redirect, maintains cookies
Benchmarks
Page load:
| Page | Obscura | Chrome |
|---|---|---|
| Static HTML | 51 ms | ~500 ms |
| JS + XHR + fetch | 84 ms | ~800 ms |
| Dynamic scripts | 78 ms | ~700 ms |
Stealth Mode
Enable with --features stealth.
Anti-fingerprinting
- Per-session fingerprint randomization (GPU, screen, canvas, audio, battery)
- Realistic
navigator.userAgentData(Chrome 145, high-entropy values) event.isTrusted = truefor dispatched events- Hidden internal properties (
Object.keys(window)safe) - Native function masking (
Function.prototype.toString()→[native code]) navigator.webdriver = undefined(matches real Chrome)
Tracker Blocking
- 3,520 domains blocked
- Blocks analytics, ads, telemetry, and fingerprinting scripts
- Prevents trackers from loading entirely
- Enabled automatically with
--stealth
CDP API
Obscura implements the Chrome DevTools Protocol for Puppeteer/Playwright compatibility.
| Domain | Methods |
|---|---|
| Target | createTarget, closeTarget, attachToTarget, createBrowserContext, disposeBrowserContext |
| Page | navigate, getFrameTree, addScriptToEvaluateOnNewDocument, lifecycleEvents |
| Runtime | evaluate, callFunctionOn, getProperties, addBinding |
| DOM | getDocument, querySelector, querySelectorAll, getOuterHTML, resolveNode |
| Network | enable, setCookies, getCookies, setExtraHTTPHeaders, setUserAgentOverride |
| Fetch | enable, continueRequest, fulfillRequest, failRequest (live interception) |
| Storage | getCookies, setCookies, deleteCookies |
| Input | dispatchMouseEvent, dispatchKeyEvent |
| LP | getMarkdown (DOM-to-Markdown conversion) |
CLI Reference
obscura serve
Start a CDP WebSocket server.
| Flag | Default | Description |
|---|---|---|
--port |
9222 |
WebSocket port |
--proxy |
— | HTTP/SOCKS5 proxy URL |
--stealth |
off | Enable anti-detection + tracker blocking |
--workers |
1 |
Number of parallel worker processes |
--obey-robots |
off | Respect robots.txt |
obscura fetch <URL>
Fetch and render a single page.
| Flag | Default | Description |
|---|---|---|
--dump |
html |
Output: html, text, or links |
--eval |
— | JavaScript expression to evaluate |
--wait-until |
load |
Wait: load, domcontentloaded, networkidle0 |
--timeout |
30 |
Maximum navigation time in seconds |
--selector |
— | Wait for CSS selector |
--stealth |
off | Anti-detection mode |
--output |
— | Write dump or eval output to a file |
--quiet |
off | Suppress banner |
obscura scrape <URL...>
Scrape multiple URLs in parallel with worker processes.
| Flag | Default | Description |
|---|---|---|
--concurrency |
10 |
Parallel workers |
--eval |
— | JS expression per page |
--format |
json |
Output: json or text |
--quiet |
off | Suppress scrape progress on stderr |
License
Apache 2.0