Commit Graph
73 Commits
Author SHA1 Message Date
SGavrl c8a5477296 Improve modern rendering and output parity 2026-08-03 19:46:24 +00:00
SGavrl 51a3eb1a81 Prepare fast document-space captures 2026-08-03 16:33:47 +00:00
SGavrl 9fac700464 Load stylesheet graphs through page transport 2026-08-03 15:53:29 +00:00
SGavrl edb1785313 Implement document title and referrer semantics 2026-08-03 13:46:11 +00:00
SGavrl f1b1f5af5e Trust client-side hydration mutations 2026-08-03 13:35:09 +00:00
SGavrl 41a8e1c1a6 Preserve script execution state across DOM moves 2026-08-03 11:49:08 +00:00
SGavrl 34373c309e Support current document import maps 2026-08-03 11:25:54 +00:00
SGavrl c69fafae21 Fix modern stylesheet and image geometry 2026-07-31 08:59:43 +00:00
SGavrl b5b141353b Preserve linked stylesheet cascade positions 2026-07-31 04:35:38 +00:00
SGavrl f61493fbd1 Reject unsuccessful script responses 2026-07-30 23:25:09 +00:00
SGavrl 5057715358 Share prepared layout with CSSOM 2026-07-30 22:08:01 +00:00
SGavrl 10b8492116 Improve modern layout and viewport rendering 2026-07-30 20:24:57 +00:00
SGavrl f7fec080ad render: preserve stylesheet asset bases 2026-07-26 22:33:44 +00:00
SGavrl 9f33db610e test: align paired browser settle timing 2026-07-26 14:41:10 +00:00
SGavrl 3618549842 render: unify responsive geometry and inline gradients 2026-07-26 11:51:44 +00:00
SGavrl 228a5723ce render: support modern framework content and geometry 2026-07-26 10:50:31 +00:00
SGavrl 5de5d07c24 render: support streamed framework hydration
Make attribute maps live, preserve parser script execution state, and expose
element scrolling used by React commit callbacks. Connect fallback web streams
so React Router and Remix hydration payloads flow through codec transforms.
2026-07-23 10:59:51 +00:00
SGavrl 8fa387faed render: respect stylesheet gating (disabled/media) and follow @import
Skip <link rel=stylesheet> sheets that are disabled or gated to
print/dark, and resolve @import chains inside fetched sheets so the
imported CSS reaches the cascade.

- disabled links (JS-toggled alternate themes) no longer paint pages
  dark under a light Chromium
- media="(prefers-color-scheme: dark)" link sheets are skipped
- @import url(...) is fetched recursively; Sphinx docs pull their layout
  from basic.css via classic.css and it was being dropped

Pixel fidelity vs headless Chromium: docs.python 95.5 to 10.2 percent,
web.dev 91.1 to 10.5, developer.chrome 88.0 to 18.9. No regressions on
the good-site set; obstacle course still 33/33.
2026-07-16 11:56:28 +00:00
SGavrl 58dee6330e browser: skip print-only stylesheets when fetching linked CSS
A <link rel=stylesheet media="print"> must not apply on screen; fetching
and injecting it let print rules override the screen layout (e.g. the
Guardian's print sheet). Skip links whose media is print-only.
2026-07-11 20:36:58 +00:00
SGavrl 920977776c Fix render engine: CSS grid, cascade correctness, SVG, inline flow
The render layer had no CSS Grid support, so Vector 2022's page layout
  (three nested grids) collapsed to a vertical stack with the article
  landing tens of thousands of pixels down the page. Wikipedia rendered
  as a near-blank page and Hacker News only worked through per-site
  hardcoded CSS overrides.

  Removed the HN-specific hardcoding and fixed the general engine instead:

  - CSS Grid layout (grid-template-areas/columns/rows via taffy's grid mode)
  - Indexed, specificity-correct cascade in a new css.rs, replacing the
    old test-every-selector-against-every-node scan. Cascade time on
    Wikipedia went from 9.5s to under 200ms.
  - Ancestor bloom-filter fast-rejection for selector matching in
    obscura-dom, needed to make the indexed cascade actually fast on
    descendant combinators.
  - :link/:visited pseudo-classes were previously unparseable, so any
    rule using them (a:link{color:...}, extremely common) silently
    dropped the whole rule from the cascade.
  - overflow:hidden clipping, fixing the pervasive visually-hidden-but-
    accessible pattern used for skip links and screen-reader-only labels.
  - @media parser robustness: it did a literal substring search for
    max-width: and silently dropped the constraint on max-width : 750px
    (space before colon) or max-width:calc(750px - 1px), treating
    mobile-only or narrow-viewport rules as always active. The calc()
    case was hiding Wikipedia's TOC sidebar via a rule meant only for
    narrow screens.
  - Table auto-layout approximation: the trailing auto-width td in a row
    absorbs leftover space, so a width:100% table's unsized last column
    reaches the right edge instead of shrinking to content.
  - Inline formatting context approximation: a block holding inline-level
    children (a paragraph or list item with links) is promoted to a
    wrapping flex row so text and links read as continuous prose instead
    of one fragment per line.
  - Whitespace-only text nodes no longer get a full line-height box. They
    used to stack up as blank rows, turning Wikipedia's collapsed TOC
    subsections and the page header into hundreds of pixels of dead space.
  - SVG rendering via resvg/usvg, plus resolving relative image URLs
    against the page's base URL.
  - background-image support including background-size/background-position
    for correctly sized and positioned icons.
  - Per-screenshot image fetch cache.
  - Fixed the layout-only (no paint feature) build, which didn't compile
    due to an unconditional call into paint-gated code.
2026-07-03 12:04:03 +00:00
SGavrl cdd52ade43 render: PNG screenshots via CLI and CDP
End-to-end rasterization behind the render feature (which now enables
obscura-render/paint):

- obscura-render: screenshot_png(dom, viewport) paints and encodes to PNG
  via tiny-skia's encode_png (no extra dep).
- obscura-browser: Page::screenshot renders the live DOM at a viewport.
- CLI: obscura fetch --screenshot out.png writes the settled page to a PNG.
- CDP: Page.captureScreenshot returns the PNG base64-encoded, so
  Puppeteer/Playwright page.screenshot() works. captureSnapshot still
  errors (it is a DOM/layer snapshot, not an image).

Verified: both paths produce correct 1280x720 RGBA PNGs with elements
laid out and colored as the CSS specifies (block stacking, backgrounds,
white page). Default build unchanged; obstacle course 33/33.

Text, borders, and images are not yet painted; the box layout and
background colors are rendered.
2026-07-01 21:34:15 +00:00
SGavrl 399181b975 render: expose real geometry to JS behind the render feature
getBoundingClientRect, offsetWidth/Height, and offsetTop/Left now read
real box geometry from the obscura-render layout cache when the render
feature is compiled in. elementFromPoint inherits correct hit-testing
because it already delegates to getBoundingClientRect per element.

Adds a render feature on obscura-js (propagated through obscura-browser
and obscura-cli), a lazily-computed layout cache and viewport in
ObscuraState (feature-gated, cleared on navigation), and an
op_layout_geometry op. bootstrap.js probes the op with typeof, so the
default scraping build is unchanged and falls through to the synthetic
rect.

With --features render, block siblings stack and flex children place
horizontally at correct positions, and elementFromPoint resolves the
right element. Default build returns the synthetic rect unchanged.
Obstacle course 33/33; obscura-js unit failures identical to main.
2026-06-30 20:36:43 +00:00
lisa0314 a6bb7411b2 fix(js): let dynamic scripts finish settling
Keep pumping the event loop past the general 500ms settle limit only while the dynamic-script queue remains pending, with a 3s hard cap.

Add a deterministic CDP regression for direct and nested external scripts whose fetch crosses the original settle deadline.
2026-07-24 15:03:53 +08:00
SGavrl 409ac781d2 Isolate CDP browser contexts 2026-07-22 10:45:36 +00:00
sycatle 319c6034fe revert(js): drop the "cancellation-safe" event loop pumping
The premise was wrong. deno_core 0.350's run_event_loop is:

    pub async fn run_event_loop(&mut self, opts) -> Result<(), CoreError> {
        poll_fn(|cx| self.poll_event_loop(cx, opts)).await
    }

so replacing `timeout(budget, run_event_loop())` with a hand-rolled
`poll_fn(poll_event_loop)` loop is the same code with the same cancellation
semantics, not safer ones. The mechanism I described cannot happen either:
`poll` is a synchronous `&mut self` call, a future can only be dropped between
polls, and the HandleScope/ContextScope live inside poll_event_loop as stack
locals that are destroyed before it returns. The isolate is exited between
polls both before and after that change.

What actually fixes #430 is confining each connection's isolates to their own
OS thread, which the next commits do. The regression test still reproduces the
abort on the pre-thread-per-connection tree and still passes here without this
commit, so nothing depended on it.

It was not merely a no-op. Two regressions it introduced:

- resolve_promises_until inverted the meaning of tick_ms. It was "pump the
  event loop for up to tick_ms, returning early on idle"; it became one single
  poll followed by an unconditional sleep(tick_ms). With the 1->50ms backoff
  that settles at 20 event-loop turns per second, so an awaited promise chain
  needing many turns crawls, and past the budget the caller silently reads a
  stale __obscura_await_meta.
- drive_module_eval replaced `e?` with `let _ = poll_event_loop(...)`,
  discarding event-loop errors. A module stalled on a top-level await used to
  exit immediately with a logged error; it instead spun for the whole budget
  logging nothing -- the exact #374 starvation the surrounding comment claims
  to prevent.

Keep the drive_module_eval extraction (it genuinely dedupes load_module and
load_inline_module) but restore the original select!-based body.

Refs #430.
2026-07-20 18:41:00 +02:00
sycatle 1f5bffc4de refactor: drop dead code and refresh comments after the per-connection move
- Delete the now-unused CdpContext::new_full (its only caller switched to
  new_with_shared_context); have _new_inner build the shared BrowserContext and
  delegate, so the struct's field list lives in one place.
- Extract drive_module_eval, shared verbatim by load_module and
  load_inline_module.
- Update comments that still described the deleted process-wide V8 lock and the
  single-LocalSet model to the per-connection reality, drop the dangling
  reference to the removed obscura-js/src/v8_lock.rs, and soften the
  ISOLATE_CREATE_LOCK note now that the main-thread warmup is the real guard
  against the InitializeBuiltinJSDispatchTable segfault.

Refs #430.
2026-07-20 16:37:51 +02:00
sycatle 45df429d99 fix(js): drive the event loop cancellation-safely to avoid isolate leaks
Several event-loop operations bounded their runtime with
tokio::time::timeout(budget, run_event_loop()) or a select! against a
timer. When the timer wins, tokio drops the run_event_loop future
mid-poll; deno_core enters the isolate while polling, so the drop leaves
the isolate entered on the OS thread. Under concurrent CDP work a second
page's execute_script then trips heap->isolate() == TryGetCurrent() and
aborts the process (#430).

Drive the loop with poll_event_loop inside a poll_fn instead. It creates
and drops its own isolate scope per call, so stopping between ticks never
leaves the isolate entered. Convert resolve_promises_until,
run_event_loop_bounded, resolve_promises, the mod_evaluate settle in
load_module and load_inline_module, and the two navigation settle loops
in page.rs. Add poll_event_loop_once as the cancellation-safe single-tick
primitive the settle loops use.

Keeps the obstacle course at 33/33. This is hardening toward #430; the
residual concurrency abort still needs the thread-per-runtime refactor
that v8_lock.rs calls the "Option 2" fix.

Refs #430.
2026-07-20 16:37:50 +02:00
Jim Zhang @ wonderhoi f5dc97b131 fix(net): scope passive callbacks to the registering page (#414)
The on_request/on_response Vec lived on the ObscuraHttpClient shared by
every page in the context, so a callback registered on one page fired for
all pages (#408 cross-page bleed) and outlived its page.

Callbacks now live in a per-page CallbackRegistry (Arc shared with the
page's JS runtime state). Page-driven fetch paths (navigation, POST,
scripts, stylesheets, robots.txt, module loads, JS fetch/XHR in both
plain and stealth modes) pass the registry through; the HTTP client
itself is callback-free.

Adds callbacks_do_not_bleed_across_pages: page A's callback stays silent
for page B's requests and stops firing after page A is dropped.
2026-07-16 12:43:13 +02:00
SGavrl 99ece0e12d feat(net): make passive request/response callbacks detachable (#416)
on_request/on_response stored callbacks in append-only Vecs with no way
to remove one, so a crawler that installed a response-body capture hook
for one phase could never detach it: it fired for the shared client's
whole lifetime, re-registering across navigations piled up duplicates,
and (because the client is shared per context) it saw sibling pages'
requests too.

Tag each callback with a stable id. on_request/on_response now return
the id and off_request/off_response (remove_on_request/remove_on_response
on the client) detach a specific one. Storage is Vec<(u64, Callback)>;
all fire sites destructure the tuple. Closes #408.
2026-07-14 13:57:45 +00:00
SGavrl d0b6fc4ffd fix(cdp): emit Network events for script-initiated requests (#415)
Requests made by page JS (fetch/XHR/dynamic script and stylesheet loads)
run in the V8 op layer and never produced a NetworkEvent, so CDP emitted
Network.requestWillBeSent / responseReceived only for the static
navigation subresources. Puppeteer/Playwright page.on('request') and
on('response') therefore missed every XHR/JSON response, which breaks
response capture (and is the root cause of the Aviasales half of #394).

op_fetch_url now records a JsNetworkEvent keyed by the same fetch-{N} id
under which it already stores the response body. The Page drains these
into its network_events before the CDP layer emits, so they flow through
emit_navigation_events as requestWillBeSent / responseReceived /
loadingFinished alongside the document subresources, and
Network.getResponseBody resolves for them via the shared id. Applied to
both navigation paths (server process_navigation and in-process
do_navigate). The queue is capped at 4096 events per page.

Closes #406.
2026-07-14 13:23:55 +00:00
Andrey Mnatsakanov c915a11f59 fix(cdp): truncate client-controlled log previews on char boundaries (#397)
server.rs logged previews of attacker-controlled strings with raw byte
slicing: `&text[..text.len().min(200)]` for a malformed CDP frame (inside a
`warn!`, which is active at the worker's default log level) and the same for
an intercepted event JSON. `&s[..n]` panics when byte `n` falls inside a
multi-byte UTF-8 character, so a single WebSocket frame that is invalid JSON,
>=201 bytes, with a multi-byte char straddling byte 200 panics the CDP
processor task — an unauthenticated one-frame crash. page.rs had the same
pattern (behind --verbose debug logging) for evaluated JS expressions.

Add truncate_on_char_boundary (str::floor_char_boundary is still unstable) and
route all four sites through it. Regression test constructs the exact 'a'*199 +
'€' input, asserts the old raw slice panics (via catch_unwind) and the helper
truncates on the boundary instead.
2026-07-13 22:45:57 +00:00
SGavrl 1ca6047351 feat(cdp): stream large response bodies via takeResponseBodyAsStream + IO (#381)
Network.getResponseBody returns the whole body in one base64 blob and the
cache drops anything over 2 MiB, so large downloads (images, PDFs,
archives) either failed or forced the long-running server to hold the
full body in memory (issue #360).

Add Fetch.takeResponseBodyAsStream + IO.read + IO.close. The body is moved
out of the page cache into a stream handle and delivered in bounded chunks
via IO.read, then freed on IO.close, so it is held once and released after
the download. Binary bodies are byte-exact (stored base64). To stream a
large body, raise OBSCURA_NETWORK_BODY_BUFFER_BYTES so it is cached.

This is entirely opt-in: nothing runs unless a client calls the new
methods, so the default fetch/render/scrape path is unchanged (verified:
obstacle course 33/33, obscura-vs-Chrome head-to-head within noise of the
pre-change baseline).

Fixes #360.
2026-07-10 20:38:28 +00:00
SGavrl fa9ba7f13d fix(browser): raise script-execution budget so heavy SPAs boot (#371)
The script-execution phase had a 10s soft deadline. On a heavy SPA the
remaining scripts were skipped before the app finished booting, so it
never fired its XHR/fetch calls and page.on('response') captured zero API
responses (issue #361). The knob to extend it, OBSCURA_SCRIPT_DEADLINE_MS,
existed but was undocumented, so it was undiscoverable.

Raise the default to 30s and document the env var. Only pages that keep
running past the deadline are affected; fast pages finish and return well
before it, so the larger budget costs them nothing, and the per-phase
watchdog (armed at deadline + 1s) still bounds a real synchronous hang.
Very heavy SPAs on slow networks can raise it further with
OBSCURA_SCRIPT_DEADLINE_MS, paired with a matching client nav timeout.

Fixes #361.
2026-07-09 11:07:34 +02:00
SGavrl 630f9894f6 fix(cdp): expose main-resource response body and real mime on navigation
Navigating straight to an image, PDF, or other resource could not read the
main response body over CDP, and Page.frameNavigated always reported
text/html.

Two problems: the main resource's body was stored under its internal
request id, but the client sees it as the navigation loaderId (Chrome's
requestId === loaderId convention), so Network.getResponseBody(loaderId)
missed. And binary bodies were stored as UTF-8-lossy text, corrupting the
bytes. Alias the stored body to the loaderId, store non-text-like content
types (images, PDF, octet-stream) base64, and report the response's real
Content-Type in frameNavigated. HTML pages are unchanged.

Fixes #340.
2026-07-01 06:55:37 +00:00
SGavrl c19a21c142 api: honor interception Continue overrides and fix pre-navigation enable
op_fetch_url ignored the url/method/headers/body fields on
InterceptResolution::Continue, so the interception channel could pass, mock, or
block a request but not rewrite it. Apply them: shadow url/method/body and
derive custom_headers from the override after the channel resolves Continue.

Also fix enable_interception() called before the first navigation: init_js
propagated intercept_tx to the new runtime but not intercept_enabled, so the
runtime started with interception disabled and op_fetch_url never intercepted.
Propagate intercept_enabled in init_js as well.

Add a test that rewrites a fetch's url via Continue and asserts the rewritten
response is captured.
2026-06-24 21:51:21 +00:00
SGavrl 1f5066fc30 api: add request/response interception API to Page (#306)
Expose the interception infrastructure that already existed internally but was
private, so embedders can intercept, observe, mock, or block JS fetch()/XHR
without forking or unsafe casts.

On obscura::Page:
- add_preload_script: inject JS before any page <script> (CDP
  Page.addScriptToEvaluateOnNewDocument). The field already ran the scripts;
  this only adds the method.
- enable_interception: returns a receiver for every JS fetch()/XHR request;
  resolve each through its resolver with InterceptResolution::{Continue,
  Fulfill, Fail} to pass, mock, or block (CDP Fetch.enable equivalent).
- on_request / on_response: passive callbacks. on_response is the main path for
  capturing API response payloads from SPAs.

on_request/on_response previously fired only for navigation; op_fetch_url now
invokes them for JS fetch()/XHR too (both reqwest and stealth paths), gated on
a non-empty callback vector so the common no-callback case pays nothing.

Re-export InterceptedRequest, InterceptResolution, RequestInfo, ResourceType,
RequestCallback, ResponseCallback, Response from the obscura crate.

Integration test tests/interception.rs exercises all three against a local echo
server.
2026-06-24 21:36:01 +00:00
SGavrl 8582fcceca Merge stealth-overhaul into main
Resolved conflicts:
- page.rs: keep both set_blocked_urls (#315) and stealth client wiring
- ops.rs: keep SHA-512/224 + SHA-512/256 from stealth branch; keep
  the unreachable catch-all (vec![]) from #314 instead of silent
  SHA-256 fallback; updated comment accordingly
- bootstrap.js: extend the digest algorithm allowlist to include
  SHA-512/224 and SHA-512/256 to match the Rust additions
2026-06-23 18:05:55 +00:00
SGavrl 2e0981ef3f stealth: route scripted fetch()/XHR through the wreq client
op_fetch_url backed JS fetch() and XHR with a plain reqwest/rustls client, so
in stealth mode scripted requests carried a non-browser TLS fingerprint and
none of the Chrome client hints the main navigation sends. Bot managers that
score subresource requests (the AWS WAF challenge verify call, Akamai sensors)
read that as a script and reject it. Add StealthHttpClient::send_single and a
stealth_fetch_all path that op_fetch_url uses when a stealth client is
installed, preserving the redirect, SSRF, and CORS handling so JS-level
requests share the navigation's Chrome TLS fingerprint and sec-ch-ua client
hints. The reqwest path is unchanged for non-stealth. Adds a stealth feature to
obscura-js gated on obscura-net/stealth.
2026-06-23 10:28:33 +00:00
SGavrl 4309935abd stealth: keep the Chrome 145 identity consistent across TLS, JS, and WebGL
Stealth mode forced a Linux navigator UA and platform but reported Windows
in userAgentData.getHighEntropyValues and served Windows Direct3D11 WebGL
renderers, a combination no real Linux Chrome ever produces and that
cross-layer fingerprint checks flag. Present one consistent Windows Chrome
145 identity instead: the wreq emulation uses Platform::Windows, and the
navigator surfaces read the UA and platform from shared constants so the
HTTP User-Agent header, sec-ch-ua-platform, navigator.userAgent, and
userAgentData all agree. userAgentData brands are now derived from the
Chrome major version via Chromium's GREASE algorithm rather than a single
hardcoded token, and a Linux WebGL renderer pool is added so a Linux UA can
never report a Direct3D11 renderer.
2026-06-22 22:19:16 +00:00
RohitMadhu 3bf0da2fae Add Network.setBlockedURLs support (#315)
Added Network.setBlockedURLs dispatch handling.
Stores blocked URL patterns on each page.
Applies blocking to page subresources and JS fetch / XHR requests.
Supports wildcard CDP-style patterns like *://*.gstatic.com/*.woff2.
2026-06-22 11:02:49 +00:00
SGavrl ab27e9153d cli: list fetch()/XHR resources in --dump assets
--dump assets only walked static DOM attributes (script[src], img[src], ...),
so resources a page pulled in with fetch() or XHR never showed up (issue #301).

Record each absolute URL requested through op_fetch_url in ObscuraState,
expose it via the runtime and Page, and append those URLs to --dump assets as
type "fetch", deduped against URLs already listed from the DOM.
2026-06-17 16:36:08 +00:00
Rohit Madhu 570ca1544c add Network.getResponseBody support 2026-06-13 20:26:26 -04:00
SGavrl 56ab54e615 stealth: gate JS behaviours, upgrade wreq, fix fp consistency
wreq upgraded to 6.0.0-rc.29 and wreq-util to 3.0.0-rc.12 to apply the
latest Chrome 145 TLS fingerprint (16 cipher suites, proper extension
order, ALPS, GREASE).

__obscura_stealth global variable introduced and passed from Rust to JS,
allowing stealth-specific JS behaviors to run only when the stealth flag
is enabled.

Tightened hardwareConcurrency and deviceMemory pools in stealth mode
to desktop-realistic values (e.g. min 4GB RAM, 4 cores), eliminating
mobile-tier values that leak the spoof.

navigator.userAgentData and getHighEntropyValues() are now overridden
in stealth mode to perfectly match the sec-ch-ua header sent by
wreq_util::Profile::Chrome145.

Fixed usedJSHeapSize sometimes exceeding totalJSHeapSize.
2026-06-11 17:22:50 +00:00
SGavrl ac75cf52a6 stealth: fix platform/UA mismatch and hide internal globals from Object.keys
navigator.platform returned "Win32" while navigator.userAgent said Linux when
using the stealth client. page.rs now calls set_platform("Linux x86_64", "Linux", "")
alongside set_user_agent when the stealth client is active.

bootstrap.js: pre-declare 71 internal _* globals as enumerable:false before
var runs, so they don't show up in Object.keys(window) on real pages.
2026-06-11 09:35:58 +00:00
SGavrl 82b7a0baa8 Make navigator.geolocation coordinates configurable (#272)
The geolocation shim returned a fixed Frankfurt position. With the timezone now
configurable, a deployment in another region would report a location that does
not match its exit IP or timezone, the mismatch issue #228 describes.

Set OBSCURA_GEOLOCATION="lat,lon" to override the reported coordinates; the
value is range-checked and falls back to the Frankfurt default when unset or
malformed. The existing positional jitter is preserved.
2026-06-08 21:40:38 +00:00
johnnogueiraandSGavrl a23dad7cd9 Stealth profile consistency (#269)
* feat(stealth): randomize browser profile per session

Add a pool of 8 realistic browser profiles (Win10/Win11/macOS, Chrome
143-146). One is picked randomly per BrowserContext instead of always
advertising Linux, which is a strong bot signal.

Each profile keeps navigator.platform, userAgentData.platform, and
userAgentData.platformVersion internally consistent with the UA string.

- profiles.rs: static PROFILES pool + random_profile() picker
- context.rs: pick profile on construction, store platform fields
- page.rs: call rt.set_platform() alongside rt.set_user_agent()
- runtime.rs: add set_platform() to inject __obscura_platform globals
- bootstrap.js: read platform/userAgentData from globals (default Win32)

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* feat(stealth): match GPU renderer pool to OS profile

Windows profiles get ANGLE Direct3D11 renderers; macOS profiles get
ANGLE Metal renderers. Vendor strings updated to match (Google Inc. (Apple)
for Apple Silicon/Intel Mac, Google Inc. (NVIDIA/AMD/Intel) for Windows).

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix(stealth): navigator.webdriver returns false instead of undefined

Browsers set navigator.webdriver to false when automation is not active.
Returning undefined is a detectable bot signal — fingerprinting scripts
check typeof navigator.webdriver === 'boolean' as part of headless detection.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* Make stealth profile stable by default, rotation opt-in

Picking a random profile per BrowserContext meant one address would cycle
through different browser identities, which is itself a bot signal, and the
rotated profile carries no matching TLS or timezone fingerprint. Default to a
single stable profile; opt into rotation with OBSCURA_ROTATE_PROFILE, or pin
one with OBSCURA_PROFILE=<index>.

Drive timezone from the process TZ so Intl.DateTimeFormat and Date report one
consistent zone. Default Europe/Berlin; override with OBSCURA_TIMEZONE.

Update navigator.webdriver test to expect false, matching real Chrome.

---------

Co-authored-by: SGavrl <[email protected]>
2026-06-08 21:23:50 +00:00
SGavrl 3fdf0a0be4 perf: bound slow ES modules with a DOM-rendered-aware budget
A slow non-essential ES module could block navigation for ~10s. On real
pages (www.ycombinator.com) obscura spent ~11.5s loading YC's bookface
module, whose top-level eval idle-waits forever, until the phase watchdog
killed it. The page renders fully without it (CPU was ~3s, the rest pure
idle), but wall time was ~2.5x lightpanda.

load_module / load_inline_module now take a per-module budget and bound
both the import-graph fetch and the eval loop with it. page.rs sizes the
budget by whether the body is already rendered: an enhancement module on
a populated page (>50 body descendants) gets a short budget (3s, override
OBSCURA_MODULE_BUDGET_MS); an unmounted SPA shell keeps the full script
budget so the app module still mounts (#205). Fetch concurrency is
unchanged; only the idle eval wait is bounded.

www.ycombinator.com: wall 13.1s -> 6.6s, still 475 links, CPU unchanged.
teragrowth.org (Vite SPA) still renders. Obstacle course 33/33 including
the es-modules / ssr-hydrate / spa stages. Post-drop eval works.
2026-06-07 12:57:31 +02:00
SGavrl 5135e3cb4d perf: DOM read fast paths (constant nodeType, borrowed op reads, base64 table) (#259)
* DOM read fast paths: constant nodeType, fewer ops and allocations

- Element.nodeType is now a constant getter instead of inheriting Node's
  dynamic op-backed getter. nodeType is invariant (1) for element wrappers
  (wrappers are only built for element nodes and node ids are never freed
  and reused), so this drops one JS/Rust op per read on a pervasive path.
  Micro-bench: ~600k nodeType reads go from ~380ms to ~5ms.
- attributes getter reads each attribute once instead of twice.
- dataset Proxy is cached per element instead of reallocated on every
  access, which also makes el.dataset === el.dataset (spec-correct).
- stylesheet <link> scan borrows the node (with_node) instead of deep
  cloning it and compares rel via eq_ignore_ascii_case, dropping a clone
  and a lowercased String alloc per link per navigation.

No regression: obstacle course 33/33, WPT dom/nodes Node- subset identical
to baseline (3551/4163 subtests), targeted DOM probe identical.

* DOM op reads: borrow the node instead of deep-cloning it

The scalar read arms in op_dom (node_type, node_name, tag_name,
get_attribute, attribute_names, parent/first/last/next/prev lookups,
has_child_nodes, pi_target, doctype_name, doctype_public_id) and the
node_root / node_ancestors / node_child_index walks called get_node,
which clones the whole Node (for elements a deep clone of the attribute
Vec and its Strings) just to read one field. Switch them to with_node,
which borrows under the RefCell. element_children and the
document_element / document_doctype child scans stay on get_node on
purpose (they iterate, so the borrow story is left untouched).

Output is identical. Traversal/attribute micro-bench ~25-30% faster with
fewer allocations per op. No regression: obstacle course 33/33, WPT
dom/nodes identical to baseline (7057/10620 across 314 files).

* base64 decode: reverse-lookup table instead of indexOf scan

_base64ToUint8Array decodes every fetch/XHR response body. It ran an
O(64) alphabet.indexOf plus a one-char substring alloc per character.
Build a charCode->6-bit reverse table once at module load and index it by
charCodeAt, folding out-of-range (NaN or code >= 128) to -1 so the
bitmath stays byte-identical to the old indexOf path on valid and
malformed input alike.

~20-25% faster decoding a 2MB body; output byte-identical (differential
test across valid, unpadded, '='-position, >127, non-alphabet and
truncated inputs matches the old code exactly). No regression: obstacle
course 33/33.
2026-06-07 11:39:18 +02:00
SGavrl 1a0644f7bc feat(cdp): bound every CDP command with a shared V8 watchdog (#250)
A hung page (a runaway Runtime.evaluate, a synchronous DOM op) holds the
process-wide V8 lock and wedges every other session forever. The one-shot CLI has
a process-level hard deadline for this; the long-running server cannot force-exit,
so terminate just the offending isolate instead.

dispatch() arms a watchdog on the session page's isolate after taking the V8 lock
and disarms it after the handler; on overrun it terminates the isolate and clears
the termination flag so the session keeps working. OBSCURA_CDP_COMMAND_TIMEOUT_MS
tunes the bound (default 60s, 0 disables); navigation already self-bounds well
under it.

The watchdog is a single long-lived shared thread (cdp_watchdog), not a thread
per command: the V8 lock serializes commands so at most one is ever armed, and
arm/disarm are a mutex plus notify with no measurable per-command cost. Also
fixes a lost-wakeup in the per-call watchdog (spawn_watchdog) where disarming a
fast call before the watchdog thread started blocked for the full budget.

Verified: a hung Runtime.evaluate is terminated at the budget and the session
recovers; CDP integration 37/37, obstacle course 33/33; per-command latency
unchanged (watchdog on equals off within noise).
2026-06-05 21:28:21 +00:00
zhangyang-crazy-oneandzhangyang-crazy-one 98b62e53e8 fix(api/js): Browser cookie jar + base href + Node.baseURI + XHR EventTarget + Page::settle (closes #241,#242,#244,#245,#247) (#243)
* fix(api): Browser::cookies() now shares context's CookieJar

Previously Browser::build() created its own empty CookieJar,
while BrowserContext internally creates a separate jar that
actually receives Set-Cookie headers from HTTP responses.

This caused browser.cookies().get_all() to always return
empty after navigation — the populated jar was unreachable.

Fix: use context.cookie_jar instead of creating a separate
one. Also removes redundant cookie-loading logic in
Browser::build() since BrowserContext::_new_inner already
handles loading from storage_dir.

Closes #241

* fix(browser): add <base href> support for script URL resolution (closes #242)

Previously resolve_base_url() returned the document URL directly,
causing relative <script src> URLs to resolve against the page URL
rather than the <base href> target. On SPAs served from sub-paths
(Angular/Vite/Next.js), this caused all bundle scripts to 404.

Add resolve_base_url() that queries DOM for <base href> and uses it
as the base for relative URL resolution per HTML spec. Applied to
both regular scripts and ES modules in execute_scripts().

* fix(js): implement Node.baseURI getter per WHATWG DOM spec (closes #244)

Node.baseURI was missing from the bootstrap.js Node class, causing
document.baseURI to return undefined. This broke Angular's
PlatformLocation.getBaseHref which calls new URL(href, document.baseURI)
-- with baseURI empty, the URL constructor throws Invalid URL.

Per https://dom.spec.whatwg.org/#dom-node-baseuri, Node.baseURI must
return the href of the first <base href> resolved against the document
URL, falling back to the document URL when no <base> is present.

* fix(js): add XMLHttpRequestEventTarget + XHR dispatchEvent (closes #245)

XHR shim lacked dispatchEvent and XMLHttpRequestEventTarget, causing
zone.js (Angular/zone.js) to crash with 'Cannot read properties of
undefined (reading call)' when Angular's HttpClient makes XHR calls.

Add spec-compliant XMLHttpRequestEventTarget base class with
addEventListener/removeEventListener/dispatchEvent. Make XHR extend
it so zone.js can walk the proto chain correctly.

* fix(js): fix XHR super() call order in ES6 class extends

* fix(js): use base href for dynamic script/module URL resolution

Angular lazy-loaded chunks were resolving against document.location.href
instead of the <base href>. This caused all dynamic import() chunks on
the Deutsche Boerse SPA to 404 (/bond/7591.xxx.js instead of /7591.xxx.js).

Fix: resolve dynamic script src against <base href> when present, falling
back to location.href per HTML spec.

* Revert "fix(js): use base href for dynamic script/module URL resolution"

This reverts commit 88af239be38f2aa11c3c978a4db2a7482f2ed6c0.

* feat(api): expose Page::settle() to pump JS event loop (closes #247)

obscura_browser::Page already had settle(max_ms) but it was unreachable
from the public API. Exposing it allows callers to resolve Promises,
fetch callbacks, and microtasks before reading results with evaluate().

---------

Co-authored-by: zhangyang-crazy-one <[email protected]>
2026-06-05 20:52:39 +02:00