Commit Graph
218 Commits
Author SHA1 Message Date
Andrey MnatsakanovandSGavrl b2e4bb49a7 fix(js): structuredClone cycles through Error.cause and clones own __proto__ (#419, #420) (#421)
* fix(js): structuredClone cycles through Error.cause and clones own __proto__ (#419, #420)

The structuredClone polyfill had two correctness/parity gaps in _structuredClone:

- The Error branch recursed into `cause` before recording the clone in `seen`,
  so a cycle through the error (e.cause === e) recursed until the stack
  overflowed. Chrome clones such graphs and preserves identity. Fix: seen.set
  the clone before recursing into cause. (#419)

- The generic-object branch built the clone on the source's prototype and
  copied properties by assignment, so an own enumerable `__proto__` data
  property (exactly what JSON.parse('{"__proto__":…}') produces) went through
  the inherited __proto__ setter and reparented the clone instead of copying
  the property. Fix: clone plain objects onto Object.prototype (like Chrome)
  and define each property instead of assigning it. (#420)

Adds two parity tests to structured_clone_crypto_parity.rs; full obscura-cdp
suite (73) green.

* perf(js): keep structuredClone fast path for all keys but __proto__

The own-__proto__ fix copied every own property with Object.defineProperty
to avoid triggering the inherited prototype setter. That is ~3x slower
than plain assignment per key, so cloning a large object regressed
(~75ms -> ~228ms for a 1000-key object x200).

Only the __proto__ key needs defineProperty; every other key is safe to
assign. Special-case it so the fast path applies to all other keys.
Measured back to parity with main (85ms vs 85ms); the own-__proto__
correctness test still passes.

---------

Co-authored-by: SGavrl <[email protected]>
2026-07-16 12:56:10 +02:00
Jim Zhang @ wonderhoi f5dc97b131 fix(net): scope passive callbacks to the registering page (#414)
The on_request/on_response Vec lived on the ObscuraHttpClient shared by
every page in the context, so a callback registered on one page fired for
all pages (#408 cross-page bleed) and outlived its page.

Callbacks now live in a per-page CallbackRegistry (Arc shared with the
page's JS runtime state). Page-driven fetch paths (navigation, POST,
scripts, stylesheets, robots.txt, module loads, JS fetch/XHR in both
plain and stealth modes) pass the registry through; the HTTP client
itself is callback-free.

Adds callbacks_do_not_bleed_across_pages: page A's callback stays silent
for page B's requests and stops firing after page A is dropped.
2026-07-16 12:43:13 +02:00
Andrey Mnatsakanov ccfa5fb87b fix(js): validate the submitter argument to requestSubmit() (#424) (#426)
HTMLFormElement.requestSubmit(submitter) accepted any element and submitted
silently. Per the HTML spec (and Chrome) it must throw a TypeError when the
submitter is not a submit button, and a NotFoundError DOMException when the
submitter is not owned by the form; both checks run before the submit event.

Adds an _isSubmitButton predicate (a <button> whose type is submit — the
default, including a missing/invalid type — or an <input> of type submit/image)
and validates the argument at the top of requestSubmit. submit() is untouched:
it correctly bypasses the event (#395).

Element.click() now gates on the same predicate rather than its own looser
check, so an internal click can never hand requestSubmit a submitter it would
reject (e.g. <div type="submit">, which is not a submit button and no longer
submits — matching Chrome). That also aligns click() with the CDP click path in
input.rs, which already treats <input type=image> as a submit button. The
input.rs call sites are already guarded by equivalent conditions and pass no
submitter on the Enter-key path, so they are unaffected.

Constraint validation is deliberately out of scope: obscura has no validity
engine (checkValidity is a stub returning true), so wiring it here would be a
no-op. Noted in #424 as separate work.

Full obscura-cdp suite (72) green.
2026-07-16 12:42:42 +02:00
Andrey Mnatsakanov 8698afc065 fix(js): CryptoKey clone hook honours the structuredClone seen map (#423) (#425)
_structuredClone dispatches platform-object hooks as hook(value, seen), but the
CryptoKey hook took only `src` and never touched the memo, and hook branches
return before the generic seen.set. So one CryptoKey referenced twice in a graph
cloned into two distinct objects (and re-registered its key material each time),
where structuredClone preserves reference identity within a graph.

Consult and populate `seen` in the hook. Adds a parity test asserting a key
reached twice clones to one shared object that crypto.subtle still accepts.
2026-07-16 12:42:07 +02:00
Jim Zhang @ wonderhoi 891d850d19 fix(js): do not redefine img src when a page made it non-configurable (#418)
Booking.com's anti-bot script pre-defines a non-configurable own `src` on
<img> elements. The Image shim's load emulation then threw "TypeError:
Cannot redefine property: src" out of the constructor, crashing any page
that calls new Image() after that instrumentation runs.

Check the own descriptor first and skip the load emulation when `src` is
already non-configurable: a page that owns `src` is instrumenting loads
itself. The normal path (configurable src, emulated load event) is
unchanged and now covered by a test.

Fixes the crash half of #394.
2026-07-16 12:41:28 +02:00
SGavrl 99ece0e12d feat(net): make passive request/response callbacks detachable (#416)
on_request/on_response stored callbacks in append-only Vecs with no way
to remove one, so a crawler that installed a response-body capture hook
for one phase could never detach it: it fired for the shared client's
whole lifetime, re-registering across navigations piled up duplicates,
and (because the client is shared per context) it saw sibling pages'
requests too.

Tag each callback with a stable id. on_request/on_response now return
the id and off_request/off_response (remove_on_request/remove_on_response
on the client) detach a specific one. Storage is Vec<(u64, Callback)>;
all fire sites destructure the tuple. Closes #408.
2026-07-14 13:57:45 +00:00
SGavrl d0b6fc4ffd fix(cdp): emit Network events for script-initiated requests (#415)
Requests made by page JS (fetch/XHR/dynamic script and stylesheet loads)
run in the V8 op layer and never produced a NetworkEvent, so CDP emitted
Network.requestWillBeSent / responseReceived only for the static
navigation subresources. Puppeteer/Playwright page.on('request') and
on('response') therefore missed every XHR/JSON response, which breaks
response capture (and is the root cause of the Aviasales half of #394).

op_fetch_url now records a JsNetworkEvent keyed by the same fetch-{N} id
under which it already stores the response body. The Page drains these
into its network_events before the CDP layer emits, so they flow through
emit_navigation_events as requestWillBeSent / responseReceived /
loadingFinished alongside the document subresources, and
Network.getResponseBody resolves for them via the shared id. Applied to
both navigation paths (server process_navigation and in-process
do_navigate). The queue is capped at 4096 events per page.

Closes #406.
2026-07-14 13:23:55 +00:00
Jim Zhang @ wonderhoi b460b37e93 fix(js): expose Document.adoptNode and Element.toggleAttribute (#392)
adoptNode was only stubbed on the createHTMLDocument shim object, never on
Document.prototype, so document.adoptNode(node) threw. importNode already had a
Document.prototype polyfill returning the node, so adoptNode mirrors it: with no
second document to transfer ownership from the node is already ours, return it.

toggleAttribute was missing entirely. Implement spec semantics: no force arg
toggles, force true adds, force false removes, returns the new presence. Lit,
Stencil, and several ad SDKs call it and die on the missing method today.

Fixes #393.
2026-07-14 12:30:40 +00:00
SGavrl c8c78ba229 fix(js): fire load/error for dynamically inserted stylesheet links (#412)
A <link rel=stylesheet href> appended via JS (createElement +
appendChild) was never fetched and fired neither load nor error, so any
Promise.all awaiting the link's onload (lazy CSS loaders, antd/bootstrap
fetchers) hung forever and the page stalled at its pre-load stage.

appendChild already drives this for <script src>; mirror it for links:
resolve the href against <base>/document URL, fetch it, and dispatch
load on success or error on failure. CSS is not applied (no layout
engine yet) but the load-event contract matches Chrome.

Read the stylesheet intent from both the rel attribute and the rel
property, since obscura does not yet reflect the rel IDL attribute back
to the content attribute. Closes #409.
2026-07-14 12:13:25 +00:00
Jim Zhang @ wonderhoiandSGavrl a92166897f fix(js): real structuredClone so CryptoKey and buffers survive cloning (#390)
* fix(js): real structuredClone so CryptoKey and buffers survive cloning

structuredClone was JSON.parse(JSON.stringify), which silently drops
ArrayBuffer and TypedArray to {}. A cloned CryptoKey also lost its WeakMap
entry, so crypto.subtle.sign/encrypt/deriveBits threw "Argument is not a
valid CryptoKey" on the clone.

Replace the JSON fallback with a recursive cloner covering ArrayBuffer,
TypedArray, DataView, Map, Set, Date, RegExp, Error, and plain objects,
with cycle and own symbol-keyed property handling. CryptoKey opts in via a
clone hook (keyed by Symbol.toStringTag) that rebuilds the key through
makeKey, so the clone re-enters the WeakMap and stays usable.

Fixes #389.

* fix(js): structuredClone handles DataView and rejects functions and symbols

The TypedArray branch called value.slice(), which DataView lacks, so
cloning any DataView threw TypeError, a new crash in the buffers
category this feature targets. Copy the underlying buffer slice over
the view range and wrap a fresh DataView.

The function/symbol rejection lived after the primitive early-return
and was dead code, so functions and symbols were passed through by
reference instead of throwing DataCloneError (HTML structured clone).
Check first and throw.

Register ArrayBuffer, TypedArray, and DataView in the seen map so
repeated references share one clone, and move the cycle check up.

---------

Co-authored-by: SGavrl <[email protected]>
2026-07-13 23:21:44 +00:00
sickoandSGavrl 7e2cabf411 fix(js): HTMLFormElement.submit() must not fire a cancelable submit event; add requestSubmit() (#395)
* fix(js): HTMLFormElement.submit() must not fire a cancelable submit event; add requestSubmit()

Per the HTML standard, the submit() method submits without firing a submit
event and cannot be canceled by a page submit listener; only requestSubmit()
and user-initiated submits fire the cancelable event. bootstrap.js dispatched a
cancelable submit event from submit() and aborted navigation when canceled,
which breaks the common pattern where a submit listener preventDefault()s the
native submit and a callback later calls form.submit() to actually send it
(e.g. an invisible-reCAPTCHA data-callback): the listener also canceled the
programmatic submit, so no navigation was ever issued.

- submit(): navigate unconditionally, no submit event
- requestSubmit(): fire the cancelable submit event, navigate only if not canceled
- submit-button clicks route through requestSubmit() (a real click fires the event)

Adds a regression test; full obscura-cdp suite (56 tests) passes.

* fix(cdp): route synthetic submit-button clicks and Enter through requestSubmit

A CDP Input.dispatchMouseEvent on a submit button and an Enter key in a
form field are user-initiated submits, so per the HTML spec they must
fire the cancelable submit event. input.rs called form.submit() (the
method) directly, bypassing any preventDefault() submit listener, which
mirrors the bootstrap.js bug fixed in the parent commit. Use
requestSubmit() with a submit() fallback so the listener can veto.

---------

Co-authored-by: SGavrl <[email protected]>
2026-07-13 23:21:15 +00:00
Vincenzo Territo 87fefef1ec fix(js): move navigator fingerprint props onto Navigator.prototype (#388)
* Move navigator fingerprint props onto Navigator.prototype

Spoofed navigator properties (userAgent, appVersion, platform, language,
languages, plugins, mimeTypes, hardwareConcurrency, deviceMemory, webdriver,
share, canShare) lived as own-instance accessors/data props on the navigator
literal, which pixelscan's Navigator group flags via hasOwnProperty /
getOwnPropertyDescriptor. Real Chrome keeps them on Navigator.prototype.

Generalize the existing webdriver prototype IIFE to define every spoofed
accessor on a thin prototype one hop above Navigator.prototype, so
navigator.hasOwnProperty('platform') === false while navigator instanceof
Navigator stays true. Getters read the __obscura_* globals lazily (snapshot
vs per-page timing) and are _markNative'd so toString reports [native code].
plugins/mimeTypes are cached singletons.

hardwareConcurrency/deviceMemory now back onto __obscura_hw/__obscura_mem
globals written by __obscura_init, avoiding own data props. Those two globals
are pre-declared in _preHideInternals so the snapshot-time hide list captures
them and they stay invisible to reflection APIs.

* Tighten comments on navigator prototype fingerprint setup

Drop the long narrative blocks in bootstrap.js; keep short notes on why
props sit on the prototype hop, plugin caching, and hw/mem globals.
2026-07-13 22:46:45 +00:00
Andrey Mnatsakanov c2b79b67fe fix(js): fire trusted input/change from setFileInputFiles (#385)
DOM.setFileInputFiles dispatched plain input/change events, so
event.isTrusted was false. Upload flows and anti-bot code that gate their
change handler on isTrusted ignored them, which silently broke the exact
case the feature targets. The Input domain already marks its synthesized
clicks/keys trusted via __obscura_markTrusted; do the same here.

Fixes #384.
2026-07-11 16:37:10 +02:00
seffignoz 4c33f6d95e fix(stealth): hide internal globals from reflection and mark builtins native (#377)
* fix(stealth): hide internal globals from reflection and mark builtins native

Two anti-fingerprinting leaks let pixelscan's bot-check flag obscura as
automated even with --stealth:

- unusualWindowProperties: internal globals (_fp, _wrap, __obscura_*, ...)
  were only made non-enumerable, so they stayed hidden from Object.keys and
  for-in but not from Object.getOwnPropertyNames / Reflect.ownKeys, which
  fingerprinting scripts use. Filter __obscura_hide_list out of the reflection
  APIs (getOwnPropertyNames, ownKeys, keys, getOwnPropertyDescriptors) when
  they target the global object.

- tamperedFunctions: DOM/Web platform members reimplemented in JS leaked their
  source via toString(), e.g. Element.prototype.nodeType returned
  "get nodeType() {...}" instead of "[native code]". Per-member _markNative
  calls covered methods but missed property accessors and several constructors.
  Sweep every builtin constructor reachable from the global object at snapshot
  build time and mark its prototype methods, accessors, and the constructor
  itself native. Also name the Function.prototype.toString override so it
  reports the correct native form for itself.

Verified against pixelscan.net/bot-check: Webdriver -> Clear (tamperedFunctions
and unusualWindowProperties both Clear).

* fix(stealth): build hide list with getOwnPropertyNames

__obscura_hide_list was built with Object.keys, which omits the internal
globals that _preHideInternals already made non-enumerable. Those names were
therefore missing from the list and stayed visible to the reflection-API
filter and to fingerprinting scripts (e.g. __obscura_ua, _registerIframe).
Use getOwnPropertyNames so the list is complete.
2026-07-11 16:07:22 +02:00
SGavrl d3a8b9a482 fix(js): return module eval on completion, not loop idle (Vite mount) (#380)
load_module and load_inline_module drove run_event_loop to full idle
before reading the module's mod_evaluate result. A page timer keeps the
loop busy forever (Vite's HMR / React-Refresh client installs a
setInterval), so the preamble inline module burned the entire budget
waiting for an idle that never came, and the later module that mounts the
app was skipped, leaving #root empty while the production preview rendered
fine (issue #374).

Drive the event loop and the mod_evaluate result together and return as
soon as the module finishes evaluating, so a busy background timer no
longer starves the remaining modules. Post-eval async work is still
drained by the normal post-load settle.

Fixes #374.
2026-07-10 20:36:08 +00:00
SGavrl 6788996b0b feat(cdp): implement DOM.setFileInputFiles for file uploads (#369)
Puppeteer's ElementHandle.uploadFile and Playwright's setInputFiles drive
an <input type=file> through CDP DOM.setFileInputFiles, which obscura
rejected as an unknown method, so any upload flow failed (issue #359).

Implement it: resolve the node, read each local file, and hand the bytes
(base64) to the JS layer, which builds real File objects (backed by the
bytes, so page code can read them via FileReader or upload them via
fetch/FormData) and fires input+change like a genuine selection. The file
input exposes a FileList via .files, and .value reads C:\fakepath\<name>
to match Chrome.

Fixes #359.
2026-07-09 10:51:18 +02:00
SGavrl 5c3d560d8e test(js): lock script-error isolation for the babel-polyfill case (#358)
Regression test for #355: an uncaught `throw` in one inline <script> must
not prevent later independent <script>s from running. This was fixed by
the #147 poison-guard (present on main) but reported again against 0.1.9
with an explicit throw; this pins the exact scenario so it cannot regress.

Closes #355.
2026-07-07 09:35:49 +02:00
SGavrl ad7a7a93d1 fix(js): support in/Object.keys on style and dataset (CSSOM parity) (#357)
el.style (CSSStyleDeclaration) and el.dataset (DOMStringMap) were plain
Proxies with only get/set traps, so `'color' in el.style`, `Object.keys
(el.style)`, `'foo' in el.dataset`, and `Object.keys(el.dataset)` all
failed, and Object.keys(el.style) leaked the internal `_props` (issue
#356). cssText also serialized camelCase names without a trailing
semicolon.

Add has / ownKeys / getOwnPropertyDescriptor traps to both proxies so the
`in` operator and enumeration behave like real CSSOM. CSSStyleDeclaration
now stores properties by their dashed name (matching the spec), so
el.style.fontSize and getPropertyValue('font-size') stay in sync, and
cssText serializes dashed names with a trailing semicolon. A known CSS
property list backs `in`/enumeration so feature detection ('object-fit'
in el.style) works for unset properties.

Closes #356.
2026-07-07 09:26:23 +02:00
SGavrl 08c1f0d721 fix(cdp): fire onChange for React/Vue controlled inputs (#353)
Typing over CDP into a React controlled input never fired onChange, so
the value never reached component state (issue #324). Two root causes,
both needed:

1. Value tracker. React/Preact/Vue install a value tracker by redefining
   `value` on the element instance to record the last value they wrote.
   obscura's CDP typing assigned `el.value = ...` directly, which runs
   that wrapper and keeps the tracker in sync, so React saw no change on
   the input event. Synthetic value/backspace writes now go through a new
   __obscura_setFieldValue helper that writes via the element's prototype
   setter, bypassing the tracker. The tracker stays stale, so the input
   event reads as a real edit.

2. Event handler feature detection. React decides whether to use the
   modern `input` event via `("oninput" in document)`. obscura exposed the
   GlobalEventHandlers on* attributes only on window, so that check failed
   and React fell back to a legacy change-detection path that ignores
   input events. The on* attributes are now also present on Document and
   Element, matching the HTML spec (GlobalEventHandlers is mixed into
   Window, Document, and HTMLElement).

Verified end to end over CDP against the real React 18 UMD build: typing
into controlled inputs updates state, the component re-renders, and a
submit handler reads the captured values.

Closes #324.
2026-07-03 14:10:03 +02:00
SGavrl a5a8de7c8c fix(js): make new Image() a real HTMLImageElement so .style works
new Image() used a bare class shim with no .style, so new Image().style
returned undefined and libraries touching style on a detached image threw
(issue #350). Build the image from document.createElement('img') instead,
which gives a working CSSStyleDeclaration, attribute reflection, and event
dispatch. Assigning .src still emulates a successful decode: it flips
complete and fires load (both the onload property and load listeners) so
lazy loaders and preloaders do not hang.
2026-07-03 13:24:20 +02:00
MrBeanDev ed75730e2d feat(js): implement WebCrypto SubtleCrypto secret-key algorithms
crypto.subtle only implemented digest. sign returned a fixed 32-byte
buffer, verify always returned true, encrypt and decrypt threw, and
generateKey, importKey and deriveBits handed back placeholder data.
Pages that use WebCrypto (anti-bot probes, PKCE and other auth flows,
client-side crypto) silently got wrong results.

Implement the secret-key half of WebCrypto on RustCrypto, following the
existing op_subtle_digest pattern:

- HMAC sign and verify (SHA-1/256/384/512)
- AES-GCM, AES-CBC and AES-CTR encrypt and decrypt
- PBKDF2 and HKDF deriveBits and deriveKey
- raw and symmetric JWK (oct) importKey and exportKey, generateKey,
  wrapKey and unwrapKey
- getRandomValues and randomUUID now draw from the OS CSPRNG instead of
  Math.random, so they are uniform across typed-array widths and no
  longer a fingerprinting tell

Unsupported algorithms (RSA, ECDSA, ECDH) and key formats (pkcs8, spki)
throw NotSupportedError instead of returning fake data. The crypto runs
in stateless Rust ops; the bootstrap.js shim only marshals bytes and
normalizes algorithm parameters, so the native fast paths are untouched
and there is no cost unless a page calls crypto.

Verified against RFC 4231 (HMAC), the NIST GCM vectors, NIST SP800-38A
(AES-CTR), RFC 6070 (PBKDF2) and RFC 5869 (HKDF), plus roundtrip and
tag/padding tamper-rejection checks.
2026-07-01 12:11:55 +05:30
SGavrl 25ce541ec8 fix(js): make NodeList a real type and childNodes a NodeList
childNodes returned a plain Array (constructor "Array"), and NodeList
extended Array, so Array.isArray(list) was true and
Object.prototype.toString reported "[object Array]". A real browser
reports "[object NodeList]" and Array.isArray is false, which
feature-detection and fingerprinting scripts check.

NodeList is now its own type with the array-like surface scripts use
(indexed access, length, item, forEach, entries/keys/values, iteration),
and childNodes goes through it. Verified spread, for..of, and Array.from
still work.

Refs #335.
2026-06-30 15:18:21 +00:00
SGavrl a16e8d4fe3 fix(js): default checkbox and radio value to "on"
A checkbox or radio input with no value attribute returns "on" in a real
browser; obscura returned the empty string. Page scripts that read
input.value to build form submissions (and browser-environment checks)
saw the wrong default. An explicit value attribute still wins.

Refs #335.
2026-06-30 15:18:21 +00:00
SGavrl 50ef1eecbd fix: SVGAnimatedString, :has() selectors, and cookie persistence on serve 2026-06-29 11:00:22 +00:00
SGavrl 117415848e js: implement SVGAnimatedString for SVG className and href (#330)
SVG className and href reflected as plain strings, so el.className.animVal
and el.href.animVal were undefined, which made new URL(svgEl.href.animVal)
throw and broke any SVG-aware script. Add an SVGAnimatedString class
(baseVal/animVal both read the live attribute; baseVal writable) and make
the className and href getters return it for SVG-namespace elements. href
falls back to the legacy xlink:href. HTML elements are unchanged.
2026-06-29 10:48:20 +00:00
RezoandSGavrl 260c4c01fa Add fetch body serialization support for Blob, ArrayBuffer, and TypedArray (#329)
* Add fetch body serialization support for Blob, ArrayBuffer, and TypedArray

* js: fix Blob fetch body serialization

The Blob branch passed the Blob to _bodyToUint8Array, which did not know
about obscura's Blob (its data lives in _bytes) and fell through to
encoding String(blob), so a Blob body was sent as "[object Blob]".
Teach _bodyToUint8Array about _bytes, and build the binary string with
the loop-based _bytesToBinaryString instead of String.fromCharCode.apply
so large blobs do not overflow the argument limit.

---------

Co-authored-by: SGavrl <[email protected]>
2026-06-26 20:43:19 +00:00
SGavrl bd39512ffb api: re-validate intercept URL rewrites against the SSRF gate
A Continue resolution that rewrites the request URL applied the new URL
after validate_fetch_url had already run on the original, so a rewrite
could reach a private or internal address the original request was
blocked from. Run the overridden URL back through validate_fetch_url,
matching how redirects are re-validated, and return a blocked response
if it fails.

Also fix a doc comment that ended up attached to the new fetch_response
helper instead of stealth_fetch_all.
2026-06-24 23:06:54 +00:00
SGavrl c19a21c142 api: honor interception Continue overrides and fix pre-navigation enable
op_fetch_url ignored the url/method/headers/body fields on
InterceptResolution::Continue, so the interception channel could pass, mock, or
block a request but not rewrite it. Apply them: shadow url/method/body and
derive custom_headers from the override after the channel resolves Continue.

Also fix enable_interception() called before the first navigation: init_js
propagated intercept_tx to the new runtime but not intercept_enabled, so the
runtime started with interception disabled and op_fetch_url never intercepted.
Propagate intercept_enabled in init_js as well.

Add a test that rewrites a fetch's url via Continue and asserts the rewritten
response is captured.
2026-06-24 21:51:21 +00:00
SGavrl 1f5066fc30 api: add request/response interception API to Page (#306)
Expose the interception infrastructure that already existed internally but was
private, so embedders can intercept, observe, mock, or block JS fetch()/XHR
without forking or unsafe casts.

On obscura::Page:
- add_preload_script: inject JS before any page <script> (CDP
  Page.addScriptToEvaluateOnNewDocument). The field already ran the scripts;
  this only adds the method.
- enable_interception: returns a receiver for every JS fetch()/XHR request;
  resolve each through its resolver with InterceptResolution::{Continue,
  Fulfill, Fail} to pass, mock, or block (CDP Fetch.enable equivalent).
- on_request / on_response: passive callbacks. on_response is the main path for
  capturing API response payloads from SPAs.

on_request/on_response previously fired only for navigation; op_fetch_url now
invokes them for JS fetch()/XHR too (both reqwest and stealth paths), gated on
a non-empty callback vector so the common no-callback case pays nothing.

Re-export InterceptedRequest, InterceptResolution, RequestInfo, ResourceType,
RequestCallback, ResponseCallback, Response from the obscura crate.

Integration test tests/interception.rs exercises all three against a local echo
server.
2026-06-24 21:36:01 +00:00
SGavrl 8582fcceca Merge stealth-overhaul into main
Resolved conflicts:
- page.rs: keep both set_blocked_urls (#315) and stealth client wiring
- ops.rs: keep SHA-512/224 + SHA-512/256 from stealth branch; keep
  the unreachable catch-all (vec![]) from #314 instead of silent
  SHA-256 fallback; updated comment accordingly
- bootstrap.js: extend the digest algorithm allowlist to include
  SHA-512/224 and SHA-512/256 to match the Rust additions
2026-06-23 18:05:55 +00:00
SGavrl db24d7a18c js: prompt for sensitive permissions in navigator.permissions.query
permissions.query returned "granted" for every permission except notifications.
Real Chrome defaults camera, microphone, geolocation, and midi to "prompt", so
answering "granted" for them is a bot tell. Return "prompt" for those (keeping
notifications in sync with Notification.permission) and "granted" for the rest.
2026-06-23 17:35:37 +00:00
SGavrl 3eb28da5c6 js: serialize FormData bodies to multipart/form-data in fetch
fetch() and XHR did String(body), so passing a FormData as the request body
sent the literal "[object Object]" and lost the multipart payload along with
its boundary. Servers expecting multipart/form-data replied 400 "Invalid
boundary for multipart/form-data", which is what blocked booking.com: the AWS
WAF challenge POSTs its proof-of-work solution to /mp_verify as FormData, so
the verify always failed and no aws-waf-token was issued. Serialize FormData to
a multipart/form-data body with a generated boundary and set the matching
Content-Type, the way a real browser does; URLSearchParams bodies get
application/x-www-form-urlencoded. XHR.send routes through fetch() so it is
covered too. With this, obscura --stealth renders the real booking.com landing
page the same as headless Chrome.
2026-06-23 10:52:57 +00:00
SGavrl 402de261aa js: fix Blob-URL Worker race and update location synchronously on navigation
URL.createObjectURL stored blob content asynchronously (blob.text().then) but
the Worker constructor read __blobStore synchronously, so new Worker(blobURL)
in the same tick raced the store, fell through to fetch(blobURL), and failed
with net::ERR_FAILED. This broke Blob-URL workers including the AWS WAF
challenge proof-of-work worker. Store synchronously from Blob._bytes, which the
obscura Blob materializes in its constructor.

location.reload was a no-op, so challenges that reload after setting a token
cookie never re-fetched. It now navigates like assign/replace.

location.href/assign/replace/reload cleared __virtualUrl immediately, so a
script that set location.href and re-read it saw the old URL until the async
navigation completed; real browsers update location.href synchronously. Preview
the resolved target in __virtualUrl and clear it in __obscura_init once the
navigation completes, so immediate reads match and redirects still win.
2026-06-23 10:28:42 +00:00
SGavrl 2e0981ef3f stealth: route scripted fetch()/XHR through the wreq client
op_fetch_url backed JS fetch() and XHR with a plain reqwest/rustls client, so
in stealth mode scripted requests carried a non-browser TLS fingerprint and
none of the Chrome client hints the main navigation sends. Bot managers that
score subresource requests (the AWS WAF challenge verify call, Akamai sensors)
read that as a script and reject it. Add StealthHttpClient::send_single and a
stealth_fetch_all path that op_fetch_url uses when a stealth client is
installed, preserving the redirect, SSRF, and CORS handling so JS-level
requests share the navigation's Chrome TLS fingerprint and sec-ch-ua client
hints. The reqwest path is unchanged for non-stealth. Adds a stealth feature to
obscura-js gated on obscura-net/stealth.
2026-06-23 10:28:33 +00:00
SGavrl 4309935abd stealth: keep the Chrome 145 identity consistent across TLS, JS, and WebGL
Stealth mode forced a Linux navigator UA and platform but reported Windows
in userAgentData.getHighEntropyValues and served Windows Direct3D11 WebGL
renderers, a combination no real Linux Chrome ever produces and that
cross-layer fingerprint checks flag. Present one consistent Windows Chrome
145 identity instead: the wreq emulation uses Platform::Windows, and the
navigator surfaces read the UA and platform from shared constants so the
HTTP User-Agent header, sec-ch-ua-platform, navigator.userAgent, and
userAgentData all agree. userAgentData brands are now derived from the
Chrome major version via Chromium's GREASE algorithm rather than a single
hardcoded token, and a Linux WebGL renderer pool is added so a Linux UA can
never report a Direct3D11 renderer.
2026-06-22 22:19:16 +00:00
SGavrl dc780d77e4 js: handle SHA-512/224 and SHA-512/256 in crypto.subtle.digest
The digest op fell through to SHA-256 for any unrecognized algorithm name,
so SHA-512/224 and SHA-512/256 silently returned a SHA-256 hash. Add the
two FIPS 180-4 truncated variants, which the AWS WAF challenge proof-of-work
(used by booking.com and other CloudFront/WAF sites) requires.

Fixes #314.
2026-06-22 22:19:02 +00:00
Gabriel edde67ddd4 crypto.subtle.digest: reject unsupported algorithms (#314) (#319)
Unknown algorithm names (SHA-512/224, MD5, etc.) were silently falling
back to SHA-256, producing wrong bytes with no error. Now throws
DOMException('NotSupportedError') to match Chrome behavior.

Supported: SHA-1, SHA-256, SHA-384, SHA-512.
2026-06-22 13:02:05 +00:00
Gabriel a8358cc82d add Plugin, MimeType, MimeTypeArray globals (#305) (#318)
These three interfaces were missing from the global scope entirely.
Site bundles that reference Plugin as a constructor threw
"ReferenceError: Plugin is not defined".

Added as plain function declarations (same pattern as PluginArray) so
they survive the V8 snapshot. navigator.plugins now returns real Plugin
instances and navigator.mimeTypes returns a real MimeTypeArray with
MimeType instances, so instanceof checks work correctly.
2026-06-22 13:01:29 +00:00
RohitMadhu 3bf0da2fae Add Network.setBlockedURLs support (#315)
Added Network.setBlockedURLs dispatch handling.
Stores blocked URL patterns on each page.
Applies blocking to page subresources and JS fetch / XHR requests.
Supports wildcard CDP-style patterns like *://*.gstatic.com/*.woff2.
2026-06-22 11:02:49 +00:00
Rohit Madhu 5d53dfef03 Add XPath compatibility for Maps scraping 2026-06-17 23:11:05 -04:00
SGavrl ab27e9153d cli: list fetch()/XHR resources in --dump assets
--dump assets only walked static DOM attributes (script[src], img[src], ...),
so resources a page pulled in with fetch() or XHR never showed up (issue #301).

Record each absolute URL requested through op_fetch_url in ObscuraState,
expose it via the runtime and Page, and append those URLs to --dump assets as
type "fetch", deduped against URLs already listed from the DOM.
2026-06-17 16:36:08 +00:00
Gabriel d526a4da7d js: make event.isTrusted false for page-constructed events
isTrusted was hardcoded to true for every Event, so new Event('x').isTrusted returned true. Real browsers return false for script-constructed events and true only for user-agent dispatched ones, so this was a trivial bot-detection tell (Closes #303).

Track trusted events in a closure-private WeakSet that page JS cannot read or forge; default isTrusted to false. The CDP input pipeline marks its synthesized mouse/key events trusted via a non-enumerable __obscura_markTrusted helper, so automation clicks stay trusted like real Chrome over CDP while page-built events report false.
2026-06-17 15:15:25 +00:00
SGavrl 6264e6bfad js: make event.isTrusted false for page-constructed events
isTrusted was hardcoded to true for every Event, so new Event('x').isTrusted
returned true. Real browsers return false for script-constructed events and
true only for user-agent dispatched ones, so this was a trivial bot-detection
tell (issue #303).

Track trusted events in a closure-private WeakSet that page JS cannot read or
forge; default isTrusted to false. The CDP input pipeline marks its synthesized
mouse/key events trusted via a non-enumerable __obscura_markTrusted helper, so
automation clicks stay trusted like real Chrome over CDP while page-built
events report false.
2026-06-17 15:12:04 +00:00
Gabriel 1a3888c056 js: mark Event constructor as native for stealth compatibility
Mark the Event constructor as native so that Event.toString() returns "[native code]" instead of leaking the JavaScript implementation. This closes a stealth fingerprinting vector.
2026-06-17 14:54:15 +00:00
yzyf1312 37b33c1109 js: mark Event constructor as native for stealth compatibility 2026-06-14 22:38:04 +08:00
Rohit Madhu 570ca1544c add Network.getResponseBody support 2026-06-13 20:26:26 -04:00
SGavrl 09fe30ec39 js: implement Element.getAttributeNames()
The method was missing, so calling it threw TypeError: ... is not a
function, which broke Vue 3/Nuxt client hydration (issue #297). Back it with
the existing attribute_names op, the same source as the attributes getter, so
the order matches attributes and outerHTML.
2026-06-13 15:05:29 +00:00
SGavrl cd889d5659 Fix/stealth consistency
hasIframeProxy fix contentWindow now returns null when parentNode === null (iframe never inserted into any tree). The previous attempt used isConnected, which broke getBehemothIframe by returning null for iframes inside the phantom iframe's detached document. That caused PHANTOM_DARKNESS to fall back to self (main window), making all DOM measurement ops run on the main document instead of the isolated iframe body, which corrupted the fingerprint element lookup.

Worker auto-execution Worker constructor now resolves relative URLs against location.href, fetches the script, and auto-runs it. The worker scope exposes WorkerGlobalScope so IS_WORKER_SCOPE resolves correctly in creepjs. All three worker types now complete within the 3s timeout instead of silently timing out.

AudioBuffer + AudioContext AudioBuffer class added to globalThis with getChannelData, copyFromChannel, copyToChannel. AudioContext.createBuffer returns a real instance. AudioContext.addEventListener was a no-op; startRendering now dispatches to stored listeners so the complete event fires.

Crypto Crypto class added to globalThis with randomUUID and getRandomValues on its prototype. Fixes ReferenceError in getPlatformEstimate.

NetworkInformation + ContentIndex NetworkInformation class with downlinkMax on the prototype clears noDownlinkMax. ContentIndex stub on globalThis clears noContentIndex. navigator.connection is now a real NetworkInformation instance.

getElementById / querySelector stale index fix id_index only registers nodes at creation time. When replaceChild(fragment, el) swapped in new nodes their IDs hit .or_insert() and were skipped, leaving the index pointing at the detached original. getElementById now verifies the indexed node is in the live document and falls back to a full attribute-selector scan. query_selector_from falls through to the full scan on both index miss and stale entry instead of returning None early.

matchMedia Handles common queries instead of always returning false. prefers-color-scheme: dark returns true (headless Chrome defaults to light mode, so returning true for light is a bot signal). any-pointer: fine, hover: hover, prefers-reduced-motion: no-preference, min/max-width, and (color) return sensible values.

Fingerprint consistency jsHeapSizeLimit raised to 4294705152 (4 GB, matching 64-bit Chrome). getHighEntropyValues platform corrected from "Linux" to "Windows", platformVersion from "6.1.0" to "10.0.0" to match the Windows NT 10.0 UA string. Canvas: real PNG encoder, arc fill, multiply blend mode. Screen class, performance.now, chrome runtime errors fixed.

Build note deviceMemory and hardwareConcurrency use the high-value pools ([4, 8] GB and [4, 6, 8, 12, 16] cores) only when built with --features stealth.
2026-06-11 22:27:44 +00:00
SGavrl 4f722dcfd3 stealth: fix fingerprint consistency, matchMedia, jsHeapSizeLimit
jsHeapSizeLimit raised to 4294705152 (~4 GB) to match 64-bit Chrome.
Was 2172649472 (~2 GB).

getHighEntropyValues platform corrected to "Windows" and platformVersion
to "10.0.0" to match the Windows NT 10.0 user agent string. Was "Linux"
/ "6.1.0", which mismatched the UA and could be detected.

matchMedia now handles common queries rather than always returning false:
prefers-color-scheme:dark returns true (avoiding prefersLightColor headless
flag — headless Chrome defaults to light so true is a bot signal),
any-pointer:fine, hover:hover, prefers-reduced-motion:no-preference,
min/max-width, and (color) return sensible values.

Result: 0% like-headless, 0% headless, 0% stealth, 33/33 obstacle course.
Note: build must include --features stealth for deviceMemory and
hardwareConcurrency to use the high-value pools ([4,8] and [4,6,8,12,16]).
2026-06-11 22:14:11 +00:00
SGavrl 4e8960e323 stealth: add NetworkInformation and ContentIndex shims
NetworkInformation class with downlinkMax on its prototype clears
noDownlinkMax from the like-headless flag list. creepjs checks
'downlinkMax' in (window.NetworkInformation?.prototype || {}).

ContentIndex stub on globalThis clears noContentIndex. creepjs checks
'ContentIndex' in window.

navigator.connection is now a real NetworkInformation instance instead
of a plain object literal.

Result: 0% like-headless, 0% headless, 0% stealth on creepjs. 33/33
obstacle course unchanged.
2026-06-11 22:01:41 +00:00