Repo guidance for AI coding agents and contributors, covering the things you
can't infer from the code: build (V8 from source, cmake for stealth, per-crate
scoping), why tests run under cargo nextest and not cargo test (one V8 isolate
per process), the obstacle course as the behavioral gate, the DOM/runtime
gotchas, the hang/panic-safety invariants, and a pre-finish validation
checklist.
Pages and scrapers that call document.evaluate() hit "TypeError: document.evaluate is not a function", which can break script execution and hydration. Add document.evaluate() plus the XPathResult global with the standard result-type constants so those paths work (needed for Maps scraping).
The evaluator handles the common relative forms: //tag and .//tag with attribute predicates (@attr, @attr='v', contains(@attr,'v'), starts-with(@attr,'v')), backed by querySelectorAll and attribute filtering. Absolute paths, positional predicates ([n]), text() and axes are not supported and return an empty result.
--dump assets only walked static DOM attributes (script[src], img[src], ...), so resources a page pulled in with fetch() or XHR never showed up (Closes#301).
Record each absolute URL requested through op_fetch_url in ObscuraState, expose it via the runtime and Page, and append those URLs to --dump assets as type "fetch", deduped against URLs already listed from the DOM.
--dump assets only walked static DOM attributes (script[src], img[src], ...),
so resources a page pulled in with fetch() or XHR never showed up (issue #301).
Record each absolute URL requested through op_fetch_url in ObscuraState,
expose it via the runtime and Page, and append those URLs to --dump assets as
type "fetch", deduped against URLs already listed from the DOM.
isTrusted was hardcoded to true for every Event, so new Event('x').isTrusted returned true. Real browsers return false for script-constructed events and true only for user-agent dispatched ones, so this was a trivial bot-detection tell (Closes#303).
Track trusted events in a closure-private WeakSet that page JS cannot read or forge; default isTrusted to false. The CDP input pipeline marks its synthesized mouse/key events trusted via a non-enumerable __obscura_markTrusted helper, so automation clicks stay trusted like real Chrome over CDP while page-built events report false.
isTrusted was hardcoded to true for every Event, so new Event('x').isTrusted
returned true. Real browsers return false for script-constructed events and
true only for user-agent dispatched ones, so this was a trivial bot-detection
tell (issue #303).
Track trusted events in a closure-private WeakSet that page JS cannot read or
forge; default isTrusted to false. The CDP input pipeline marks its synthesized
mouse/key events trusted via a non-enumerable __obscura_markTrusted helper, so
automation clicks stay trusted like real Chrome over CDP while page-built
events report false.
Mark the Event constructor as native so that Event.toString() returns "[native code]" instead of leaking the JavaScript implementation. This closes a stealth fingerprinting vector.
The method was missing, so calling it threw TypeError: ... is not a function, which broke Vue 3/Nuxt client hydration (Closes#297). Back it with the existing attribute_names op, the same source as the attributes getter, so the order matches attributes and outerHTML.
The method was missing, so calling it threw TypeError: ... is not a
function, which broke Vue 3/Nuxt client hydration (issue #297). Back it with
the existing attribute_names op, the same source as the attributes getter, so
the order matches attributes and outerHTML.
cargo install --features stealth failed on Windows with E0119 in cookie 0.18.1: its blanket From<T: Into<Option>> impl fails the coherence check on some toolchains (issue #295). obscura never uses reqwest's cookie store; it manages cookies through its own CookieJar and sets the Cookie/Set-Cookie headers directly. The cookies feature only pulled in the cookie/cookie_store crates (and the whole time subtree).
Drop it. Removes 12 packages including cookie, cookie_store, and time, so the failing crate is no longer compiled in any build. No behavior change: the cookie jar unit tests and the obstacle-course cookies stage still pass.
cargo install --features stealth failed on Windows with E0119 in cookie
0.18.1: its blanket From<T: Into<Option<OffsetDateTime>>> impl fails the
coherence check on some toolchains (issue #295). obscura never uses reqwest's
cookie store; it manages cookies through its own CookieJar and sets the
Cookie/Set-Cookie headers directly. The cookies feature only pulled in the
cookie/cookie_store crates (and the whole time subtree).
Drop it. Removes 12 packages including cookie, cookie_store, and time, so the
failing crate is no longer compiled in any build. No behavior change: the
cookie jar unit tests and the obstacle-course cookies stage still pass.
The x86_64-apple-darwin release crashed at V8 isolate init with SIGSEGV in ReadOnly snapshot deserialization (issue #290). obscura-js bakes a V8 startup snapshot at build time via a build script, which runs on the host arch. The job built x86_64 on macos-latest (Apple Silicon), so an arm64 snapshot got embedded into the Intel binary and V8 crashed deserializing it. Build on macos-15-intel so the snapshot matches the target.
Drop the aarch64-linux-cross artifact: it has the same defect (x86_64-host snapshot in an arm64 binary) and the native ARM runner already covers it.
Add a post-build smoke test that runs the V8 path on a data: URL so a release can never ship a binary that crashes at isolate init again.
Closes#290
The x86_64-apple-darwin release crashed at V8 isolate init with SIGSEGV in
ReadOnly snapshot deserialization (issue #290). obscura-js bakes a V8 startup
snapshot at build time via a build script, which runs on the host arch. The
job built x86_64 on macos-latest (Apple Silicon), so an arm64 snapshot got
embedded into the Intel binary and V8 crashed deserializing it. Build on
macos-15-intel so the snapshot matches the target.
Drop the aarch64-linux-cross artifact: it has the same defect (x86_64-host
snapshot in an arm64 binary) and the native ARM runner already covers it.
Add a post-build smoke test that runs the V8 path on a data: URL so a release
can never ship a binary that crashes at isolate init again.
hasIframeProxy fix contentWindow now returns null when parentNode === null (iframe never inserted into any tree). The previous attempt used isConnected, which broke getBehemothIframe by returning null for iframes inside the phantom iframe's detached document. That caused PHANTOM_DARKNESS to fall back to self (main window), making all DOM measurement ops run on the main document instead of the isolated iframe body, which corrupted the fingerprint element lookup.
Worker auto-execution Worker constructor now resolves relative URLs against location.href, fetches the script, and auto-runs it. The worker scope exposes WorkerGlobalScope so IS_WORKER_SCOPE resolves correctly in creepjs. All three worker types now complete within the 3s timeout instead of silently timing out.
AudioBuffer + AudioContext AudioBuffer class added to globalThis with getChannelData, copyFromChannel, copyToChannel. AudioContext.createBuffer returns a real instance. AudioContext.addEventListener was a no-op; startRendering now dispatches to stored listeners so the complete event fires.
Crypto Crypto class added to globalThis with randomUUID and getRandomValues on its prototype. Fixes ReferenceError in getPlatformEstimate.
NetworkInformation + ContentIndex NetworkInformation class with downlinkMax on the prototype clears noDownlinkMax. ContentIndex stub on globalThis clears noContentIndex. navigator.connection is now a real NetworkInformation instance.
getElementById / querySelector stale index fix id_index only registers nodes at creation time. When replaceChild(fragment, el) swapped in new nodes their IDs hit .or_insert() and were skipped, leaving the index pointing at the detached original. getElementById now verifies the indexed node is in the live document and falls back to a full attribute-selector scan. query_selector_from falls through to the full scan on both index miss and stale entry instead of returning None early.
matchMedia Handles common queries instead of always returning false. prefers-color-scheme: dark returns true (headless Chrome defaults to light mode, so returning true for light is a bot signal). any-pointer: fine, hover: hover, prefers-reduced-motion: no-preference, min/max-width, and (color) return sensible values.
Fingerprint consistency jsHeapSizeLimit raised to 4294705152 (4 GB, matching 64-bit Chrome). getHighEntropyValues platform corrected from "Linux" to "Windows", platformVersion from "6.1.0" to "10.0.0" to match the Windows NT 10.0 UA string. Canvas: real PNG encoder, arc fill, multiply blend mode. Screen class, performance.now, chrome runtime errors fixed.
Build note deviceMemory and hardwareConcurrency use the high-value pools ([4, 8] GB and [4, 6, 8, 12, 16] cores) only when built with --features stealth.
jsHeapSizeLimit raised to 4294705152 (~4 GB) to match 64-bit Chrome.
Was 2172649472 (~2 GB).
getHighEntropyValues platform corrected to "Windows" and platformVersion
to "10.0.0" to match the Windows NT 10.0 user agent string. Was "Linux"
/ "6.1.0", which mismatched the UA and could be detected.
matchMedia now handles common queries rather than always returning false:
prefers-color-scheme:dark returns true (avoiding prefersLightColor headless
flag — headless Chrome defaults to light so true is a bot signal),
any-pointer:fine, hover:hover, prefers-reduced-motion:no-preference,
min/max-width, and (color) return sensible values.
Result: 0% like-headless, 0% headless, 0% stealth, 33/33 obstacle course.
Note: build must include --features stealth for deviceMemory and
hardwareConcurrency to use the high-value pools ([4,8] and [4,6,8,12,16]).
NetworkInformation class with downlinkMax on its prototype clears
noDownlinkMax from the like-headless flag list. creepjs checks
'downlinkMax' in (window.NetworkInformation?.prototype || {}).
ContentIndex stub on globalThis clears noContentIndex. creepjs checks
'ContentIndex' in window.
navigator.connection is now a real NetworkInformation instance instead
of a plain object literal.
Result: 0% like-headless, 0% headless, 0% stealth on creepjs. 33/33
obstacle course unchanged.
Worker auto-execution: Worker constructor resolves relative URLs against
location.href, fetches the script, and auto-runs it. WorkerGlobalScope is
exposed so IS_WORKER_SCOPE resolves correctly. Worker data (navigator,
locale, timezone) now arrives via postMessage within the 3s timeout.
AudioBuffer class added to globalThis with getChannelData, copyFromChannel,
copyToChannel. AudioContext.createBuffer returns a real AudioBuffer. Fixes
ReferenceError that blocked the audio fingerprint section.
Crypto class added to globalThis with randomUUID and getRandomValues on its
prototype. Fixes ReferenceError in getPlatformEstimate.
AudioContext.addEventListener was a no-op; startRendering only fired
oncomplete. Add _listeners map and dispatch to stored listeners.
getElementById and querySelector('#id') returned null for elements inserted
via replaceChild(documentFragment, el). id_index only registers nodes at
creation time; .or_insert skipped IDs on cloned fragments; None caused
query_selector_from to short-circuit the full scan.
Fix querySelector: fall through to full scan on index miss or stale entry.
Fix getElementById: verify node is in live document; fall back to attribute
selector scan if not.
contentWindow getter: return null when parentNode is null (iframe was never
inserted into any tree). Previously returning null for all disconnected
iframes broke getBehemothIframe, which caused PHANTOM_DARKNESS to fall back
to self (main window) instead of the phantom iframe. That made DOC point to
the main document, so rect and font measurements wrote fragment nodes into
the main body, and the subsequent full-DOM scan in getElementById could not
find fingerprint-data through nested fragments in time.
Limiting the null return to parentNode===null (never inserted) means
getBehemothIframe gets a valid nested contentWindow as expected. creepjs
now reports hasIframeProxy: false and 0% stealth detection.
DocumentFragment constructor: pass nid through when provided.
Range.getBoundingClientRect / getClientRects: return commonAncestor rect.
Upgraded wreq to 6.0.0-rc.29 and wreq-util to 3.0.0-rc.12 to apply the latest Chrome 145 TLS fingerprint (16 cipher suites, proper extension order, ALPS, GREASE).
Introduced __obscura_stealth global to isolate stealth-specific JS behaviors from the standard execution path.
Overrode navigator.userAgentData and getHighEntropyValues() in stealth mode to perfectly match the sec-ch-ua network headers.
Restricted hardwareConcurrency and deviceMemory pools in stealth mode to desktop-realistic values (e.g., minimum 4 cores, 4GB RAM) to eliminate mobile-tier leaks.
Fixed usedJSHeapSize occasionally exceeding totalJSHeapSize.
wreq upgraded to 6.0.0-rc.29 and wreq-util to 3.0.0-rc.12 to apply the
latest Chrome 145 TLS fingerprint (16 cipher suites, proper extension
order, ALPS, GREASE).
__obscura_stealth global variable introduced and passed from Rust to JS,
allowing stealth-specific JS behaviors to run only when the stealth flag
is enabled.
Tightened hardwareConcurrency and deviceMemory pools in stealth mode
to desktop-realistic values (e.g. min 4GB RAM, 4 cores), eliminating
mobile-tier values that leak the spoof.
navigator.userAgentData and getHighEntropyValues() are now overridden
in stealth mode to perfectly match the sec-ch-ua header sent by
wreq_util::Profile::Chrome145.
Fixed usedJSHeapSize sometimes exceeding totalJSHeapSize.
OfflineAudioContext.startRendering() now fires oncomplete asynchronously
(microtask) so callers that set ctx.oncomplete after calling startRendering
(fpCollect pattern) get the callback. The rendered buffer contains a
triangle wave scaled to produce the Chrome Linux audio fingerprint sum
(~124.04). createBuffer() now uses persistent Float32Array channels so
written sample values are read back correctly.
screen is now an instance of a Screen class with native getters on the
prototype. Object.getOwnPropertyDescriptor(Object.getPrototypeOf(screen),
"width").get.toString() now returns the expected native-code format instead
of throwing.
chrome.runtime.connect() and sendMessage() now throw the Chrome error
("Could not establish connection. Receiving end does not exist.") instead
of returning silently, matching what fpCollect records for detailChrome.
performance.now() now advances by 0.1ms per call within the same clock
tick, simulating ~100us timer resolution. Previously it aliased Date.now()
which returns the same integer for all calls in one tick, making
redPill2 = Infinity (serialized as null).
WebSocket constructor now throws SyntaxError for non-ws/wss URLs, matching
Chrome's behavior and populating errorsGenerated correctly.
toDataURL() now encodes the real pixel buffer as a valid PNG using
stored-block DEFLATE, CRC32, and Adler32. The 106-byte stub it replaced
was a static value that any canvas fingerprinter could trivially flag.
fill() rasterizes arc paths from this._path using a distance-check
circle fill, so arc-based canvas tests produce real pixel data.
_setPixel() handles globalCompositeOperation='multiply' so fpCollect's
colored-circle blend test composites correctly.
Canvas dimensions now read from getAttribute('width'/'height') so
explicitly sized canvases (e.g. 400x200) use their declared dimensions
instead of always defaulting to 300x150.
canPlayType was undefined on <audio> and <video> elements. Adds HTMLMediaElement
extending Element with canPlayType returning values that match Chrome 145 on Linux
(probably/maybe/empty per MIME type). HTMLAudioElement and HTMLVideoElement extend
HTMLMediaElement. _elementClassFor routes AUDIO and VIDEO tags to the correct class.
Navigator: adds a global Navigator constructor so typeof Navigator !== 'undefined'.
Links it into the navigator prototype chain (_wdProto = Object.create(Navigator.prototype))
so navigator instanceof Navigator returns true, matching Chrome.
These are two of the signals in server-side fingerprint classifiers (fpCollect et al).
PluginArray: real browsers expose a native PluginArray type. navigator.plugins
was returning a plain Array, so `navigator.plugins instanceof PluginArray` and
`navigator.plugins[0].toString()` (which should return [object Plugin]) were
failing sannysoft-style checks. Adds a PluginArray constructor whose prototype
inherits from Array.prototype, with Symbol.toStringTag='PluginArray'. Each
plugin entry now carries Symbol.toStringTag='Plugin' so Object.prototype.toString
returns [object Plugin].
webdriver: moved the webdriver getter off the navigator instance and onto a
thin prototype so Object.getOwnPropertyDescriptor(navigator, 'webdriver')
returns undefined, matching Chrome where the property lives on Navigator.prototype.
Getter is marked native. sannysoft "WebDriver (New)" now reports "missing (passed)".
navigator.platform returned "Win32" while navigator.userAgent said Linux when
using the stealth client. page.rs now calls set_platform("Linux x86_64", "Linux", "")
alongside set_user_agent when the stealth client is active.
bootstrap.js: pre-declare 71 internal _* globals as enumerable:false before
var runs, so they don't show up in Object.keys(window) on real pages.
deviceandbrowserinfo.com detects CDP by setting Error.prepareStackTrace as a spy, then calling console.log(new Error()). _consoleFn accessed a.stack, which fires prepareStackTrace if set. Fix: save/clear/restore around the .stack access.
NetworkInformation (navigator.connection) and ServiceWorkerContainer (navigator.serviceWorker) are EventTargets in real browsers. React 18 and mobile SPAs call addEventListener on them; the stubs lacked it, so the app bundle crashed with 'addEventListener is not a function' (e.g. goofish.com).
Reject request bodies above 16 MiB with 413 before allocating, stopping an
unauthenticated Content-Length OOM (vec![0u8; len]). Add an optional,
default-off OBSCURA_MCP_ALLOWED_ORIGINS allowlist; unset keeps the permissive
default so issue #175 hosted dashboards are unaffected.
SameSite is case-insensitive per RFC 6265bis but servers send it in any
casing (lax, LAX, Lax). Normalize to title-case at parse time so
downstream comparisons work regardless. Unknown values fall back to Lax
per spec.
Add is_forbidden_ip() (loopback, RFC1918, link-local incl. 169.254.169.254,
broadcast, documentation, 0.0.0.0/::, IPv6 unique-local fc00::/7, IPv4-mapped
IPv6) and SsrfGuardResolver (impl reqwest::dns::Resolve) that resolves and
rejects forbidden addresses at connect time, closing the DNS-rebinding bypass.
Wired into the navigation client and the JS fetch()/XHR path. The
--allow-private-network / OBSCURA_ALLOW_PRIVATE_NETWORK escape hatch is kept.
Co-authored-by: charlesmamane26-sketch <charlesmamane1826@gmail.com>
DDoS-Guard's view.js challenge calls Element.prototype.insertAdjacentText
and dies with 'TypeError: t.insertAdjacentText is not a function', blocking
any DDoS-Guard-protected site (e.g. pikabu.ru). insertAdjacentElement was
missing too. insertAdjacentHTML was already implemented, so this just lifts
the same insertion pattern for Text-node and Element insertion semantics.
Both methods mirror the spec: case-insensitive position keywords, four
insertion points (beforebegin/afterbegin/beforeend/afterend), and
insertAdjacentElement returns the inserted element (or null when
beforebegin/afterend has no parent). Registered with _markNative so
Function.prototype.toString keeps reporting [native code] — consistent
with the existing Element.prototype.insertAdjacentHTML stealth registration.
CDP clients (browser-use and similar) focus an input via DOM.focus before
typing; Input.dispatchKeyEvent then targets document.activeElement. Without a
DOM.focus handler the focus call errored, keystrokes landed on nothing, and the
field stayed empty. Resolve the node and call its JS focus(), which sets
document.activeElement. Completes the type path for DOM-agent frameworks.
CDP DOM-agent frameworks (browser-use and similar) could connect but saw a
blank page: navigation never refreshed their cached target url/title, and their
DOM build aborted with "Unknown domain: DOMSnapshot".
- Emit Target.targetInfoChanged after navigation (both CDP Page.navigate and
JS-initiated) carrying the post-nav url/title. Strict clients refresh a target
only on this event; it also fixes Puppeteer/Playwright page.url() tracking.
- Implement DOMSnapshot.captureSnapshot, built from the DOM with synthetic
geometry (no paint engine) and plausible computed styles, keyed by the same
backendNodeId as DOM.getDocument. Enough for tag/ARIA/accessibility element
detection; it runs only when a client requests it, so the fetch path is
unaffected. Coordinate clicking still requires a real layout engine.
Gabriel