* feat(stealth): randomize browser profile per session
Add a pool of 8 realistic browser profiles (Win10/Win11/macOS, Chrome
143-146). One is picked randomly per BrowserContext instead of always
advertising Linux, which is a strong bot signal.
Each profile keeps navigator.platform, userAgentData.platform, and
userAgentData.platformVersion internally consistent with the UA string.
- profiles.rs: static PROFILES pool + random_profile() picker
- context.rs: pick profile on construction, store platform fields
- page.rs: call rt.set_platform() alongside rt.set_user_agent()
- runtime.rs: add set_platform() to inject __obscura_platform globals
- bootstrap.js: read platform/userAgentData from globals (default Win32)
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
* feat(stealth): match GPU renderer pool to OS profile
Windows profiles get ANGLE Direct3D11 renderers; macOS profiles get
ANGLE Metal renderers. Vendor strings updated to match (Google Inc. (Apple)
for Apple Silicon/Intel Mac, Google Inc. (NVIDIA/AMD/Intel) for Windows).
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
* fix(stealth): navigator.webdriver returns false instead of undefined
Browsers set navigator.webdriver to false when automation is not active.
Returning undefined is a detectable bot signal — fingerprinting scripts
check typeof navigator.webdriver === 'boolean' as part of headless detection.
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
* Make stealth profile stable by default, rotation opt-in
Picking a random profile per BrowserContext meant one address would cycle
through different browser identities, which is itself a bot signal, and the
rotated profile carries no matching TLS or timezone fingerprint. Default to a
single stable profile; opt into rotation with OBSCURA_ROTATE_PROFILE, or pin
one with OBSCURA_PROFILE=<index>.
Drive timezone from the process TZ so Intl.DateTimeFormat and Date report one
consistent zone. Default Europe/Berlin; override with OBSCURA_TIMEZONE.
Update navigator.webdriver test to expect false, matching real Chrome.
---------
Co-authored-by: SGavrl <[email protected]>
CDP / navigation:
- Route all Page.navigate through process_with_interception so the
parent dispatcher stays responsive during long navs (fixes
Target.createTarget timeouts after a handful of newPage calls).
- Default waitUntil to domcontentloaded. Puppeteer and Playwright
don't pass it server-side and wait client-side via lifecycleEvent,
so the old Load default blocked nearly every nav on slow sites.
- 30s navigation hard ceiling, OBSCURA_NAV_TIMEOUT_MS to override.
- execute_scripts loop now checks wall-clock deadline per iteration.
- Emit Network.requestWillBeSent before Page.frameNavigated so
Playwright's FrameManager attaches the navigation response (#189).
- Use loaderId as the Document request id so page.goto() resolves
to a Response instead of null (#189).
- Fresh executionContextId per navigation, no more 'Cannot find
context with specified id' after the first nav (#192).
- POST navigation params plumbed through the intercept path.
DOM / Web API surface:
- MutationObserver: subtree filter walks parent chain, fires on
textContent / CharacterData.data / innerHTML, type filter respected,
oldValue passed.
- IntersectionObserver: multi-fire (initial + 120/500/1500ms),
disconnect / unobserve / takeRecords, root / rootMargin / thresholds.
- ResizeObserver: per-target firing from getBoundingClientRect, drop
the no-op override that was clobbering the real class.
- crypto.subtle.digest: real SHA-1/256/384/512 via op_subtle_digest.
- DOMParser / document.implementation.createHTMLDocument /
createDocument: detached doc with body, head, documentElement,
querySelector*, getElementById, getElementsByTagName.
- WebSocket / EventSource / BroadcastChannel: real EventTarget,
open fires on microtask.
- indexedDB: request, object store, transaction, database with
async onsuccess and in-memory Map storage.
- getComputedStyle: width/height/top/left from bbox so React
virtualization libs render items.
Site compatibility went from 3/40 to 175/175 reachable sites in
the sweep. Perf is equal or faster than pristine main on every
site tested (DCL default removes the block on long-tail subresources).
28/28 integration tests pass.
Surfaces the existing LP.getMarkdown CDP machinery as a CLI flag so
text-pipeline users (LLM agents, scrapers feeding markdown to other
tools) don't need a separate HTML-to-markdown converter step.
obscura fetch --dump markdown 'https://example.com'
Layout:
- crates/obscura-js/src/markdown.rs holds HTML_TO_MARKDOWN_JS as a
pub const so every layer can share the same DOM-walker. obscura-js
is at the bottom of the dep DAG (obscura-browser -> obscura-js,
obscura-cdp -> obscura-js, obscura-cli -> obscura-js) so a single
source of truth works without a circular dependency.
- crates/obscura-browser/src/lib.rs re-exports it for ergonomics.
- crates/obscura-cdp/src/domains/lp.rs drops its inline duplicate
and calls the shared const.
- crates/obscura-cli/src/main.rs adds the `Markdown` DumpFormat
variant + dump_markdown(&mut Page) which page.evaluate's the
shared script and prints the result.
Net diff: a new 75-line markdown.rs, a 70+ line deletion in lp.rs
that's reborn as 3 lines, and ~10 lines added across the other four
files. No behavior change for `--dump html|text|links`, and
LP.getMarkdown over CDP keeps the exact same JSON shape.
Four new unit tests in obscura-js cover headings, inline emphasis +
links, ordered/unordered lists, and that <script> / <style> contents
do not leak into the output.
Generated by Ora Studio
Vibe coded by ousamabenyounes
Co-authored-by: Ora Studio <[email protected]>
