The on_request/on_response Vec lived on the ObscuraHttpClient shared by
every page in the context, so a callback registered on one page fired for
all pages (#408 cross-page bleed) and outlived its page.
Callbacks now live in a per-page CallbackRegistry (Arc shared with the
page's JS runtime state). Page-driven fetch paths (navigation, POST,
scripts, stylesheets, robots.txt, module loads, JS fetch/XHR in both
plain and stealth modes) pass the registry through; the HTTP client
itself is callback-free.
Adds callbacks_do_not_bleed_across_pages: page A's callback stays silent
for page B's requests and stops firing after page A is dropped.
Booking.com's anti-bot script pre-defines a non-configurable own `src` on
<img> elements. The Image shim's load emulation then threw "TypeError:
Cannot redefine property: src" out of the constructor, crashing any page
that calls new Image() after that instrumentation runs.
Check the own descriptor first and skip the load emulation when `src` is
already non-configurable: a page that owns `src` is instrumenting loads
itself. The normal path (configurable src, emulated load event) is
unchanged and now covered by a test.
Fixes the crash half of #394.
on_request/on_response stored callbacks in append-only Vecs with no way
to remove one, so a crawler that installed a response-body capture hook
for one phase could never detach it: it fired for the shared client's
whole lifetime, re-registering across navigations piled up duplicates,
and (because the client is shared per context) it saw sibling pages'
requests too.
Tag each callback with a stable id. on_request/on_response now return
the id and off_request/off_response (remove_on_request/remove_on_response
on the client) detach a specific one. Storage is Vec<(u64, Callback)>;
all fire sites destructure the tuple. Closes#408.
op_fetch_url ignored the url/method/headers/body fields on
InterceptResolution::Continue, so the interception channel could pass, mock, or
block a request but not rewrite it. Apply them: shadow url/method/body and
derive custom_headers from the override after the channel resolves Continue.
Also fix enable_interception() called before the first navigation: init_js
propagated intercept_tx to the new runtime but not intercept_enabled, so the
runtime started with interception disabled and op_fetch_url never intercepted.
Propagate intercept_enabled in init_js as well.
Add a test that rewrites a fetch's url via Continue and asserts the rewritten
response is captured.
Expose the interception infrastructure that already existed internally but was
private, so embedders can intercept, observe, mock, or block JS fetch()/XHR
without forking or unsafe casts.
On obscura::Page:
- add_preload_script: inject JS before any page <script> (CDP
Page.addScriptToEvaluateOnNewDocument). The field already ran the scripts;
this only adds the method.
- enable_interception: returns a receiver for every JS fetch()/XHR request;
resolve each through its resolver with InterceptResolution::{Continue,
Fulfill, Fail} to pass, mock, or block (CDP Fetch.enable equivalent).
- on_request / on_response: passive callbacks. on_response is the main path for
capturing API response payloads from SPAs.
on_request/on_response previously fired only for navigation; op_fetch_url now
invokes them for JS fetch()/XHR too (both reqwest and stealth paths), gated on
a non-empty callback vector so the common no-callback case pays nothing.
Re-export InterceptedRequest, InterceptResolution, RequestInfo, ResourceType,
RequestCallback, ResponseCallback, Response from the obscura crate.
Integration test tests/interception.rs exercises all three against a local echo
server.
The wrapper Page cloned a second Arc<BrowserContext> the struct never read.
The inner obscura_browser::Page already holds the context and keeps it alive,
so the field was dead weight. Removes the dead_code warning reported on
aarch64
Closes#273
* fix(api): Browser::cookies() now shares context's CookieJar
Previously Browser::build() created its own empty CookieJar,
while BrowserContext internally creates a separate jar that
actually receives Set-Cookie headers from HTTP responses.
This caused browser.cookies().get_all() to always return
empty after navigation — the populated jar was unreachable.
Fix: use context.cookie_jar instead of creating a separate
one. Also removes redundant cookie-loading logic in
Browser::build() since BrowserContext::_new_inner already
handles loading from storage_dir.
Closes#241
* fix(browser): add <base href> support for script URL resolution (closes#242)
Previously resolve_base_url() returned the document URL directly,
causing relative <script src> URLs to resolve against the page URL
rather than the <base href> target. On SPAs served from sub-paths
(Angular/Vite/Next.js), this caused all bundle scripts to 404.
Add resolve_base_url() that queries DOM for <base href> and uses it
as the base for relative URL resolution per HTML spec. Applied to
both regular scripts and ES modules in execute_scripts().
* fix(js): implement Node.baseURI getter per WHATWG DOM spec (closes#244)
Node.baseURI was missing from the bootstrap.js Node class, causing
document.baseURI to return undefined. This broke Angular's
PlatformLocation.getBaseHref which calls new URL(href, document.baseURI)
-- with baseURI empty, the URL constructor throws Invalid URL.
Per https://dom.spec.whatwg.org/#dom-node-baseuri, Node.baseURI must
return the href of the first <base href> resolved against the document
URL, falling back to the document URL when no <base> is present.
* fix(js): add XMLHttpRequestEventTarget + XHR dispatchEvent (closes#245)
XHR shim lacked dispatchEvent and XMLHttpRequestEventTarget, causing
zone.js (Angular/zone.js) to crash with 'Cannot read properties of
undefined (reading call)' when Angular's HttpClient makes XHR calls.
Add spec-compliant XMLHttpRequestEventTarget base class with
addEventListener/removeEventListener/dispatchEvent. Make XHR extend
it so zone.js can walk the proto chain correctly.
* fix(js): fix XHR super() call order in ES6 class extends
* fix(js): use base href for dynamic script/module URL resolution
Angular lazy-loaded chunks were resolving against document.location.href
instead of the <base href>. This caused all dynamic import() chunks on
the Deutsche Boerse SPA to 404 (/bond/7591.xxx.js instead of /7591.xxx.js).
Fix: resolve dynamic script src against <base href> when present, falling
back to location.href per HTML spec.
* Revert "fix(js): use base href for dynamic script/module URL resolution"
This reverts commit 88af239be38f2aa11c3c978a4db2a7482f2ed6c0.
* feat(api): expose Page::settle() to pump JS event loop (closes#247)
obscura_browser::Page already had settle(max_ms) but it was unreachable
from the public API. Exposing it allows callers to resolve Promises,
fetch callbacks, and microtasks before reading results with evaluate().
---------
Co-authored-by: zhangyang-crazy-one <[email protected]>
* feat(api): add obscura-api crate with Browser + Page API
Adds a Rust library crate exposing Obscura as a programmable browser:
- Browser: build with config (proxy, stealth, UA, storage_dir)
- Page: goto, url, evaluate, content
- BrowserBuilder with fluent API
- examples/basic.rs
Part of #169
* feat(api): add Element API + CookieStore + wait_for_selector
- Element: text(), attribute(), click() via JS evaluation
- CookieStore: set/get_all/get_for_url/save_to_file/load_from_file
- Page::wait_for_selector with 100ms polling
- Page::query_selector for single element lookup
- Updated examples/basic.rs with full interaction demo
Part of #169
* feat(api): add Browser::cookies() method
* refactor(api): move to obscura crate with feature flag
- Renamed obscura-api → obscura (per maintainer feedback #169)
- API enabled by default via 'api' feature flag
- Usage: cargo add obscura (default=api) or cargo add obscura --features api
- Removed separate obscura-api crate
* chore: exclude docker workflow from fork
* fix(api): make the obscura crate build, run, and document install correctly
- examples/basic.rs and the lib.rs doctest imported `obscura_api`, but the crate
is named `obscura`, so both failed to compile. Use the real crate name.
- wait_for_selector / query_selector read the node id with `Value::as_u64`, but
`evaluate` returns JS numbers as f64, so it was always None and every
wait_for_selector timed out. Accept an integer or a non-negative finite float.
- restore .github/workflows/docker.yml, which the PR removed (a fork-specific
change that should not land upstream).
- add crates/obscura/README.md with honest install instructions: the crate is not
on crates.io, so depend on it via git (and it builds the embedded V8 from
source). The `cargo add obscura` form does not work yet.
Verified: workspace + example build, the lib doctest passes, the example runs end
to end (goto, wait_for_selector, text, attribute), and the integration suite is
28/28 with no regressions.
---------
Co-authored-by: zhangyang-crazy-one <[email protected]>
Co-authored-by: SGavrl <[email protected]>
SGavrl