Commit Graph
432 Commits
Author SHA1 Message Date
SGavrl 935cb5c0da fix(cli,js): add --version flag and script-error-resilience test (#147)
Two improvements while investigating #147:

1. `obscura --version` / `-V` now print the CARGO_PKG_VERSION instead of
   erroring out. The issue reporter explicitly called out that there was
   no version flag, and the existing `obscura serve` banner is the only
   way to discover the version.

2. New regression test in obscura-js verifies that a TypeError thrown
   from one script does not poison the runtime: subsequent scripts and
   DOM queries must continue to work. The reporter's symptom was that
   offside.js crashing collapsed --dump text output to 1 byte; that
   cascade should never happen and this test pins it down.

The classList error itself is offside.js's own bug (it indexes into the
querySelectorAll result without checking for an empty array) and only
fires when the .slideout-navigation element is absent from the DOM.
On current main that element parses correctly and offside.js runs
without error against a captured copy of the affected page.
2026-05-16 15:11:18 +00:00
SGavrl 8cb0cb429b fix(cli): refuse --stealth + socks5:// proxy instead of silent breakage (#160)
The previous workaround in page.rs rewrote socks5:// to http:// before
handing the proxy to the stealth client. That works for Clash-style
mixed-mode proxies, but breaks plain SOCKS5 servers like `ssh -ND`,
producing a confusing `TunnelUnexpectedEof` deep in the network layer.

The stealth client is backed by wreq, which does not speak SOCKS5,
so silently substituting schemes can never be correct in general.

Refuse the combination at CLI startup with a clear message that tells
the user what to do (use --proxy http://... or drop --stealth) and
revert the rewrite in Page::new. SOCKS5 without --stealth still works
through the reqwest path.
2026-05-15 21:23:53 +00:00
SGavrl b2b41529c4 ci: add aarch64-linux release binary (#127)
Uses GitHub's native ubuntu-22.04-arm runner so it's a native build
matching the same glibc 2.35 floor as the x86_64 target. README gets
the matching install snippet.
2026-05-15 21:10:29 +00:00
SGavrl aaeaf8f662 chore: drop author working notes that landed with #69 2026-05-15 20:44:46 +00:00
SGavrl b07863b5ce chore: drop author working notes that landed with #69 2026-05-15 20:44:32 +00:00
SGavrl 9c6dec4086 chore: drop author working notes that landed with #69 2026-05-15 20:44:21 +00:00
Alexandr KondakovandSGavrl 6dd1e4e65f Fix CDP click submit navigation parity (#69)
* Fix Runtime.evaluate awaitPromise settling

* Fix DOM event navigation side effects

* Fix CDP click-triggered navigation parity

* Strengthen CDP inline script click parity test

* Handle positions-like submit URLs in CDP parity


Co-authored-by: SGavrl <[email protected]>
2026-05-15 20:38:54 +00:00
SGavrl 3a6f3e6c22 Update README.md
cleaning up
2026-05-15 19:51:59 +00:00
Adam Ma 597fe5ef8d docs(readme): document --v8-flags and add Tuning V8 section (#152) (#153)
Adds a Global flags subsection to CLI Reference (covering --verbose,
--proxy, --user-agent, --obey-robots, --v8-flags) and a Tuning V8
section with the OOM-on-large-pages motivating example, so users
hitting JavaScript heap out of memory can find --v8-flags from the
use-case side rather than only from --help.

Closes #152. Depends on #131.
2026-05-15 19:51:09 +00:00
Adam Ma cb701fc2d8 feat(cli): add --v8-flags plumbing to configure V8 at startup (#130) (#131)
* feat(cli): add --v8-flags plumbing to configure V8 at startup (#130)

Obscura had no way to pass flags to V8, so users hitting OOM on pages
larger than ~5MB had no escape hatch. Adds a generic passthrough rather
than a one-off heap knob, so future V8 tuning needs no new code.

- New `obscura_js::set_v8_flags` (Once-guarded, empty-string no-op).
- Global `--v8-flags <FLAGS>` arg with `allow_hyphen_values = true`,
  applied once in `main` before any isolate is created.
- Usage: `obscura --v8-flags "--max-old-space-size=4096" fetch <url>`.

Closes #130.

* test(cli): expand --v8-flags coverage with normalize helper

Extract normalize_v8_flags() from main() so the trim/empty-check logic
is unit-testable without needing to invoke V8. Adds parse-coverage tests
for --v8-flags with serve/scrape subcommands and empty-string values,
plus pure-function tests for the normalizer.
2026-05-15 19:46:03 +00:00
h4ckf0r0day 7b209dc655 v0.1.5 2026-05-14 23:52:41 +02:00
SGavrl 18dc796df0 fix(cdp): gate file:// navigation behind --allow-file-access
GHSA-q55h-vfv9-qcr5. Any process able to reach the CDP port could
send Page.navigate {"url":"file:///etc/shadow"} and read arbitrary
local files (the process's UID's worth of read access). When obscura
runs as root inside Docker or as a systemd service that translates
straight into /etc/shadow, /root/.ssh/id_rsa, /proc/1/environ, etc.

Validate at the CDP boundary, not in obscura-net, so direct CLI
callers ("obscura fetch file://...") keep working. A new
BrowserContext.allow_file_access flag defaults to false; Page.navigate
returns an error when the requested URL is file:// and the flag is
off. CdpContext::new_with_security threads the flag from the server
entry point, and "obscura serve --allow-file-access" opts in for
local-HTML testing workflows.

Tested:
- Default obscura serve: CDP Page.navigate {file:///etc/passwd} ->
  refused with descriptive error.
- obscura serve --allow-file-access: same call succeeds.
- obscura fetch file://... CLI path: unchanged, still reads the file.
- All earlier PoCs (GHSA-6pj9 JS->file SOP, GHSA-4789 sub-resource
  SSRF, GHSA-4789 CSS template injection) still pass.
- cargo test --workspace clean.
2026-05-14 23:46:35 +02:00
SGavrl ac0efd1798 fix(browser): patch sub-resource SSRF, CSS injection, and silent redirect cap
Three findings from GHSA-4789-rq5x-jv3c.

HIGH-1: Sub-resource fetches (<script src>, <link rel=stylesheet>) had
no scheme gate, so an http(s) page could include
<script src="file:///etc/passwd"> and the body would come back as
script source. Added subresource_allowed() that mirrors the top-level
file:// SOP rule: file: sub-resources only load when the page itself
is on file:. Off-origin schemes (data:, javascript:, etc.) are
refused regardless. Private/loopback hosts were already blocked at
the obscura-net layer; this closes the file:// hole on top of that.

HIGH-2: CSS pulled from the network was string-formatted into a JS
template literal with a three-rule escaper (backslash, backtick,
${). That escaper did not handle U+2028 / U+2029 which terminate
ECMAScript template literals, letting attacker CSS break out and
run arbitrary code in the page's V8 realm. Replaced the inline
escape with escape_for_js_template_literal() which also handles
U+2028, U+2029, null, carriage return, and every other C0 control.

LOW-2: navigate_with_wait_post() capped JS-driven navigation chains
at ten but returned Ok(()) when the cap was hit, so callers could
not tell a successful load from a redirect storm. Now returns
PageError::TooManyRedirects(REDIRECT_LIMIT).

Verified with three PoCs (sub-resource SSRF, CSS template literal
break with U+2028, and the original GHSA-6pj9-j389-226p reproducer)
plus negative tests (file:// direct read, http -> http JS chain) all
pass. cargo test --workspace clean.
2026-05-14 23:45:26 +02:00
SGavrl 9edde41faf fix(browser): block JS-driven http -> file navigation, reset realm per nav
A page on http(s) could set window.onload, assign location.href to a
file:// URL, and then read document.body from the local file because
two things lined up: obscura-net allows file: URLs, and the JS
runtime was reused across navigations so window state from the prior
origin survived into the new document's load event.

Two fixes:

1. Refuse a JS-initiated chain step when it crosses from a non-file
   scheme into file:. Direct file:// navigations from the CLI/CDP
   still work because the gate only fires on the take_pending_nav
   path. file -> file chains also still work.

2. Drop and rebuild the V8 runtime on every navigation instead of
   only rebinding globalThis.document. Even within the same origin
   we should not carry over window.onload, custom properties, or
   registered event listeners from the previous document.

Tested with the PoC from the report: pages on http://127.0.0.1.nip.io
no longer leak the contents of file:///tmp/.../pwn.txt.
2026-05-14 23:15:37 +02:00
Ben Younes 09df3050f7 feat(cli): add --dump original for binary-safe raw response output (#117) (#148)
Fixes #117. Adds an `Original` variant to `--dump` that streams the raw
HTTP response body verbatim to stdout (or `--output`), bypassing the
browser/JS/DOM stack entirely. Useful for binary payloads (jpg, png,
fonts) and any non-HTML resource where parsing through the DOM layer
would corrupt or discard data.

The new path uses ObscuraHttpClient directly, honours --proxy and
--user-agent, and writes bytes through tokio::io::stdout so binary
payloads (including embedded NUL bytes) round-trip cleanly.
2026-05-14 10:00:07 +00:00
Ben Younes 2ff0f8fe1d fix(cdp): read and validate contextId in Runtime.evaluate / callFunctionOn (#51) (#149)
Fixes #51. `Runtime.evaluate` and `Runtime.callFunctionOn` previously
ignored the `contextId` / `executionContextId` parameter entirely, so
clients targeting an isolated world (e.g. Playwright's locator path,
which calls `Page.createIsolatedWorld` → contextId 100 → evaluates the
UtilityScript against that id) silently fell through to the default
context, breaking the contract advertised via
Runtime.executionContextCreated.

Track the set of advertised execution context ids on CdpContext
(pre-seeded with 1 and 2 — the default-frame contexts emitted by
Runtime.enable and post-navigation) and extend it from
Page.createIsolatedWorld. Both handlers now reject unknown ids with
`Cannot find context with specified id: <id>` — matching the wording
real Chrome returns — and log accepted ids at debug for observability.

Obscura still routes every JS call against the page's single V8 isolate,
so this PR is spec-compliance scaffolding rather than full multi-world
execution. Follow-up work to physically isolate worlds can build on the
same valid_context_ids registry.
2026-05-14 09:15:19 +00:00
Ben Younes f3c783f6f5 fix(net,js): route op_fetch_url + ES module loader through proxy_url (#139) (#151)
Fixes the remaining two paths from #139: JS-level fetch()/XHR
(op_fetch_url) and dynamic ES module imports (ObscuraModuleLoader) both
built their reqwest clients with `Client::builder().build()` — no
proxy — so every JS-initiated network call silently bypassed
BrowserContext.proxy_url. (The third path, StealthHttpClient, was
already fixed in main: page.rs:61 uses with_proxy.)

Changes:

* obscura-net: expose ObscuraHttpClient::proxy_url() so callers outside
  the crate can route their own reqwest requests through the same
  upstream proxy. The field was already there — just lacked an accessor.

* obscura-js/module_loader: add `proxy_url: Option<String>` and
  `with_proxy(base_url, proxy_url)`. ObscuraModuleLoader::new keeps the
  pre-#139 direct-connection behaviour. load() builds a reqwest client
  per import, threading the proxy and bubbling up "Invalid module proxy"
  on bad URLs.

* obscura-js/ops.rs: replace the process-wide
  `static SHARED_HTTP_CLIENT: OnceLock<reqwest::Client>` (which could
  never be reconfigured after first call) with a per-request
  `build_request_client(proxy_url)` that pulls the configured proxy out
  of SharedState.http_client.proxy_url() each call. Per-request build
  cost is ~1ms vs the network round-trip, and we no longer have a
  global singleton that lies about its config.

* obscura-js/runtime: add ObscuraJsRuntime::with_base_url_and_proxy.
  with_base_url delegates to it with None so existing tests / callers
  are unaffected.

* obscura-browser/page.rs: thread context.proxy_url through to
  with_base_url_and_proxy when constructing the JS runtime.
2026-05-14 08:25:55 +00:00
h4ckf0r0day a750797d15 v0.1.4 Release 2026-05-14 00:34:22 +02:00
SGavrl aeb1784c32 docs: add Docker install section to README 2026-05-13 22:31:44 +00:00
SGavrl 56260822b9 ci: auto-build and push Docker image on release tag
Adds a workflow that triggers on v* tag pushes and:
- Logs into Docker Hub using DOCKERHUB_USERNAME + DOCKERHUB_TOKEN secrets
- Builds the image from the existing Dockerfile
- Tags it with both the version (e.g. 0.1.4) and 'latest'
- Pushes both tags

GitHub Actions cache is used for layers to speed up subsequent builds.
2026-05-13 22:22:31 +00:00
SGavrl a23cdc0ae7 feat: add Dockerfile + --host flag for containerized deployment
Adds a multi-stage Dockerfile (rust:1-slim-bookworm builder → distroless/cc
runtime) plus a --host flag for obscura serve that defaults to 127.0.0.1
for safety but can be set to 0.0.0.0 so the container's port is reachable
via docker run -p mapping.

Dockerfile cherry-picked from #84. The SPA chunk loading commit from that
PR was left out — verified it didn't actually fix the Next.js webpack
errors on kiwi.com (#86).
2026-05-13 22:01:22 +00:00
Jordi Llonch bce055fd2a MCP support (#65)
* add mcp support

* add MCP docs to README

* wire up mcp subcommand in cli

* fix(stealth): propagate stealth feature to obscura-mcp
2026-05-13 21:38:10 +00:00
h4ckf0r0day f1cbd58894 chore(cli): bump banner version to v0.1.3 2026-05-13 23:03:52 +02:00
h4ckf0r0day 7be84b0fb7 ci: cross-compile x86_64-apple-darwin from macos-latest 2026-05-13 23:02:59 +02:00
h4ckf0r0day 27cd10156b fix(cli): use obscura-worker.exe on Windows (#140) 2026-05-13 22:56:09 +02:00
Timothy Olaleke deb5be90ee docs(skills): add agent skill for Obscura (#34)
* docs(skills): add agent skill for Obscura

Adds skills/obscura/SKILL.md following the agent-skill convention used
by addyosmani/agent-skills and vercel-labs/agent-browser. AI coding
agents that scan a repo for SKILL.md files will discover Obscura and
know when to reach for it instead of Chrome.

Covers: build (with and without stealth), CLI fetch, CDP server with
Puppeteer/Playwright examples, localhost / private-network opt-in,
scaling profile, and known limits.

* docs(skills): address review feedback

- Remove --allow-private-network and OBSCURA_ALLOW_PRIVATE_NETWORK examples (not on main)
- Drop "screenshot" trigger word; Obscura doesn't support screenshots (#37, #123)
- Reword Known Limits: screenshots are not implemented, not low-fidelity
2026-05-13 22:37:56 +02:00
Ben YounesandOra Studio d8b7c5fad1 feat(cli): add --dump markdown output mode (#54) (#78)
Surfaces the existing LP.getMarkdown CDP machinery as a CLI flag so
text-pipeline users (LLM agents, scrapers feeding markdown to other
tools) don't need a separate HTML-to-markdown converter step.

  obscura fetch --dump markdown 'https://example.com'

Layout:
- crates/obscura-js/src/markdown.rs holds HTML_TO_MARKDOWN_JS as a
  pub const so every layer can share the same DOM-walker. obscura-js
  is at the bottom of the dep DAG (obscura-browser -> obscura-js,
  obscura-cdp -> obscura-js, obscura-cli -> obscura-js) so a single
  source of truth works without a circular dependency.
- crates/obscura-browser/src/lib.rs re-exports it for ergonomics.
- crates/obscura-cdp/src/domains/lp.rs drops its inline duplicate
  and calls the shared const.
- crates/obscura-cli/src/main.rs adds the `Markdown` DumpFormat
  variant + dump_markdown(&mut Page) which page.evaluate's the
  shared script and prints the result.

Net diff: a new 75-line markdown.rs, a 70+ line deletion in lp.rs
that's reborn as 3 lines, and ~10 lines added across the other four
files. No behavior change for `--dump html|text|links`, and
LP.getMarkdown over CDP keeps the exact same JSON shape.

Four new unit tests in obscura-js cover headings, inline emphasis +
links, ordered/unordered lists, and that <script> / <style> contents
do not leak into the output.

Generated by Ora Studio
Vibe coded by ousamabenyounes

Co-authored-by: Ora Studio <[email protected]>
2026-05-13 22:36:17 +02:00
F0Rextasy 9fbd0c73a3 Plumb proxy into fetch and scrape (#88) 2026-05-13 22:34:02 +02:00
Ben YounesandOra Studio f2ce18cfb7 fix(cdp): implement Target.sendMessageToTarget for headless_chrome (#26) (#83)
headless_chrome (and older Puppeteer) wrap every CDP call inside
Target.sendMessageToTarget. Obscura was returning "Unknown Target
method" for the wrapper, which broke navigation through the high-level
client API even though the WebSocket itself connected.

Add a wrapper-level handler that:
1. Parses the inner JSON-encoded CDP request from params.message
2. Recursively dispatches it (boxed future to sidestep async-fn
   recursion limits) using the wrapper's sessionId so routing works
3. Re-emits the inner response as a Target.receivedMessageFromTarget
   event — the legacy contract these clients listen on instead of
   correlating responses by id
4. Returns {} for the wrapper itself

This addresses the headless_chrome leg of #26. The chromiumoxide
"data did not match any variant of untagged enum Message" decode
failure is a separate event/response shape issue — needs a focused
investigation against a chromiumoxide repro that's not in the issue,
so it's left out of this PR.

Generated by Ora Studio
Vibe coded by ousamabenyounes

Co-authored-by: Ora Studio <[email protected]>
2026-05-13 22:31:59 +02:00
chebee7i 1c295f5b32 fix: set CXXFLAGS/SDKROOT so boring-sys compiles on macOS 26 (Tahoe) (#138)
macOS 26 moved libc++ headers inside the SDK, breaking Apple clang 17's
default header search. The boring-sys cmake step fails with
'fatal error: memory file not found' when building --features stealth.

Adding CXXFLAGS and SDKROOT to .cargo/config.toml [env] propagates them
to all build scripts (Cargo's documented mechanism), so cmake picks up the
SDK's C++ include path without any manual env-var ceremony.

force=false preserves CI overrides and Xcode users who already have
SDKROOT set. Non-macOS platforms ignore SDKROOT; clang silently skips
-isystem paths that do not exist on disk.

Fixes #136
2026-05-13 22:29:01 +02:00
cpp1992andhandsome.cheng c6f6ce2cc2 fix(stealth): pass proxy config to StealthHttpClient (#141)
When --stealth is enabled, StealthHttpClient was created without
the proxy URL from --proxy, causing all stealth-mode requests to
bypass the proxy.

Also convert SOCKS5 proxy URL to HTTP when passing to wreq-based
StealthHttpClient, because wreq does not support SOCKS5 protocol
(Clash mixed port supports both protocols on the same port).

Fixes proxy connectivity for 'obscura serve --stealth --proxy socks5://...'

Co-authored-by: handsome.cheng <[email protected]>
2026-05-13 22:25:18 +02:00
SGavrl 5596c79490 docs: add Arch Linux AUR install instructions (#134)
Adds install instructions for Arch Linux via the AUR package obscura-browser.

AUR: https://aur.archlinux.org/packages/obscura-browser
2026-05-10 23:43:15 +02:00
Timothy Olaleke 85739d3f6a fix(#19): serialize V8 work across pages with process-wide async lock (#36)
* fix(#19): serialize V8 work across pages with process-wide async lock

Concurrent CDP clients (≥5) trip V8's
`heap->isolate() == Isolate::TryGetCurrent()` invariant and abort the
process via V8_Fatal — no Rust panic, just abort(3). Each Page owns its
own JsRuntime/Isolate and they all run on a single OS thread via
LocalSet+spawn_local; when two pages' V8-touching futures interleave
across an .await, V8 trips the check.

This adds a process-wide tokio::sync::Mutex (obscura_js::v8_lock) that
is acquired:

  - At the top of obscura_cdp::dispatch::dispatch (covers every CDP
    handler that may call into a JsRuntime).
  - In server.rs's process_with_interception spawn_local that runs
    page.navigate_with_wait + execute_preload_script (this path
    bypasses dispatch).

The guard keeps each V8-touching block contiguous on the thread: V8
fully exits one Isolate before another page is allowed in. Trade-off:
non-V8 handlers (Browser.*, Storage.*, Emulation.*) also serialize,
but those are cheap and the safety win dominates.

The properly concurrent fix is to pin each JsRuntime to its own OS
thread and message-pass; tracked as the larger #19 follow-up.

Adds an opt-in smoke test (`--ignored`) that boots the CDP server and
runs 5 parallel clients through createTarget+navigate. Note: the
deterministic abort needs subresource-fetch yields (page.rs:285
join_all) which data: URLs skip — full repro requires JS-heavy real
traffic (issue author used scrapegraphai.com).

* fix(#19): suspend other Isolates and defer foreign CDP during interception nav

Address maintainer feedback on PR #36 (comment 4328238087): the V8 fatal
still reproduces with `Fetch.enable patterns:["*"]` + 5 concurrent
`Page.navigate`s. The `obscura_js::v8_lock` mutex was insufficient
because V8's actual invariant is "only one Isolate may be entered per
OS thread", and the mutex can't enforce that across `.await` points
inside V8 ops.

Two distinct paths violated the invariant:

1. Foreign CDP messages dispatched while a nav task was running.
   `process_with_interception`'s `select!` routed every non-Fetch
   message through `process_cdp_message → dispatch → page::handle::*`.
   `get_session_page_mut` then `suspend_js`'es other pages (drops
   their `JsRuntime` → `Isolate::Exit`) while the spawned nav task is
   parked at `op_fetch_url`'s `resolve_rx.await` with its Isolate
   still entered. Drop trips
   `heap->isolate() == Isolate::TryGetCurrent()`.

2. Subsequent navs through `process_with_interception` did not
   suspend prior pages. After nav-1 completes (page-1's JsRuntime
   stays alive in `ctx.pages`), nav-2's `init_js` constructs
   Isolate-2 while Isolate-1 is still current. The next V8 scope
   unwind aborts via `Context::Exit`.

Fix (3 changes in `crates/obscura-cdp/src/server.rs`):

- `cdp_processor`: add a `VecDeque<ServerMessage> deferred` drained
  before each new `rx.recv()`, so deferred messages are processed
  with no nav task in flight.
- `process_with_interception` `select!`: only handle `NewConnection`
  and `Fetch.{continue,fulfill,fail}Request` inline (safe — no V8
  enter on this side); push everything else into `deferred`.
- `process_with_interception` entry: explicitly `suspend_js()` every
  other page in `ctx.pages` before spawning the nav task, mirroring
  the invariant the regular dispatch path already enforces via
  `get_session_page_mut`.

Verification:

- New test `crates/obscura-cdp/tests/concurrent_navigations_with_fetch.rs`
  is the maintainer's exact repro (5 clients, `Fetch.enable patterns:["*"]`,
  concurrent `Page.navigate` to https://example.com, auto-reply
  `Fetch.continueRequest`). Aborts on PR #36 head, passes after fix
  (5/5 stable runs, ~1.2s each).
- Existing data-URL smoke test (`concurrent_navigations`) still passes.
- Full workspace test suite (90 tests) still passes.

Run with:

  cargo test -p obscura-cdp --test concurrent_navigations_with_fetch \
    -- --nocapture --ignored

Honest caveat. This is still the lock/serialize approach — it converts
the abort into latency. The properly concurrent fix (Option 2 in
issue #19: pin each `JsRuntime` to its own OS thread and message-pass)
remains the right long-term direction.

* fix(cdp): bound deferred message queue to prevent OOM during stalled navigation

PR #36 comment 4341743194: process_with_interception deferred foreign
CDP messages into an unbounded VecDeque while a navigation task was in
flight. A stalled navigation (e.g. waiting for Fetch interception
resolution) could accumulate arbitrary queued messages and OOM the
process.

Add MAX_DEFERRED_MESSAGES = 256. When the cap is reached, return an
explicit CDP error (-32000) to the client instead of silently dropping
or growing without bound.

All 90 workspace tests pass.
2026-05-09 18:26:38 +02:00
404HumanandObscura Contributor 0de133eb2c fix(cdp): fast-path Target.getBrowserContexts for Puppeteer connect (#120)
Puppeteer's first CDP call after connect is Target.getBrowserContexts.
When the single cdp_processor is busy (e.g. Page.navigate with fetch
interception), queued Target commands stall and Puppeteer hits
protocolTimeout on getBrowserContexts before any user code runs.

Answer getBrowserContexts synchronously in fast_path_response with the
same payload as domains::target::handle (default context id "default"),
matching Browser context initialization in CdpContext.

Co-authored-by: Obscura Contributor <[email protected]>
2026-05-09 17:55:34 +02:00
GapesteinandSixSeven-Labs 0054c177c2 fix(stealth): gate wreq's prefix-symbols feature to Linux/Android (#39) (#106)
Co-authored-by: SixSeven-Labs <[email protected]>
2026-05-09 17:54:47 +02:00
Adam Ma a8b1ec0968 chore: silence unused import/variable/dead-code warnings (#101) (#126)
- obscura-net/cookies.rs: #[allow(dead_code)] on CookieEntry.same_site
  (parsed today, kept for future SameSite enforcement).
- obscura-js/runtime.rs: drop vestigial BOOTSTRAP_JS const; bootstrap.js
  is already embedded into the V8 snapshot via build.rs.
- obscura-js/ops.rs: prefix the four unused fields in
  InterceptResolution::Continue with _.
- obscura-cdp/dispatch.rs: drop unused InterceptResolution import.
- obscura-cdp/domains/fetch.rs: drop unused Arc / Mutex / CdpEvent imports.
- obscura-cdp/domains/input.rs: prefix _button / _click_count; CDP
  parses them but the JS click stub is hardcoded for now.
- obscura-cdp/server.rs: prefix _processor_handle and _ctx; drop dead
  initial Ok(())/None for navigate_result/page_back - they are
  unconditionally overwritten in the only loop break arm.

cargo build --workspace: 0 warnings.
cargo test --workspace: 113 passed.
2026-05-09 17:52:32 +02:00
h4ckf0r0day 61a3b9f44e fix(cli): consolidate three duplicate mod tests blocks 2026-05-09 17:43:12 +02:00
Ben YounesandOra Studio 023caf9ce5 fix(cdp): return descriptive error for Page.printToPDF instead of generic Unknown Page method (#53) (#112)
Playwright's `page.pdf()` issues a `Page.printToPDF` CDP request which
Obscura was not handling, so the dispatcher fell through to the generic
catch-all and clients saw:

    Page.pdf: Protocol error (Page.printToPDF): Unknown Page method: printToPDF

This reads like an oversight rather than a deliberate limitation.
Obscura intentionally has no layout engine — option (2) from the issue —
so make the limitation explicit by handling the method and returning a
clear error that names the method, says why it isn't supported, and
points users at a working alternative (extract HTML via
Runtime.evaluate, then render to PDF in their client).

The behavior change is small but meaningful for DX: the JSON-RPC error
code stays -32601 (the dispatcher's mapping for `Err(_)`), but the
human-readable message becomes actionable instead of looking like a
forgotten method.

Adds a unit test that pins the regression: printToPDF must NOT fall
through to the "Unknown Page method" catch-all and must mention both
"not supported" and a workaround.

Generated by Ora Studio
Vibe coded by ousamabenyounes

Co-authored-by: Ora Studio <[email protected]>
2026-05-09 17:36:11 +02:00
Ben YounesandOra Studio 7dfc2a9c01 fix(cli): strip nav/header/footer/aside from --dump text output (#55) (#82)
`obscura fetch --dump text` walked the entire <body> and emitted the
text of navigation menus, sidebars, and footers alongside the article
body. The skip list inside extract_readable_text already excluded
<script>/<style>; extend it with the four standard semantic-boilerplate
tags so the dump is closer to the article content users actually want
to scrape.

Behaviour change for users who relied on the noise being included is
acknowledged but unlikely — these tags are the canonical wrappers
boilerplate goes in. Anyone needing the full body should use --dump
html and post-process.

Generated by Ora Studio
Vibe coded by ousamabenyounes

Co-authored-by: Ora Studio <[email protected]>
2026-05-09 17:32:59 +02:00
F0Rextasy 81b9cf0f42 Add fetch output file option (#95) 2026-05-09 17:30:24 +02:00
F0Rextasy e00729e7c6 Add quiet mode for scrape (#91) 2026-05-09 17:28:52 +02:00
Ben YounesandOra Studio 84115ca06e fix(cli): make --quiet suppress JS console errors on stderr (#56) (#74)
* fix(cli): make --quiet suppress JS console errors on stderr (#56)

The --quiet flag was only gating banner prints in run_fetch but not
the global tracing_subscriber, so ERROR-level JS console output kept
leaking to stderr.

Detect quiet from the parsed Fetch subcommand before initializing
the subscriber and switch the filter to "off" in that case. Verbose
still wins over quiet to keep diagnostic mode usable.

Generated by Ora Studio
Vibe coded by ousamabenyounes

Co-Authored-By: Ora Studio <[email protected]>

* test(cli): add integration tests for --quiet matcher wiring

Address review feedback on PR #74: previous tests only covered the
pure helper select_log_filter, not the matches!() wiring in main()
that maps Command::Fetch { quiet: true, .. } → quiet flag.

Extract the matcher into is_quiet_command(&Option<Command>) and add
six new tests that parse real CLI args via Args::try_parse_from and
assert the full chain:

- parsed_fetch_with_quiet_flag_is_detected: --quiet on fetch → true
- parsed_fetch_without_quiet_is_not_detected: bare fetch → false
- parsed_serve_command_is_not_quiet: serve subcommand → false
- no_subcommand_is_not_quiet: None → false
- parsed_fetch_quiet_resolves_to_off_filter: end-to-end clap → "off"
- matcher_still_uses_fetch_variant: regression guard if Command::Fetch
  variant is renamed/restructured

cargo test -p obscura-cli: 10 passed; 0 failed.

Co-Authored-By: Ora Studio <[email protected]>

---------

Co-authored-by: Ora Studio <[email protected]>
2026-05-09 17:27:03 +02:00
h4ckf0r0day e60da1af39 docs: announce Obscura Cloud waitlist 2026-05-05 00:01:34 +02:00
Ben YounesandOra Studio 536072b323 fix(serve): plumb --user-agent through CDP server (#71) (#76)
obscura serve --user-agent <ua> only logged the value via tracing::info!
and never passed it to the CDP server, so every navigation kept using
the default Chrome UA string and httpbin.org/user-agent reported the
default rather than the override.

Plumb the option end-to-end:

- BrowserContext::with_full_options(id, proxy, stealth, user_agent)
  initializes both BrowserContext::user_agent and the underlying
  ObscuraHttpClient::user_agent RwLock at construction time so the
  first navigation already sees the override.
- CdpContext::new_with_full_options forwards the UA to the default
  BrowserContext.
- obscura_cdp::start_with_full_options accepts the UA and passes it
  through cdp_processor.
- main.rs Serve arm calls start_with_full_options with the parsed UA.
- run_multi_worker_serve forwards --user-agent <ua> when spawning
  worker subprocesses so multi-worker mode honors the flag too.

Existing two-arg start_with_options and three-arg with_options stay
as thin shims (call the full variant with None) so older callers and
the public crate API remain source-compatible.

Three new unit tests in obscura-browser cover the propagation, the
Chrome default fallback, and the legacy with_options path.

Generated by Ora Studio
Vibe coded by ousamabenyounes

Co-authored-by: Ora Studio <[email protected]>
2026-05-03 17:32:38 +02:00
F0Rextasy 9d8d3a02a9 Add fetch navigation timeout (#92) 2026-05-03 17:30:46 +02:00
Ben YounesandOra Studio a7b1acfa18 fix(cdp): accept Audits domain as no-op for Puppeteer compatibility (#57) (#80)
Puppeteer's FrameManager.initialize calls Audits.enable on connect.
Without it puppeteer.connect() fails with "Unknown domain: Audits"
before any user code runs. Playwright doesn't enable Audits, which is
why the workaround in #57 sidesteps the issue.

Add Audits to the existing accepted-but-no-op domain list, alongside
Log/Performance/Security/etc. that already swallow Puppeteer's connect
chatter. No new module needed.

Generated by Ora Studio
Vibe coded by ousamabenyounes

Co-authored-by: Ora Studio <[email protected]>
2026-05-03 17:22:24 +02:00
Ben YounesandOra Studio 111150a4f8 fix(js): add elementFromPoint / elementsFromPoint stubs (#63) (#75)
document.elementFromPoint(x, y) was undefined, which caused Google
Publisher Tag, viewability libs and any code doing hit-testing to
throw on bootstrap and trigger retry loops in the surrounding
framework. This is one of the throwing APIs feeding into the wedge
described in #62.

Without a layout engine we can't return the actual topmost element,
but a non-throwing stub is strictly better than "undefined" for the
headless-scraping use case:

- coords inside the viewport return document.body (or
  document.documentElement as a fallback) so callers that just want
  a hit target stop throwing
- coords outside the viewport (negative, > innerWidth/innerHeight,
  NaN/Infinity) return null per spec
- elementsFromPoint mirrors the same shape but returns a one- or
  zero-length array
- ShadowRoot.prototype gets the same treatment for completeness

Five new unit tests in obscura-js cover function presence, in-viewport
hit, out-of-viewport coords, non-numeric input, and the array shape.

Generated by Ora Studio
Vibe coded by ousamabenyounes

Co-authored-by: Ora Studio <[email protected]>
2026-05-03 17:19:17 +02:00
Ben YounesandOra Studio d50dd0f553 fix(js): make createEvent type-aware and add initCustomEvent (#41) (#77)
document.createEvent('CustomEvent') was returning a generic Event,
so the legacy DOM Level 2 pattern

  var e = document.createEvent('CustomEvent');
  e.initCustomEvent('foo', true, false, payload);

threw "TypeError: e.initCustomEvent is not a function" because the
returned object was an Event, not a CustomEvent — and CustomEvent
didn't define initCustomEvent anyway. The Starbucks China bundle
(and a long tail of other CDN bundles using the same shim) depended
on this and aborted bootstrap.

Two changes in crates/obscura-js/js/bootstrap.js:

1. Add CustomEvent.prototype.initCustomEvent(type, bubbles, cancelable,
   detail) matching the spec.
2. Rewrite Document.prototype.createEvent so it dispatches to the
   right class based on the type string ('CustomEvent', 'MouseEvent',
   'MouseEvents', 'KeyboardEvent', 'FocusEvent', 'InputEvent',
   'UIEvent', 'WheelEvent', 'PointerEvent', 'ErrorEvent',
   'PopStateEvent', 'AnimationEvent', 'TransitionEvent', and
   case-insensitive 'Events' aliases). The HTMLDocument variant on
   the second Document class delegates to the canonical impl.

Spec note: createEvent returns the event with an empty type until
init*Event() is called — matched exactly.

Four new unit tests cover initCustomEvent presence and field setters,
class dispatch for CustomEvent / MouseEvent / MouseEvents / KeyboardEvent,
and the unknown-type fallback to Event.

Generated by Ora Studio
Vibe coded by ousamabenyounes

Co-authored-by: Ora Studio <[email protected]>
2026-05-03 17:18:30 +02:00
KuaaMUandKuaaMU 86a868bfd6 fix: implement CharacterData DOM API for jQuery DataTables compatibility (#73)
jQuery DataTables references the standard CharacterData interface which
was undefined in obscura's DOM environment, causing ReferenceError and
empty page renders.

- Add CharacterData class extending Node with data/length/substringData/
  appendData/insertData/deleteData/replaceData
- Make Text and Comment extend CharacterData instead of Node
- Update _wrap() to create proper Text/Comment instances by nodeType
- Add create_comment_node op for Document.createComment()
- Fix set_text_content to handle Comment node mutations

Fixes #72

Co-authored-by: KuaaMU <[email protected]>
2026-05-03 17:16:26 +02:00
F0Rextasy 34205e50d1 Build Linux release on Ubuntu 22.04 (#87)
The current ubuntu-latest Linux release can require a newer glibc than common LTS servers provide. Pinning the x86_64 Linux release job to Ubuntu 22.04 keeps the prebuilt binary compatible with glibc 2.35+ environments while leaving macOS and Windows release jobs unchanged.

Also document the Linux LTS compatibility target in the install section so users understand why the release runner is pinned.
2026-05-03 17:14:54 +02:00