Commit Graph

675 Commits

Author SHA1 Message Date
F0Rextasy
986bcf5b8e Reject zero scrape concurrency (#96) 2026-05-03 16:46:01 +02:00
F0Rextasy
0e84f15711 fix(cli): remove unused AsyncReadExt import (#98) 2026-05-03 16:45:36 +02:00
F0Rextasy
9fc94b6d7b Fix license metadata and README heading (#89) 2026-05-03 16:43:58 +02:00
F0Rextasy
64c22e8d87 fix(net): remove unused IpAddr import (#97) 2026-05-03 16:43:33 +02:00
rifuki
53a04fa905 fix(wasm): preserve binary fetch responses (#48)
* fix wasm response handling

* fix URL inputs in fetch polyfill

* test(wasm): cover binary fetch responses
2026-04-27 17:31:54 +02:00
h4ckf0r0day
fdec52c70f Merge pull request #44 from zhangyang-crazy-one/feat/accessibility-getFullAXTree
feat(cdp): implement Accessibility domain (getFullAXTree)
2026-04-27 16:11:38 +02:00
zhangyang-crazy-one
266afa7c34 fix(cdp): wrap role/name/value as CDP AXValue objects
Per qodo-ai-reviewer feedback on PR #44: Accessibility.getFullAXTree
was emitting AXNode.role as a plain string and name/value as plain
strings or null, instead of CDP-compliant AXValue objects.

Changes:
- Add ax_value_role() and ax_value_string() helpers
- Change compute_name()/compute_value() to return Option<String>
- Wrap role, name, value as AXValue in build_ax_node()

The properties array already used the correct AXValue format.
2026-04-27 20:59:47 +08:00
zhangyang-crazy-one
a4591707e3 feat(cdp): implement Accessibility domain (getFullAXTree)
Implements the CDP Accessibility domain with getFullAXTree support,
enabling AI agent tools (agent-browser, Playwright MCP) that rely on
Accessibility.getFullAXTree to work with Obscura.

Changes:
- Add crates/obscura-cdp/src/domains/accessibility.rs (~380 lines)
  - Accessibility.enable: returns {}
  - Accessibility.getFullAXTree: walks DOM tree, produces AXNode array
  - Role mapping: 50+ HTML tags → ARIA roles, respects explicit role attr
  - Name computation: aria-label → aria-labelledby → alt → title → placeholder
  - Properties: focusable, editable, checked, disabled, level, required, multiline
- Update mod.rs: register accessibility module
- Update dispatch.rs: route Accessibility domain to real handler

Tested with example.com (18 nodes), httpbin.org/html (16 nodes),
news.ycombinator.com (1308 nodes). Zero structure errors.

Closes #12
2026-04-27 12:02:38 +08:00
h4ckf0r0day
99e75f1e62 fix: load balancer panics, stealth flag, shadow DOM polyfill
Fixes #1: replace unwrap() panics in run_multi_worker_serve with match on the Result.
Fixes #2: route /json round-robin via worker_port instead of hardcoded port+1.
Fixes #3: return 502 Bad Gateway when a worker is unreachable instead of silently dropping the connection.
Fixes #5: thread --stealth through start_with_options -> CdpContext -> BrowserContext::with_options so the flag actually has an effect in serve mode (was let _ before). Log message now reflects the actual capability based on the stealth cargo feature being compiled in.
Fixes #15: try/catch the parentNode assignment in the attachShadow polyfill so it doesn't throw a TypeError against the read-only Node.parentNode getter (Cloudflare Turnstile triggers this path).
2026-04-25 14:27:52 +02:00
h4ckf0r0day
74cf88d0f4 Merge pull request #30 from abdul/main
Add macOS target and installation instructions for Intel users
2026-04-25 13:57:02 +02:00
Abdul Qabiz
a458738e5a Add macOS Intel installation instructions
Added installation instructions for macOS Intel users.
2026-04-25 16:57:33 +05:30
Abdul Qabiz
6b80f3d28d Add macOS target for x86_64 architecture 2026-04-25 16:55:04 +05:30
h4ckf0r0day
efeb174d27 Merge pull request #20 from mvanhorn/fix/7-parallel-scrape-timeout
fix(scrape): add per-worker timeout to prevent indefinite hangs
2026-04-25 13:01:11 +02:00
h4ckf0r0day
d9fb4c4069 Merge pull request #29 from abdul/patch-1
Update README to remove macOS Intel instructions
2026-04-25 12:42:02 +02:00
h4ckf0r0day
00ee2db9cb Merge pull request #17 from ben1009/feat/support-file-urls
feat: support file:// URL scheme for local HTML testing
2026-04-25 12:36:57 +02:00
Abdul Qabiz
4666d34adb Update README to remove macOS Intel instructions
Removed installation instructions for macOS Intel because there is no macOS Intel build.
2026-04-25 16:03:35 +05:30
h4ckf0r0day
78a94e27fc Merge pull request #25 from mikeumus/fix-playwright-cdp
fix(cdp): five Playwright drop-in compatibility patches
2026-04-25 12:32:20 +02:00
Mike Mooring
9e3e1e4ef7 fix(cdp): five Playwright drop-in compatibility patches
Brings Obscura's CDP server in line with what Playwright's
`chromium.connectOverCDP()` expects. Each fix is small and surgical;
together they make `connect → newContext → newPage → goto → page.title()`
work end-to-end against `https://example.com`.

1. **server.rs — emit pending events before command response.**
   Chromium CDP semantics require side-effect events (e.g.
   `Target.targetCreated` + `Target.attachedToTarget` from
   `Target.createTarget`) to arrive *before* the originating command's
   response. Playwright's wire reader assumes this ordering — it
   resolves the response promise on the response and then synchronously
   looks up the freshly-created session. Reversing the order makes that
   lookup miss and the page hang at `newPage()`.

2. **domains/browser.rs — accept Browser.setWindowBounds as a no-op.**
   Playwright calls `Browser.setWindowBounds` during teardown. Returning
   "Unknown Browser method" causes Playwright to treat the target as
   broken and tear it down mid-test, surfacing as "Frame has been
   detached" on the next `page.title()`.

3. **obscura-browser/page.rs — frame_id == target_id for main frame.**
   Was `format!("{id}.1", id)`. Playwright's frame manager indexes
   frames by frame id and looks up the main frame by target id, so the
   `.1` suffix made every main-frame lookup miss.

4. **domains/page.rs + dispatch.rs — track isolated worlds and re-emit
   them post-navigation.** Playwright registers a utility world via
   `Page.createIsolatedWorld` and expects `Runtime.executionContextCreated`
   for each registered world after every navigation. Without re-emission,
   the utility-world handle goes stale post-`goto` and `page.title()`
   hangs waiting for the world's execution context. Falls back to the
   legacy `__puppeteer_utility_world__24.40.0` name when no worlds have
   been registered, preserving prior Puppeteer support.

5. **obscura-js/runtime.rs — strip trailing semicolons before wrapping.**
   Playwright's utility-script expression is an IIFE ending in `})();`.
   Both `wrap_expression` (CDP `Runtime.evaluate`) and `evaluate_for_cdp`
   (the `meta_code` IIFE) embed the user expression inside `({EXPR})` /
   `return ({EXPR});`. With the trailing `;` left in, this produces
   `({...;})` — a SyntaxError. Parse errors are not catchable, so the
   surrounding try/catch never fires and the call silently returns
   `undefined`, surfacing as `page.title() === ""`.

Verified end-to-end against `https://example.com`:
  [probe] connect OK
  [probe] newContext OK
  [probe] newPage OK
  [probe] goto OK
  [probe] title = Example Domain
2026-04-24 23:27:18 -07:00
Matt Van Horn
065b1b1a1f fix(scrape): add per-worker timeout to prevent indefinite hangs
Closes #7.

A stuck worker subprocess in `obscura scrape` could hold its semaphore
permit forever and hang the entire job. The two `BufReader::read_line`
calls and the final `child.wait()` were unbounded; a worker that never
wrote a response line, or a `--eval` expression that never terminated,
would block indefinitely.

This wraps the per-worker spawn->navigate->evaluate->shutdown sequence
in `tokio::time::timeout(worker_timeout, ...)` and adds tighter
`tokio::time::timeout(read_timeout, ...)` wrappers around each
`read_line` call. Shutdown waits and the kill-on-timeout path are
bounded with a 5s `shutdown_timeout`, so a misbehaving worker can't
block cleanup either.

Adds a `--timeout <SECS>` flag on the `scrape` subcommand (default 60,
clap-validated `>= 1`). On timeout, the child is killed and the result
JSON entry becomes `{"url": ..., "error": "timeout", "time_ms": ...}`,
keeping the output stream well-formed.

Per-worker max time is now bounded by `worker_timeout` (outer) plus the
shutdown_timeout cleanup window, regardless of worker behavior.
2026-04-24 16:00:48 -07:00
ben1009
21136ca819 feat: support file:// URL scheme for local HTML testing
Allow loading local HTML/CSS/JS files via the file:// protocol,
enabling Obscura to be used for testing static sites without
running a local HTTP server.

Changes:
- Allow "file" scheme in validate_url() and validate_fetch_url()
- Add fetch_file_url() to read local files and return synthetic
  HTTP responses with appropriate content-type headers
- Skip private IP / localhost checks for file:// URLs

Closes #16
2026-04-25 00:18:31 +08:00
h4ckf0r0day
b00a1f20e7 Update README.md 2026-04-13 16:01:07 +02:00
h4ckf0r0day
bd42160477 Update release.yml 2026-04-13 12:46:30 +02:00
h4ckf0r0day
575a2b4c55 Update release.yml 2026-04-13 12:41:50 +02:00
h4ckf0r0day
8749725b45 Initial Release 2026-04-13 12:30:28 +02:00
h4ckf0r0day
2e5d63cc35 Initial commit 2026-04-13 11:20:11 +02:00