Documents the obscura-render layer: how to enable it (--features render),
the CLI --screenshot and CDP Page.captureScreenshot paths, what it
renders, the architecture, and a head-to-head benchmark.
Benchmark vs headless Chrome (same page, 1280x720, warmed):
- local fixture: obscura ~0.06s/33MB vs chrome ~0.90s/195MB
- example.com: obscura ~0.10s/34MB vs chrome ~0.98s/199MB
Roughly 10-15x faster and ~6x less memory. Chrome's larger PNG on the
real page reflects that it paints text, which obscura does not yet.
End-to-end rasterization behind the render feature (which now enables
obscura-render/paint):
- obscura-render: screenshot_png(dom, viewport) paints and encodes to PNG
via tiny-skia's encode_png (no extra dep).
- obscura-browser: Page::screenshot renders the live DOM at a viewport.
- CLI: obscura fetch --screenshot out.png writes the settled page to a PNG.
- CDP: Page.captureScreenshot returns the PNG base64-encoded, so
Puppeteer/Playwright page.screenshot() works. captureSnapshot still
errors (it is a DOM/layer snapshot, not an image).
Verified: both paths produce correct 1280x720 RGBA PNGs with elements
laid out and colored as the CSS specifies (block stacking, backgrounds,
white page). Default build unchanged; obstacle course 33/33.
Text, borders, and images are not yet painted; the box layout and
background colors are rendered.
Adds a paint feature to obscura-render (tiny-skia: pure Rust, CPU,
deterministic, no system deps). paint_dom lays out the DOM and fills
each element's border box with its parsed background-color over a white
page, in tree order so descendants paint over ancestors.
Style parsing now also extracts background-color (hex #rgb/#rgba/
later enhancements.
getBoundingClientRect, offsetWidth/Height, and offsetTop/Left now read
real box geometry from the obscura-render layout cache when the render
feature is compiled in. elementFromPoint inherits correct hit-testing
because it already delegates to getBoundingClientRect per element.
Adds a render feature on obscura-js (propagated through obscura-browser
and obscura-cli), a lazily-computed layout cache and viewport in
ObscuraState (feature-gated, cleared on navigation), and an
op_layout_geometry op. bootstrap.js probes the op with typeof, so the
default scraping build is unchanged and falls through to the synthetic
rect.
With --features render, block siblings stack and flex children place
horizontally at correct positions, and elementFromPoint resolves the
right element. Default build returns the synthetic rect unchanged.
Obstacle course 33/33; obscura-js unit failures identical to main.
Walk a live DomTree, compute each element's style (tag UA defaults plus
inline style), build a taffy layout tree, and return border-box geometry
keyed by NodeId. Block/flex structure, explicit sizes, and the box model
are correct; text nodes do not yet contribute to size (inline/text
layout lands with the paint phase).
This completes the layout core: a real DOM now produces real geometry,
pure Rust, behind the opt-in render crate. JS exposure is next.
Parse inline style attributes into the layout-relevant subset of
LayoutStyle: display, width/height, margin, padding, and border width,
including box shorthands (1-4 values) and per-side longhands. !important
is stripped; auto and percentages leave the size unset for phase 1.
Layered on a small built-in UA default sheet. A compact hand-rolled
tokenizer handles inline attributes without a cssparser dependency; a
real cascade can harden it later.
New optional crate obscura-render, the scoped render layer for Obscura.
Phase 1 adds the layout core: a LayoutNode tree plus a viewport is laid
out with taffy (pure Rust) and border-box geometry is returned per node,
mirroring the input tree.
The crate is a workspace member but not yet a dependency of any other
crate, so the default scraping build is unchanged. Geometry-from-DOM,
JS integration, text, paint, and screenshot capture land in later
phases.
Unit tests cover block stacking, flex row layout, and padding.
The wreq dependency was declared without the gzip, brotli, deflate and zstd
features, so the stealth client returned bodies still compressed while the
reqwest path decoded them. The Chrome145 emulation advertises those encodings,
so origins do compress, and the raw gzip bytes reached the HTML parser as
document text.
Add a regression test that serves one gzip response and asserts the stealth
client decodes it.
Fixes#563.
Setting innerHTML on the <html> element parses the fragment in its own context
(added in a2fd4d1), which runs html5ever's "before head" insertion mode and
synthesizes both <head> and <body> under the fragment root. The importer's
helper walked into the root <html> and, if it found a <body> child, returned
that body -- so import_children_from imported only the body's children and any
assigned <head> content (<title>, <meta>, <link>) was silently dropped.
A <body> child of the fragment root only appears in this <html>-context case;
every other element context leaves parsed content directly under the root and
already fell through to returning it. Rename find_body_or_root -> fragment_root
and return the root <html> element so both head and body children import.
Behavior for all non-<html> contexts is unchanged.
Fixes#556
A string setTimeout/setInterval handler was compiled with new Function(fn) at
scheduling time. That diverged from the HTML spec in two ways: top-level
var/function declarations in the body stayed local to the compiled function
instead of becoming globals, and a syntax error was caught eagerly at
scheduling and turned into a silent no-op (that still returned a timer id)
rather than surfacing when the timer fires.
Defer string handlers to fire-time indirect eval ((0, eval)(src)), which runs
in the true global scope like a classic script, so declarations become global
and a SyntaxError is thrown when the timer elapses. Matches the dynamic-script
path, which already uses indirect eval for the same reason. Function handlers
are unchanged.
Fixes#558
The stealth (wreq) client gated its custom-trust-store build on
`var_os(...).is_some()`, which is true for a set-but-empty variable. Supplying
a store to wreq's `tls_cert_store` REPLACES the bundled webpki roots, so an
empty `SSL_CERT_FILE` (a common shell accident, e.g. `export SSL_CERT_FILE=$UNSET`)
built a near-empty default-paths store and made every HTTPS request fail with
CERTIFICATE_VERIFY_FAILED -- while the reqwest client, which already filters
empty values in configured_root_paths(), kept working. Same input, opposite
behavior.
Extract a shared `custom_cert_store_requested` predicate in client.rs that
applies the same `!is_empty()` filter, and call it from the stealth client's
guard. The predicate takes the values as arguments so it is unit-tested under
default features (the stealth feature pulls in boringssl and is not built in
the normal test job).
Fixes#551
The check/uncheck/select branches of browser_fill_form were left on the
pre-#530 pattern: a direct `el.checked = true` / `el.selectedIndex = i` plus a
raw, untrusted `change` event. For a React-controlled checkbox the direct
assignment runs the framework's value tracker in lockstep, so onChange never
commits and the box reverts; and any handler gating on event.isTrusted ignores
the untrusted events -- the exact bug class #530 fixed for text inputs two
lines away.
Route check/uncheck through __obscura_setFieldValue(el, 'checked', ...) so the
write goes through the prototype setter and leaves the tracker stale, and
dispatch input+change wrapped in __obscura_markTrusted for all three branches,
matching the text-input branch.
Fixes#550
The Event and CustomEvent constructors diverged from Chrome in three
observable ways:
- new Event() / new CustomEvent() with no argument did not throw, though the
type argument is required (Chrome: "1 argument required, but only 0
present."). initEvent/initMouseEvent already had this check; the
constructors did not.
- the type argument was not coerced to a string, so new Event(123).type was
the number 123 instead of "123".
- CustomEvent.detail defaulted to undefined; WebIDL CustomEventInit specifies
`any detail = null`.
Add the required-argument guard to both constructors, coerce type via
String(t) (inherited by every subclass through super), and default detail to
null. createEvent() builds events with an explicit '' type, so the arity
guard leaves the type-"" uninitialized-event path intact.
Fixes#552
CSSStyleDeclaration was added to the _preHideInternals non-enumerable
pre-declaration list (commit c7e7c70) on the assumption that its later
`globalThis.X = X` assignment would fill in the value. But it was the only
one of the 22 WebIDL interfaces with no such assignment, so its value stayed
undefined: `typeof window.CSSStyleDeclaration` was "undefined" while
`'CSSStyleDeclaration' in window` was true, and because it is the type of
element.style / getComputedStyle(), `el.style instanceof CSSStyleDeclaration`
threw "Right-hand side of 'instanceof' is not callable" (Chrome returns true).
Assign globalThis.CSSStyleDeclaration alongside the other interface globals.
The property was already declared writable/configurable, so the assignment
only updates the value and it remains enumerable:false, like a real browser.
Fixes#546
Node::set_attribute looked up the existing attribute by local name alone,
while get_attribute and the remove_attribute op both match by qualified
name. A parsed namespaced attribute is stored with a separate prefix
(xlink:href -> prefix="xlink", local="href"), so setAttribute("xlink:href")
never matched it: the update was pushed as a second attribute, getAttribute
kept returning the stale original, and the element serialized xlink:href
twice.
Match set_attribute by qualified_name_eq, consistent with the read/remove
paths, so the update lands in place and no duplicate is created.
Fixes#547
DOMParser.parseFromString always HTML-parsed, even for XML mime types, so
malformed XML never produced the <parsererror> element Chrome emits and
error-detection code (doc.querySelector('parsererror')) never triggered. obscura
has no XML parser, so full XML semantics stay out of scope; instead, for XML
mime types run a conservative well-formedness check (tag balance, single root,
terminated comment/CDATA/PI, skipping quoted attribute regions) and, on a clear
error, make the document surface a <parsererror>. The check defaults to
well-formed when ambiguous, so valid XML is never falsely flagged.
Fixes#504
* fix: add missing #[ignore] gate and local fixture for concurrent fetch test
* fix(test): make fetch fixture nonblocking
---------
Co-authored-by: SGavrl <277843151+SGavrl@users.noreply.github.com>
`bootstrap.js` publishes its DOM interfaces with `globalThis.Node = Node`,
which defaults to `enumerable: true`. A real browser exposes every WebIDL
interface on the global as `enumerable: false`, so the difference is a
one-liner for any script that cares:
Object.getOwnPropertyDescriptor(window, 'Node').enumerable
// real Chrome: false obscura: true
It also means `Object.keys(window)` and `for...in` over the global enumerate
22 names a browser would never list.
The file already has the mechanism for this -- `_preHideInternals()`
pre-declares internal globals with `enumerable: false`, and its own comment
notes that later `globalThis.x = value` assignments only update the value and
leave the descriptor intact. This just adds the WebIDL names to that list, so
no assignment site changes.
Before: 9 of 10 sampled interfaces enumerable. After: 0 of 10, matching Chrome.
Regression: interfaces still callable (`typeof Node === 'function'`),
`instanceof Node` / `instanceof Element` hold, `querySelector` works, Hacker
News still yields 30 results under `--stealth`, snapshot builds clean.
Composes with #537 -- verified with both applied: `var Node` pages run, the
JSF/RichFaces case loads (jQuery/A4J/RichFaces all live), and enumerability
stays at 0/10.
The #525 fix landed on the reqwest path (`client.rs`: `configured_root_paths()`
-> `add_root_certificate`) but did not cover `wreq_client.rs`. The stealth
client builds its `wreq::Client` without a certificate store, so those
variables are silently ignored whenever `--stealth` is on.
The consequence is inverted: the transport with the best TLS fingerprint is the
only one that cannot reach a host behind a private or incomplete CA chain --
precisely the environments #525 described.
Same binary on current main, same private-CA server, only the flag differs:
SSL_CERT_FILE=ca.pem obscura fetch https://127.0.0.1:8443/ -> ok
SSL_CERT_FILE=ca.pem obscura --stealth fetch https://127.0.0.1:8443/ -> fails
`wreq` already exposes the hook -- `ClientBuilder::tls_cert_store(CertStore)`,
and `CertStore::builder().set_default_paths()` reads both variables. It is
simply never called.
Two constraints, both of which fail silently if got wrong:
* `tls_cert_store`, not `tls_options`. `emulation()` is documented as
"This will overwrite the existing configuration" and internally calls
`self.tls_options(emulation.tls_options)`; setting TLS options here would
discard the Chrome fingerprint, which is the whole point of this client.
`tls_cert_store` is a separate config field and composes with emulation.
* Opt-in only. Supplying a store REPLACES the webpki roots (see
`set_cert_store` in `tls/conn/ext.rs`), it does not add to them. Applying it
unconditionally would break ordinary sites whenever the bundle is
incomplete. With neither variable set, behaviour is byte-for-byte unchanged.
Verified: with the variable, a private-CA host resolves under --stealth;
without it, the same host still fails; with a deliberately EMPTY bundle,
example.com fails -- which is what proves the store is actually applied rather
than the bundle merely happening to be right.
Refs #538, #525
