A whole DOM text node was one indivisible taffy leaf. Flex-wrap only
breaks between items, never inside one, so a long run of plain text
with no inline element breaking it up (an ordinary sentence with no
link in it) either fit on one line or ran straight past the
container's edge instead of wrapping. Short runs interrupted by links
every few words (extremely common in Wikipedia's markup) happened to
be narrow enough to wrap on their own, which is why paragraphs looked
fine but any long plain-text stretch overflowed straight over the
infobox and the History-section image.
Text nodes now split into one taffy leaf per word, flattened into the
same child list as sibling elements so words from different text nodes
and inline elements interleave and wrap together like real inline
flow. This changes the geometry data model: a text node's content is
now a list of (box, word) pairs (DomLayout::text_runs) instead of a
single box, since it can span several lines. Updated painting to draw
each word at its own position, and fixed a bold-text measurement
mismatch (layout width and drawn width disagreed by 1px/char, invisible
before as one blob, visible as gaps between every word once each word
is independently positioned).
Added a regression test that fails against the old single-leaf
behavior.
These were bundled in with the truly dynamic, interaction-only
pseudo-classes (:hover/:active/:focus) and always resolved to false.
That's right for :hover et al (there's no live cursor against a static
snapshot) but wrong for :enabled/:disabled/:checked: those reflect
real, static DOM state (the disabled/checked attributes), not user
interaction, and modern component systems (Codex, Material, Bootstrap)
lean on :enabled constantly for their base button/input styling. With
it always false, every such base rule was silently inert, which is why
the Wikipedia TOC toggle button (a <button> with no disabled
attribute, so :enabled should be true) kept its default black border
instead of the transparent one its :enabled.cdx-button--weight-quiet
rule sets.
:enabled now matches a form control (input/button/select/textarea/
optgroup/option/fieldset) without a disabled attribute, :disabled the
opposite, and :checked matches the checked/selected attribute.
Icons were rendering as either overlapping solid squares or not at all,
text ran under the infobox instead of beside it, and general.css parsing
had two more silent-corruption bugs on top of the ones fixed earlier:
- A CSS declaration list was split on every literal semicolon, so any
value containing one inside a quoted string or url() (a data: URI
with a MIME parameter, e.g. url(data:image/svg+xml;utf8,<svg...>),
extremely common for small inline icon SVGs) got torn into two
malformed halves and silently dropped. This is why mask-image never
applied to the TOC toggle icons at all.
- data: URIs were only decoded as base64. Inline SVGs are just as
commonly written as data:image/svg+xml;utf8,<percent-escaped text>,
which needs percent-decoding instead; added that path.
- top/left/right/bottom/width/height only handled bare lengths and a
flat calc(a +/- b). Real values nest var(--x, fallback), calc(), and
min()/max() inside each other (Wikipedia's icon offset math is
literally calc(-1 * 22px / 2), and its icon sizing is
calc(max(calc(var(--font-size-medium,1rem) + 4px),10px))). Replaced
the flat evaluator with a small recursive resolver that handles all
three, nested arbitrarily. This is what was placing the TOC toggle
button's icon on top of the section text instead of in the left
margin where it belongs.
Added actual feature support once the parsing was correct:
- mask-image/-webkit-mask-image: the pattern behind most scalable
colored icons on the modern web (an SVG or image used purely as an
alpha stencil, tinted by background-color/color). Previously every
masked icon rendered as an unmasked solid-color square.
- float: left/right, approximated without a full per-line reflow
engine (which taffy's block/flex/grid modes don't provide): a
floated element is grouped into a row alongside the flow siblings
that follow it, up to whichever comes first of the next heading or
another float, then everything from there on reverts to normal
full-width flow. This isn't general CSS float, but it directly
targets the overwhelmingly common real shape (an infobox or image
floated near the top of an article, with the rest of the content
starting at the next section heading), and is what makes article
text actually wrap beside an infobox instead of running underneath
it.
Refactored paint.rs's image fetch/decode/cache logic into shared
helpers (fetch_bytes, raster_to_pixmap) reused by plain <img> painting,
background-image, and the new mask-image path.
The render layer had no CSS Grid support, so Vector 2022's page layout
(three nested grids) collapsed to a vertical stack with the article
landing tens of thousands of pixels down the page. Wikipedia rendered
as a near-blank page and Hacker News only worked through per-site
hardcoded CSS overrides.
Removed the HN-specific hardcoding and fixed the general engine instead:
- CSS Grid layout (grid-template-areas/columns/rows via taffy's grid mode)
- Indexed, specificity-correct cascade in a new css.rs, replacing the
old test-every-selector-against-every-node scan. Cascade time on
Wikipedia went from 9.5s to under 200ms.
- Ancestor bloom-filter fast-rejection for selector matching in
obscura-dom, needed to make the indexed cascade actually fast on
descendant combinators.
- :link/:visited pseudo-classes were previously unparseable, so any
rule using them (a:link{color:...}, extremely common) silently
dropped the whole rule from the cascade.
- overflow:hidden clipping, fixing the pervasive visually-hidden-but-
accessible pattern used for skip links and screen-reader-only labels.
- @media parser robustness: it did a literal substring search for
max-width: and silently dropped the constraint on max-width : 750px
(space before colon) or max-width:calc(750px - 1px), treating
mobile-only or narrow-viewport rules as always active. The calc()
case was hiding Wikipedia's TOC sidebar via a rule meant only for
narrow screens.
- Table auto-layout approximation: the trailing auto-width td in a row
absorbs leftover space, so a width:100% table's unsized last column
reaches the right edge instead of shrinking to content.
- Inline formatting context approximation: a block holding inline-level
children (a paragraph or list item with links) is promoted to a
wrapping flex row so text and links read as continuous prose instead
of one fragment per line.
- Whitespace-only text nodes no longer get a full line-height box. They
used to stack up as blank rows, turning Wikipedia's collapsed TOC
subsections and the page header into hundreds of pixels of dead space.
- SVG rendering via resvg/usvg, plus resolving relative image URLs
against the page's base URL.
- background-image support including background-size/background-position
for correctly sized and positioned icons.
- Per-screenshot image fetch cache.
- Fixed the layout-only (no paint feature) build, which didn't compile
due to an unconditional call into paint-gated code.
Documents the obscura-render layer: how to enable it (--features render),
the CLI --screenshot and CDP Page.captureScreenshot paths, what it
renders, the architecture, and a head-to-head benchmark.
Benchmark vs headless Chrome (same page, 1280x720, warmed):
- local fixture: obscura ~0.06s/33MB vs chrome ~0.90s/195MB
- example.com: obscura ~0.10s/34MB vs chrome ~0.98s/199MB
Roughly 10-15x faster and ~6x less memory. Chrome's larger PNG on the
real page reflects that it paints text, which obscura does not yet.
End-to-end rasterization behind the render feature (which now enables
obscura-render/paint):
- obscura-render: screenshot_png(dom, viewport) paints and encodes to PNG
via tiny-skia's encode_png (no extra dep).
- obscura-browser: Page::screenshot renders the live DOM at a viewport.
- CLI: obscura fetch --screenshot out.png writes the settled page to a PNG.
- CDP: Page.captureScreenshot returns the PNG base64-encoded, so
Puppeteer/Playwright page.screenshot() works. captureSnapshot still
errors (it is a DOM/layer snapshot, not an image).
Verified: both paths produce correct 1280x720 RGBA PNGs with elements
laid out and colored as the CSS specifies (block stacking, backgrounds,
white page). Default build unchanged; obstacle course 33/33.
Text, borders, and images are not yet painted; the box layout and
background colors are rendered.
Adds a paint feature to obscura-render (tiny-skia: pure Rust, CPU,
deterministic, no system deps). paint_dom lays out the DOM and fills
each element's border box with its parsed background-color over a white
page, in tree order so descendants paint over ancestors.
Style parsing now also extracts background-color (hex #rgb/#rgba/
later enhancements.
getBoundingClientRect, offsetWidth/Height, and offsetTop/Left now read
real box geometry from the obscura-render layout cache when the render
feature is compiled in. elementFromPoint inherits correct hit-testing
because it already delegates to getBoundingClientRect per element.
Adds a render feature on obscura-js (propagated through obscura-browser
and obscura-cli), a lazily-computed layout cache and viewport in
ObscuraState (feature-gated, cleared on navigation), and an
op_layout_geometry op. bootstrap.js probes the op with typeof, so the
default scraping build is unchanged and falls through to the synthetic
rect.
With --features render, block siblings stack and flex children place
horizontally at correct positions, and elementFromPoint resolves the
right element. Default build returns the synthetic rect unchanged.
Obstacle course 33/33; obscura-js unit failures identical to main.
Walk a live DomTree, compute each element's style (tag UA defaults plus
inline style), build a taffy layout tree, and return border-box geometry
keyed by NodeId. Block/flex structure, explicit sizes, and the box model
are correct; text nodes do not yet contribute to size (inline/text
layout lands with the paint phase).
This completes the layout core: a real DOM now produces real geometry,
pure Rust, behind the opt-in render crate. JS exposure is next.
Parse inline style attributes into the layout-relevant subset of
LayoutStyle: display, width/height, margin, padding, and border width,
including box shorthands (1-4 values) and per-side longhands. !important
is stripped; auto and percentages leave the size unset for phase 1.
Layered on a small built-in UA default sheet. A compact hand-rolled
tokenizer handles inline attributes without a cssparser dependency; a
real cascade can harden it later.
New optional crate obscura-render, the scoped render layer for Obscura.
Phase 1 adds the layout core: a LayoutNode tree plus a viewport is laid
out with taffy (pure Rust) and border-box geometry is returned per node,
mirroring the input tree.
The crate is a workspace member but not yet a dependency of any other
crate, so the default scraping build is unchanged. Geometry-from-DOM,
JS integration, text, paint, and screenshot capture land in later
phases.
Unit tests cover block stacking, flex row layout, and padding.
The wreq dependency was declared without the gzip, brotli, deflate and zstd
features, so the stealth client returned bodies still compressed while the
reqwest path decoded them. The Chrome145 emulation advertises those encodings,
so origins do compress, and the raw gzip bytes reached the HTML parser as
document text.
Add a regression test that serves one gzip response and asserts the stealth
client decodes it.
Fixes#563.
Setting innerHTML on the <html> element parses the fragment in its own context
(added in a2fd4d1), which runs html5ever's "before head" insertion mode and
synthesizes both <head> and <body> under the fragment root. The importer's
helper walked into the root <html> and, if it found a <body> child, returned
that body -- so import_children_from imported only the body's children and any
assigned <head> content (<title>, <meta>, <link>) was silently dropped.
A <body> child of the fragment root only appears in this <html>-context case;
every other element context leaves parsed content directly under the root and
already fell through to returning it. Rename find_body_or_root -> fragment_root
and return the root <html> element so both head and body children import.
Behavior for all non-<html> contexts is unchanged.
Fixes#556
A string setTimeout/setInterval handler was compiled with new Function(fn) at
scheduling time. That diverged from the HTML spec in two ways: top-level
var/function declarations in the body stayed local to the compiled function
instead of becoming globals, and a syntax error was caught eagerly at
scheduling and turned into a silent no-op (that still returned a timer id)
rather than surfacing when the timer fires.
Defer string handlers to fire-time indirect eval ((0, eval)(src)), which runs
in the true global scope like a classic script, so declarations become global
and a SyntaxError is thrown when the timer elapses. Matches the dynamic-script
path, which already uses indirect eval for the same reason. Function handlers
are unchanged.
Fixes#558
The stealth (wreq) client gated its custom-trust-store build on
`var_os(...).is_some()`, which is true for a set-but-empty variable. Supplying
a store to wreq's `tls_cert_store` REPLACES the bundled webpki roots, so an
empty `SSL_CERT_FILE` (a common shell accident, e.g. `export SSL_CERT_FILE=$UNSET`)
built a near-empty default-paths store and made every HTTPS request fail with
CERTIFICATE_VERIFY_FAILED -- while the reqwest client, which already filters
empty values in configured_root_paths(), kept working. Same input, opposite
behavior.
Extract a shared `custom_cert_store_requested` predicate in client.rs that
applies the same `!is_empty()` filter, and call it from the stealth client's
guard. The predicate takes the values as arguments so it is unit-tested under
default features (the stealth feature pulls in boringssl and is not built in
the normal test job).
Fixes#551
The check/uncheck/select branches of browser_fill_form were left on the
pre-#530 pattern: a direct `el.checked = true` / `el.selectedIndex = i` plus a
raw, untrusted `change` event. For a React-controlled checkbox the direct
assignment runs the framework's value tracker in lockstep, so onChange never
commits and the box reverts; and any handler gating on event.isTrusted ignores
the untrusted events -- the exact bug class #530 fixed for text inputs two
lines away.
Route check/uncheck through __obscura_setFieldValue(el, 'checked', ...) so the
write goes through the prototype setter and leaves the tracker stale, and
dispatch input+change wrapped in __obscura_markTrusted for all three branches,
matching the text-input branch.
Fixes#550
The Event and CustomEvent constructors diverged from Chrome in three
observable ways:
- new Event() / new CustomEvent() with no argument did not throw, though the
type argument is required (Chrome: "1 argument required, but only 0
present."). initEvent/initMouseEvent already had this check; the
constructors did not.
- the type argument was not coerced to a string, so new Event(123).type was
the number 123 instead of "123".
- CustomEvent.detail defaulted to undefined; WebIDL CustomEventInit specifies
`any detail = null`.
Add the required-argument guard to both constructors, coerce type via
String(t) (inherited by every subclass through super), and default detail to
null. createEvent() builds events with an explicit '' type, so the arity
guard leaves the type-"" uninitialized-event path intact.
Fixes#552
CSSStyleDeclaration was added to the _preHideInternals non-enumerable
pre-declaration list (commit c7e7c70) on the assumption that its later
`globalThis.X = X` assignment would fill in the value. But it was the only
one of the 22 WebIDL interfaces with no such assignment, so its value stayed
undefined: `typeof window.CSSStyleDeclaration` was "undefined" while
`'CSSStyleDeclaration' in window` was true, and because it is the type of
element.style / getComputedStyle(), `el.style instanceof CSSStyleDeclaration`
threw "Right-hand side of 'instanceof' is not callable" (Chrome returns true).
Assign globalThis.CSSStyleDeclaration alongside the other interface globals.
The property was already declared writable/configurable, so the assignment
only updates the value and it remains enumerable:false, like a real browser.
Fixes#546
Node::set_attribute looked up the existing attribute by local name alone,
while get_attribute and the remove_attribute op both match by qualified
name. A parsed namespaced attribute is stored with a separate prefix
(xlink:href -> prefix="xlink", local="href"), so setAttribute("xlink:href")
never matched it: the update was pushed as a second attribute, getAttribute
kept returning the stale original, and the element serialized xlink:href
twice.
Match set_attribute by qualified_name_eq, consistent with the read/remove
paths, so the update lands in place and no duplicate is created.
Fixes#547
DOMParser.parseFromString always HTML-parsed, even for XML mime types, so
malformed XML never produced the <parsererror> element Chrome emits and
error-detection code (doc.querySelector('parsererror')) never triggered. obscura
has no XML parser, so full XML semantics stay out of scope; instead, for XML
mime types run a conservative well-formedness check (tag balance, single root,
terminated comment/CDATA/PI, skipping quoted attribute regions) and, on a clear
error, make the document surface a <parsererror>. The check defaults to
well-formed when ambiguous, so valid XML is never falsely flagged.
Fixes#504
* fix: add missing #[ignore] gate and local fixture for concurrent fetch test
* fix(test): make fetch fixture nonblocking
---------
Co-authored-by: SGavrl <277843151+SGavrl@users.noreply.github.com>
