Two structural fixes to the layout and paint model.
Mixed inline+block content: a block holding both was promoted wholesale
to a flex-row-wrap container to approximate the inline formatting
context. That collapsed contentless block children to width 0 (a
position:relative hero wrapper whose only child is an absolute fill
image vanished with it) and let text share lines with sibling blocks.
Now the parent stays a real block, block-level children stay direct
children (auto width fills the containing block), and each maximal run
of consecutive inline-level siblings becomes one anonymous block-level
box carrying the IFC. A run of pure text and foldable inline wrappers
collapses to a single shaped cosmic-text leaf (one taffy node per run
instead of one per word); runs with atomic inline boxes fall back to a
flex-wrap wrapper. Absolutely positioned children neither join nor
break runs and no longer count as inline content, so their containing
block keeps its real used width and taffy resolves width:100%/inset:0
fills against it.
Overflow clips: an inherited overflow:hidden clip was shifted by the
painted node's own accumulated translate, so a carousel track's
translateX dragged the viewport's clip along with every slide and all
slides stayed visible stacked. Clips are now stored in the clip owner's
coordinate space (owner translate baked in at resolve time) and no
longer move with descendants, for boxes, per-word text, and shaped
inline items alike. The same walk records per-node accumulated
translates once, replacing the per-painted-node ancestor walks. Also
parse translate3d (Swiper-style carousels position their track
exclusively through it) and make transform:none reset a lower-priority
transform.
Verified: abs-fill repros that rendered 0x0 now fill across
block/flex/grid parents; carousel repro shows exactly one slide;
danluu.com mismatch vs Chromium 13.0 to 5.7 percent; no regressions
across the prose and framework spot sets; obstacle course 33/33; 36
percent faster on a layout-bound synthetic page with neutral real-page
memory.
Skip <link rel=stylesheet> sheets that are disabled or gated to
print/dark, and resolve @import chains inside fetched sheets so the
imported CSS reaches the cascade.
- disabled links (JS-toggled alternate themes) no longer paint pages
dark under a light Chromium
- media="(prefers-color-scheme: dark)" link sheets are skipped
- @import url(...) is fetched recursively; Sphinx docs pull their layout
from basic.css via classic.css and it was being dropped
Pixel fidelity vs headless Chromium: docs.python 95.5 to 10.2 percent,
web.dev 91.1 to 10.5, developer.chrome 88.0 to 18.9. No regressions on
the good-site set; obstacle course still 33/33.
A direct child of a grid container is blockified per CSS Display 3, so an
inline <a>/<time>/<span> becomes its own block-level grid item instead of
being flattened into loose words in the container (which shattered grid
lists like jvns.ca's .article-list of alternating <time>/<a> into one word
per cell). Grid only, not flex, since flex-column is also our block
stand-in for <td>.
An <img src="//host/path"> (ubiquitous on Wikipedia and CDN-hosted media)
now inherits the document scheme, defaulting to https for non-network bases
so a file:// page does not turn it into file://host. Previously url::join
against a file base produced an unfetchable URL.
Chrome's default sans-serif on this class of host is Liberation Sans, not
DejaVu (measured: sans-serif == Arial == Helvetica == Liberation width,
DejaVu ~12% wider). Forcing DejaVu drifted all text wider and shifted
every line. Bundle Liberation Sans/Serif/Mono (DejaVu kept as glyph
fallback), default to Liberation Sans, and resolve font-family to the
right generic (monospace/code -> Mono, serif -> Serif, else -> Sans).
A controlled text page drops from ~15-20% to 3.5% pixel mismatch and
every line lands within 1px of Chrome.
percentage padding/margin: top/bottom percentages resolve against the
containing-block width (the responsive aspect-ratio-box trick, e.g.
padding-top:56.25%), resolved to px in the top-down layout pass so card
media boxes reserve space instead of collapsing.
box-shadow: parse and paint an outset shadow (rounded, offset, spread,
soft-edge approximation) so cards/buttons/menus have depth.
external-sprite svg: resolve <use href="url#id"> to an external sprite
file, inject the referenced symbol so inline-svg icon systems rasterize.
background-clip:text: paint transparent-with-gradient text (the common
gradient hero/button label) instead of leaving it invisible.
A <link rel=stylesheet media="print"> must not apply on screen; fetching
and injecting it let print rules override the screen layout (e.g. the
Guardian's print sheet). Skip links whose media is print-only.
object-fit: contain/cover/scale-down/none for <img> so an image whose
aspect ratio differs from its box is fit (letterbox/crop) instead of
stretched.
transform: parse translate()/translateX/translateY (and scale); apply
translate as a paint-time offset to the element and its subtree, so
translate(-50%,-50%) centering and off-screen skip-links land correctly.
inline-svg: serialize and rasterize inline <svg> subtrees via resvg so
icon sprites and brand logos paint instead of showing blank.
Colors: oklch()/oklab() and color-mix(); linear-gradient backgrounds.
Grid: named-line placement, repeat() expansion, drop whitespace-only
grid items; upgrade taffy 0.7 -> 0.12 so aspect-ratio images in
auto-height rows no longer collapse to slivers.
CSS: @layer, nesting, @property, :is()/:where(), clamp(), @media not all;
source/track/param/area and svg desc/metadata as display:none.
Images: webp + intrinsic sizing, srcset/picture, browser UA + Accept for
CDN fetches, grey placeholder on failure; width/height HTML attributes
rank below author CSS with aspect-ratio preserved for auto axes.
A whole DOM text node was one indivisible taffy leaf. Flex-wrap only
breaks between items, never inside one, so a long run of plain text
with no inline element breaking it up (an ordinary sentence with no
link in it) either fit on one line or ran straight past the
container's edge instead of wrapping. Short runs interrupted by links
every few words (extremely common in Wikipedia's markup) happened to
be narrow enough to wrap on their own, which is why paragraphs looked
fine but any long plain-text stretch overflowed straight over the
infobox and the History-section image.
Text nodes now split into one taffy leaf per word, flattened into the
same child list as sibling elements so words from different text nodes
and inline elements interleave and wrap together like real inline
flow. This changes the geometry data model: a text node's content is
now a list of (box, word) pairs (DomLayout::text_runs) instead of a
single box, since it can span several lines. Updated painting to draw
each word at its own position, and fixed a bold-text measurement
mismatch (layout width and drawn width disagreed by 1px/char, invisible
before as one blob, visible as gaps between every word once each word
is independently positioned).
Added a regression test that fails against the old single-leaf
behavior.
These were bundled in with the truly dynamic, interaction-only
pseudo-classes (:hover/:active/:focus) and always resolved to false.
That's right for :hover et al (there's no live cursor against a static
snapshot) but wrong for :enabled/:disabled/:checked: those reflect
real, static DOM state (the disabled/checked attributes), not user
interaction, and modern component systems (Codex, Material, Bootstrap)
lean on :enabled constantly for their base button/input styling. With
it always false, every such base rule was silently inert, which is why
the Wikipedia TOC toggle button (a <button> with no disabled
attribute, so :enabled should be true) kept its default black border
instead of the transparent one its :enabled.cdx-button--weight-quiet
rule sets.
:enabled now matches a form control (input/button/select/textarea/
optgroup/option/fieldset) without a disabled attribute, :disabled the
opposite, and :checked matches the checked/selected attribute.
Icons were rendering as either overlapping solid squares or not at all,
text ran under the infobox instead of beside it, and general.css parsing
had two more silent-corruption bugs on top of the ones fixed earlier:
- A CSS declaration list was split on every literal semicolon, so any
value containing one inside a quoted string or url() (a data: URI
with a MIME parameter, e.g. url(data:image/svg+xml;utf8,<svg...>),
extremely common for small inline icon SVGs) got torn into two
malformed halves and silently dropped. This is why mask-image never
applied to the TOC toggle icons at all.
- data: URIs were only decoded as base64. Inline SVGs are just as
commonly written as data:image/svg+xml;utf8,<percent-escaped text>,
which needs percent-decoding instead; added that path.
- top/left/right/bottom/width/height only handled bare lengths and a
flat calc(a +/- b). Real values nest var(--x, fallback), calc(), and
min()/max() inside each other (Wikipedia's icon offset math is
literally calc(-1 * 22px / 2), and its icon sizing is
calc(max(calc(var(--font-size-medium,1rem) + 4px),10px))). Replaced
the flat evaluator with a small recursive resolver that handles all
three, nested arbitrarily. This is what was placing the TOC toggle
button's icon on top of the section text instead of in the left
margin where it belongs.
Added actual feature support once the parsing was correct:
- mask-image/-webkit-mask-image: the pattern behind most scalable
colored icons on the modern web (an SVG or image used purely as an
alpha stencil, tinted by background-color/color). Previously every
masked icon rendered as an unmasked solid-color square.
- float: left/right, approximated without a full per-line reflow
engine (which taffy's block/flex/grid modes don't provide): a
floated element is grouped into a row alongside the flow siblings
that follow it, up to whichever comes first of the next heading or
another float, then everything from there on reverts to normal
full-width flow. This isn't general CSS float, but it directly
targets the overwhelmingly common real shape (an infobox or image
floated near the top of an article, with the rest of the content
starting at the next section heading), and is what makes article
text actually wrap beside an infobox instead of running underneath
it.
Refactored paint.rs's image fetch/decode/cache logic into shared
helpers (fetch_bytes, raster_to_pixmap) reused by plain <img> painting,
background-image, and the new mask-image path.
The render layer had no CSS Grid support, so Vector 2022's page layout
(three nested grids) collapsed to a vertical stack with the article
landing tens of thousands of pixels down the page. Wikipedia rendered
as a near-blank page and Hacker News only worked through per-site
hardcoded CSS overrides.
Removed the HN-specific hardcoding and fixed the general engine instead:
- CSS Grid layout (grid-template-areas/columns/rows via taffy's grid mode)
- Indexed, specificity-correct cascade in a new css.rs, replacing the
old test-every-selector-against-every-node scan. Cascade time on
Wikipedia went from 9.5s to under 200ms.
- Ancestor bloom-filter fast-rejection for selector matching in
obscura-dom, needed to make the indexed cascade actually fast on
descendant combinators.
- :link/:visited pseudo-classes were previously unparseable, so any
rule using them (a:link{color:...}, extremely common) silently
dropped the whole rule from the cascade.
- overflow:hidden clipping, fixing the pervasive visually-hidden-but-
accessible pattern used for skip links and screen-reader-only labels.
- @media parser robustness: it did a literal substring search for
max-width: and silently dropped the constraint on max-width : 750px
(space before colon) or max-width:calc(750px - 1px), treating
mobile-only or narrow-viewport rules as always active. The calc()
case was hiding Wikipedia's TOC sidebar via a rule meant only for
narrow screens.
- Table auto-layout approximation: the trailing auto-width td in a row
absorbs leftover space, so a width:100% table's unsized last column
reaches the right edge instead of shrinking to content.
- Inline formatting context approximation: a block holding inline-level
children (a paragraph or list item with links) is promoted to a
wrapping flex row so text and links read as continuous prose instead
of one fragment per line.
- Whitespace-only text nodes no longer get a full line-height box. They
used to stack up as blank rows, turning Wikipedia's collapsed TOC
subsections and the page header into hundreds of pixels of dead space.
- SVG rendering via resvg/usvg, plus resolving relative image URLs
against the page's base URL.
- background-image support including background-size/background-position
for correctly sized and positioned icons.
- Per-screenshot image fetch cache.
- Fixed the layout-only (no paint feature) build, which didn't compile
due to an unconditional call into paint-gated code.
Documents the obscura-render layer: how to enable it (--features render),
the CLI --screenshot and CDP Page.captureScreenshot paths, what it
renders, the architecture, and a head-to-head benchmark.
Benchmark vs headless Chrome (same page, 1280x720, warmed):
- local fixture: obscura ~0.06s/33MB vs chrome ~0.90s/195MB
- example.com: obscura ~0.10s/34MB vs chrome ~0.98s/199MB
Roughly 10-15x faster and ~6x less memory. Chrome's larger PNG on the
real page reflects that it paints text, which obscura does not yet.
End-to-end rasterization behind the render feature (which now enables
obscura-render/paint):
- obscura-render: screenshot_png(dom, viewport) paints and encodes to PNG
via tiny-skia's encode_png (no extra dep).
- obscura-browser: Page::screenshot renders the live DOM at a viewport.
- CLI: obscura fetch --screenshot out.png writes the settled page to a PNG.
- CDP: Page.captureScreenshot returns the PNG base64-encoded, so
Puppeteer/Playwright page.screenshot() works. captureSnapshot still
errors (it is a DOM/layer snapshot, not an image).
Verified: both paths produce correct 1280x720 RGBA PNGs with elements
laid out and colored as the CSS specifies (block stacking, backgrounds,
white page). Default build unchanged; obstacle course 33/33.
Text, borders, and images are not yet painted; the box layout and
background colors are rendered.
Adds a paint feature to obscura-render (tiny-skia: pure Rust, CPU,
deterministic, no system deps). paint_dom lays out the DOM and fills
each element's border box with its parsed background-color over a white
page, in tree order so descendants paint over ancestors.
Style parsing now also extracts background-color (hex #rgb/#rgba/
later enhancements.
getBoundingClientRect, offsetWidth/Height, and offsetTop/Left now read
real box geometry from the obscura-render layout cache when the render
feature is compiled in. elementFromPoint inherits correct hit-testing
because it already delegates to getBoundingClientRect per element.
Adds a render feature on obscura-js (propagated through obscura-browser
and obscura-cli), a lazily-computed layout cache and viewport in
ObscuraState (feature-gated, cleared on navigation), and an
op_layout_geometry op. bootstrap.js probes the op with typeof, so the
default scraping build is unchanged and falls through to the synthetic
rect.
With --features render, block siblings stack and flex children place
horizontally at correct positions, and elementFromPoint resolves the
right element. Default build returns the synthetic rect unchanged.
Obstacle course 33/33; obscura-js unit failures identical to main.
Walk a live DomTree, compute each element's style (tag UA defaults plus
inline style), build a taffy layout tree, and return border-box geometry
keyed by NodeId. Block/flex structure, explicit sizes, and the box model
are correct; text nodes do not yet contribute to size (inline/text
layout lands with the paint phase).
This completes the layout core: a real DOM now produces real geometry,
pure Rust, behind the opt-in render crate. JS exposure is next.
Parse inline style attributes into the layout-relevant subset of
LayoutStyle: display, width/height, margin, padding, and border width,
including box shorthands (1-4 values) and per-side longhands. !important
is stripped; auto and percentages leave the size unset for phase 1.
Layered on a small built-in UA default sheet. A compact hand-rolled
tokenizer handles inline attributes without a cssparser dependency; a
real cascade can harden it later.
New optional crate obscura-render, the scoped render layer for Obscura.
Phase 1 adds the layout core: a LayoutNode tree plus a viewport is laid
out with taffy (pure Rust) and border-box geometry is returned per node,
mirroring the input tree.
The crate is a workspace member but not yet a dependency of any other
crate, so the default scraping build is unchanged. Geometry-from-DOM,
JS integration, text, paint, and screenshot capture land in later
phases.
Unit tests cover block stacking, flex row layout, and padding.
The wreq dependency was declared without the gzip, brotli, deflate and zstd
features, so the stealth client returned bodies still compressed while the
reqwest path decoded them. The Chrome145 emulation advertises those encodings,
so origins do compress, and the raw gzip bytes reached the HTML parser as
document text.
Add a regression test that serves one gzip response and asserts the stealth
client decodes it.
Fixes#563.
Setting innerHTML on the <html> element parses the fragment in its own context
(added in a2fd4d1), which runs html5ever's "before head" insertion mode and
synthesizes both <head> and <body> under the fragment root. The importer's
helper walked into the root <html> and, if it found a <body> child, returned
that body -- so import_children_from imported only the body's children and any
assigned <head> content (<title>, <meta>, <link>) was silently dropped.
A <body> child of the fragment root only appears in this <html>-context case;
every other element context leaves parsed content directly under the root and
already fell through to returning it. Rename find_body_or_root -> fragment_root
and return the root <html> element so both head and body children import.
Behavior for all non-<html> contexts is unchanged.
Fixes#556
A string setTimeout/setInterval handler was compiled with new Function(fn) at
scheduling time. That diverged from the HTML spec in two ways: top-level
var/function declarations in the body stayed local to the compiled function
instead of becoming globals, and a syntax error was caught eagerly at
scheduling and turned into a silent no-op (that still returned a timer id)
rather than surfacing when the timer fires.
Defer string handlers to fire-time indirect eval ((0, eval)(src)), which runs
in the true global scope like a classic script, so declarations become global
and a SyntaxError is thrown when the timer elapses. Matches the dynamic-script
path, which already uses indirect eval for the same reason. Function handlers
are unchanged.
Fixes#558
The stealth (wreq) client gated its custom-trust-store build on
`var_os(...).is_some()`, which is true for a set-but-empty variable. Supplying
a store to wreq's `tls_cert_store` REPLACES the bundled webpki roots, so an
empty `SSL_CERT_FILE` (a common shell accident, e.g. `export SSL_CERT_FILE=$UNSET`)
built a near-empty default-paths store and made every HTTPS request fail with
CERTIFICATE_VERIFY_FAILED -- while the reqwest client, which already filters
empty values in configured_root_paths(), kept working. Same input, opposite
behavior.
Extract a shared `custom_cert_store_requested` predicate in client.rs that
applies the same `!is_empty()` filter, and call it from the stealth client's
guard. The predicate takes the values as arguments so it is unit-tested under
default features (the stealth feature pulls in boringssl and is not built in
the normal test job).
Fixes#551
The check/uncheck/select branches of browser_fill_form were left on the
pre-#530 pattern: a direct `el.checked = true` / `el.selectedIndex = i` plus a
raw, untrusted `change` event. For a React-controlled checkbox the direct
assignment runs the framework's value tracker in lockstep, so onChange never
commits and the box reverts; and any handler gating on event.isTrusted ignores
the untrusted events -- the exact bug class #530 fixed for text inputs two
lines away.
Route check/uncheck through __obscura_setFieldValue(el, 'checked', ...) so the
write goes through the prototype setter and leaves the tracker stale, and
dispatch input+change wrapped in __obscura_markTrusted for all three branches,
matching the text-input branch.
Fixes#550
