5225f6d985a361c65ff8f9fe734ac64f97423a39
11
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 SGavrl
|
32ddf93885 |
Fix/engine hardening (#249)
* fix(dom): read insert_before prev_sibling after detach to avoid a sibling cycle insert_before captured the reference node's prev_sibling BEFORE detaching the node being inserted. When page JS inserts a node before its own immediate previous sibling (a no-op reorder frameworks do constantly), detach moves that pointer, so the stale prev_id spliced new.next_sibling = new. The resulting sibling self-cycle made descendants() inner next_sibling walk loop forever in native Rust, uninterruptible by tokio or the V8 watchdog. This hung ebay.com, found by a 1500-page crash/hang sweep. Read prev_sibling after detach so it reflects the post-detach tree. Cap the inner sibling-collection loops in descendants() and log when a descendants cap fires, as defense in depth so a sibling cycle from any path cannot hang the engine. Add test_insert_before_previous_sibling_no_cycle. ebay.com now renders (192K chars) instead of hanging. Verified: 33/33 obscura-dom tests, obstacle course 33/33. * fix(cli): settle async --eval work before reading the page (#248) --eval ran after the selector check and returned its value immediately, so an expression that only kicks off async work (e.g. fetch().then(t => DOM write)) returned the still-pending Promise as {} and any --selector/--dump were ignored. When --eval is combined with --selector and/or --dump, run the eval to kick off its async work, settle the event loop again so it completes, then read the page (selector wait + dump) instead of returning the Promise. A bare --eval still returns its value unchanged, so synchronous expressions are not affected, and the eval stays bounded by evaluate_with_timeout. dump becomes Option<DumpFormat> so an explicit --dump is distinguishable from the default. Verified: the repro now returns the fetched content, a plain JSON.stringify eval is unchanged, 37 + 16 cli tests pass. |
||
|
SGavrl
|
1e9e9478fb |
Fix/v8 watchdog panic safety (#239)
* dom: complete detached documents; add ProgressEvent and parseHTMLUnsafe
createHTMLDocument returned a document whose body was null: parsing the
skeleton string as innerHTML of <html> collapsed through the fragment parser
and kept only <title>. Build head>title and body explicitly instead. The
DOMParser doc node also gains appendChild/removeChild/insertBefore and a
doctype getter, and createDocument carries its doctype argument.
This lets dom/common.js setupRangeTests() finish (it builds fixtures with
new Document(), createHTMLDocument(), createDocument()) instead of aborting at
harness setup, so the dom/ranges files run. Range-collapse 186/186,
Range-attributes 1/1, Range-cloneRange 55/62, StaticRange-constructor 14/17.
The large O(ranges x points) matrix files (Range-set, comparePoint,
compareBoundaryPoints) still time out: the boundary-point comparison walks the
tree through per-step DOM ops, which is too slow for those matrices. Speeding
that up (a Rust node-position op) is a separate change.
Also add globalThis.ProgressEvent (Event subclass with lengthComputable/
loaded/total) and the static Document.parseHTMLUnsafe(html).
* perf(dom): compute Range tree-order in Rust ops
The Range boundary-point algorithms compared nodes by walking ancestor chains
and sibling lists from JS, one DOM op per step. The dom/ranges matrices
(testRanges x testPoints, thousands of comparisons) made that time out.
Add node_index, compare_order, and node_root ops that do the walk in Rust over
the DOM tree (slab lookups, no per-step FFI), and use them in _rngNodeIndex,
_rngOrder, and _rngRoot. Range-comparePoint 3534/5580, Range-isPointInRange
5197/5733, Range-compareBoundaryPoints 6441/9313, all completing in under 10s
where they previously did not finish. Not on any engine hot path (querySelector,
mutation, navigation, fetch do not use these).
* fileapi: real byte-backed Blob/File, FileReader state machine, Request.blob
Blob was a one-line stub that string-joined its parts (wrong size for binary
or multibyte input) and exposed only text(). Replace it with a Uint8Array-backed
Blob: normalizes string/ArrayBuffer/typed-array/Blob parts, correct size,
slice(), text()/arrayBuffer()/bytes(), type normalization (lowercased, cleared
when non-ASCII), the endings option (native CRLF normalization), and
Symbol.toStringTag. File adds name/lastModified, the 2-argument requirement, and
its own toStringTag.
FileReader becomes a real state machine: EMPTY/LOADING/DONE, throws
InvalidStateError when a read starts while already LOADING, and fires
loadstart/progress/load/loadend (and abort) to both on* handlers and
addEventListener. readAsText/DataURL/ArrayBuffer/BinaryString produce correct
results. Request gains blob().
FileReader-multiple-reads 6/6, File-constructor-endings 10/11, Blob-slice
141/150.
* encoding: TextDecoder supports legacy charsets via encoding_rs
TextDecoder was a UTF-8-only JS shim that ignored its label, so
new TextDecoder('shift_jis'|'euc-jp'|'big5'|'gbk'|'windows-1252'|'iso-8859-1')
.decode() returned garbage. That matters for scraping: pages and scripts decode
legacy-encoded bytes (XHR/fetch responses, File/Blob text) all the time.
Back it with encoding_rs (already used by the net layer for response bodies):
obscura-net gains label_name() and decode_with_label(), exposed to JS as
op_encoding_for_label and op_text_decode. The constructor validates the label
(RangeError) and sets the canonical .encoding/.fatal/.ignoreBOM; decode()
supports fatal mode (TypeError on invalid input) and BOM handling.
Plain UTF-8 non-fatal decoding keeps a pure-JS fast path (no op, no JSON), so
Response.text()/Blob.text() on large bodies are unchanged; the op only runs for
legacy encodings or fatal mode.
textdecoder-labels 222/222, textdecoder-fatal 36/36, byte-order-marks 3/3,
utf16-surrogates 10/10, single-byte-decoder 168/504.
* selection: real Selection over Range (single instance per document)
getSelection() returned a fresh plain object each call, so it was not a Selection
instance, had no identity (window.getSelection() !== document.getSelection()),
and was missing removeRange/empty/collapseToStart/collapseToEnd/setPosition/
setBaseAndExtent/modify while collapse/extend/getRangeAt were broken stubs. That
was ~4862 failing 'selection[method] is not a function' subtests.
Replace it with a real Selection class built on the live Range: one cached
instance per document (window.getSelection() === document.getSelection(), null
for documents with no defaultView), an internal range + direction, derived
anchor/focus/isCollapsed/type/rangeCount, and the full method set
(getRangeAt with IndexSizeError, addRange, removeRange, removeAllRanges/empty,
collapse/setPosition, collapseToStart/End, extend, setBaseAndExtent,
selectAllChildren, containsNode, toString) with the spec DOMException names.
modify() stays a no-op (needs visual layout). No ops.rs changes; not on any hot
path.
setBaseAndExtent 120/120, removeAllRanges 104/104, collapseToStartEnd 51/51,
type 26/26, extend-exception 1/1.
* forms: HTMLInputElement valueAsNumber/valueAsDate/stepUp/stepDown
These were absent. Implement the number/date conversion API for the applicable
types (number, range, date, month, week, time, datetime-local): parsers and
formatters per type (Date.UTC for date/datetime-local, ISO-week Monday math for
week, months-since-epoch for month), valueAsNumber get/set, valueAsDate get/set,
and stepUp/stepDown(n) with the step scale/default per type, step-base snapping,
and min/max clamping. Non-applicable types and step="any" throw InvalidStateError;
non-finite valueAsNumber throws TypeError; range with empty value reads as the
midpoint (50 for 0..100). Plus min/max/step attribute reflection.
All on Element accessors gated by input type, only evaluated on explicit access;
not on any hot path.
input-stepup 51/53, input-valueasnumber 60/64, input-valueasdate 27/30.
* perf(encoding): skip label op for no-arg TextDecoder (Response/Blob text path)
new TextDecoder() with no argument (what Response.text()/Blob.text() and most
pages use) is UTF-8; resolve that without the op_encoding_for_label round trip.
Explicit labels still validate via the op.
* core: HTMLDialogElement, Node.normalize, Range.createContextualFragment
HTMLDialogElement was a bare alias of Element, so ~40 Core files crashed at the
first dialog.show()/showModal(). Add the dialog members on Element.prototype
(matching the popover/input convention): open reflection, returnValue (internal
slot, String coercion), oncancel/onclose defaulting to null, show()/showModal()
/close()/requestClose() with the open-state transitions, InvalidStateError
throwing (show-on-modal, showModal-on-open, showModal-disconnected), the async
non-bubbling close event, the synchronous cancelable cancel event (no re-entrant
cancel), beforetoggle/toggle, and closedBy. matches(':modal') reports a modally
shown dialog (like :popover-open). Modal top-layer/focus/render stays unhandled
(layout).
Node.normalize() now merges adjacent Text nodes and drops empty ones (was a
no-op). Range.createContextualFragment(html) parses a fragment.
createContextualFragment 34/35, dialog-open 3/3, dialog-return-value 1/1,
dialog-showModal 5/10, dialog-requestclose 22/75. Not on hot paths.
* custom-elements: CustomElementRegistry class, getName, ElementInternals, HTMLUnknownElement
customElements was a bare object literal, so 'CustomElementRegistry is not
defined' crashed ~22 Core files at top level (idlharness/feature-detect), and
HTMLUnknownElement/ElementInternals/attachInternals/getName were missing.
Promote the registry to a real CustomElementRegistry class (window.customElements
is now an instance, so 'define in CustomElementRegistry.prototype' works and the
class is exposed). Add spec define() validation/throw order (TypeError for a
non-constructor, SyntaxError for an invalid name, NotSupportedError for a
duplicate name/constructor or reentrant define), getName(), and a SyntaxError
reject in whenDefined for invalid names. The existing define -> upgrade ->
connectedCallback path (used by lit/Polymer on real pages) is preserved.
Add HTMLUnknownElement, ElementInternals (setFormValue/setValidity/checkValidity/
validity/validationMessage/form/states), and Element.attachInternals() with its
NotSupportedError cases.
Does NOT include the [[HTMLConstructor]] (new.target) construction rework or the
reaction wiring into appendChild/setAttribute (hot paths) - deferred to avoid
element-model and perf risk. getName 4/4.
* shadow-dom: ShadowRoot.adoptedStyleSheets (mirror Document)
ShadowRoot lacked adoptedStyleSheets; constructible-stylesheet adoption into a
shadow root threw. Add the getter/setter on ShadowRoot.prototype, matching
Document.adoptedStyleSheets.
* conformance: charset/URL encoding override, IDL reflection, Range and Storage fixes
Encoding: track the detected document character encoding and expose it as
document.characterSet/charset/inputEncoding instead of hardcoding UTF-8. Add the
WHATWG URL encoding override for <a>/<area> hrefs, so a legacy-charset document
percent-encodes the query in its own encoding (EUC-JP U+8108 becomes %CC%AE).
ASCII delimiters stay literal so structured queries are unaffected, and
unmappable code points become %26%23NNN%3B. UTF-8 documents take an early-return
so the href getter hot path is unchanged. fetch()/XHR text() now decode using
the response Content-Type charset instead of always UTF-8. Adds op_url_encode_query
and a document_encoding op; encoding state is plumbed through Page and the runtime.
Reflection: install the reflected IDL attribute layer once on Element.prototype
(no per-element defineProperty, so element creation and query paths are
untouched): global attributes (title, lang, dir, accessKey, slot, autofocus,
hidden, tabIndex) with correct type coercion and enum canonicalization, plus the
ARIAMixin (aria-* reflected as the ariaXxx nullable DOMString properties).
Range: compareBoundaryPoints ToUint16-converts the how argument (NaN/Infinity
no longer false-throw) and raises a real WrongDocumentError DOMException for
ranges that do not share a root.
Storage: reimplement localStorage/sessionStorage as a WHATWG Proxy so property
access (localStorage.foo = x, delete, in, Object.keys) stays in sync with
length, key() and iteration.
* conformance: HTML attribute folding, compareDocumentPosition, URLSearchParams, DOMParser, fetch UA
Next WPT Core batch, all verified live and benchmarked for no hot-path regression.
DOM:
- compareDocumentPosition rewritten to spec: real PRECEDING/FOLLOWING via the
compare_order op, CONTAINS/CONTAINED_BY, and DISCONNECTED|IMPLEMENTATION_SPECIFIC
for different roots. Adds the DOCUMENT_POSITION_* and missing nodeType constants
on Node and its prototype. WPT getPosition() depends on this, so it cascades into
the Range comparePoint/compareBoundaryPoints/isPointInRange expectations.
- HTML elements now ASCII-lowercase attribute names in getAttribute/setAttribute/
removeAttribute/hasAttribute, so el.setAttribute('accessKey') reflects via
getAttribute('accesskey'). The fold is gated behind a cheap uppercase scan and a
namespaceURI HTML guard (SVG/foreign attributes keep their case), and getAttribute
only folds on a miss, so the all-lowercase hot path is unchanged (benchmarked flat).
- Range comparePoint/isPointInRange ToUint32-coerce the offset (negative throws
IndexSizeError); createTreeWalker/createNodeIterator keep an explicit whatToShow of 0.
URL:
- URLSearchParams uses a proper form-urlencoded byte decoder (partial/invalid escapes
stay literal instead of decodeURIComponent throwing on the whole string), 2-arg
has(name,value)/delete(name,value), and a sequence-pair TypeError. Adds URL.parse
and URL.canParse statics.
Other:
- fetch()/XHR send a default Chrome User-Agent (the navigation path set one but
scripted requests went out with none, so UA-gated servers rejected them); honors an
explicit override.
- DOMParser documents expose URL/compatMode/characterSet/readyState/styleSheets/
defaultView and createTreeWalker/createNodeIterator; XMLDocument is defined globally.
* perf(dom): O(1) querySelector('#id') via the id index
querySelector('#id') went through the full selector engine, scanning descendants
in tree order until the first match (O(n), ~68us per call over a 2000-element
document in the micro-bench). A bare ASCII id selector now resolves through the
existing id index in O(1): ~68us -> ~1.25us per call, about 55x faster, with no
change to any other path (create/getAttribute/setAttribute/querySelectorAll/href
all flat). querySelector by id is one of the hottest scraping operations.
The fast path is conservative: only a bare `#name` selector with an ASCII
letter/underscore-led identifier takes it; escapes, combinators, commas,
whitespace, compounds, and non-ASCII fall through to the full engine unchanged.
For a scoped element.querySelector it returns the indexed element only when it is
a strict descendant of the scope, and falls back to the scan otherwise, so
duplicate-id scoped queries stay correct.
Also makes the id index keep the FIRST element in tree order per id (or_insert at
parse time) instead of the last, so getElementById and querySelector('#id') return
the first-in-tree-order element on duplicate ids, per spec (previously last).
* fix: bound synchronous V8 hangs with a termination watchdog and add op panic safety
Heavy pages could hang obscura fetch indefinitely: tokio::time::timeout only
cancels at await points, so synchronous V8 work (a page's script bootstrap, a
microtask storm) pinned the thread and ran unbounded.
Add a reusable V8 terminate_execution watchdog (arm_watchdog/disarm_watchdog) and
apply it to every place that runs synchronous V8 work: the post-load settle, both
navigation event-loop pumps, the whole script-execution phase, and --eval. A CLI
process-level hard deadline backstops the rest, since a hang inside a Rust op
cannot be cancelled by tokio or terminate_execution.
Also harden against panics: op_dom is wrapped in catch_unwind so a DOM-op panic
degrades to a null result instead of unwinding into V8's FFI frame and aborting
the process; the append_text unwrap is removed; panic = unwind is pinned in the
release profile.
Verified: obstacle course 33/33, no perf regression (per-op deltas within the
measurement noise floor), 6 of 8 previously-hanging real sites now render and the
other 2 exit deterministically at the hard deadline.
* fix(dom): reject cyclic reparenting so descendants() cannot hang the engine
appendChild/insertBefore only guarded the self-append case (parent == child),
never the case where the moved node is an ancestor of the new parent. When page
JS does node.appendChild(ancestorOfNode) the parent/child graph becomes cyclic;
the next querySelector runs descendants() (a DFS with no visited set) and loops
forever, growing a Vec unbounded and pinning the thread in native Rust where
neither tokio nor the V8 watchdog can interrupt it. This hung techcrunch and
kayak.
Reject the cycle at the mutation site per the DOM HierarchyRequestError rule: if
the moved node is an inclusive ancestor of its target parent, treat the move as a
no-op. The walk is gated on the moved node having children, so a fresh/leaf
append (the common case) stays O(1); only relocating a populated subtree pays
O(depth). descendants() also gets a result-length cap as defense in depth so a
cycle from any future path cannot wedge the engine.
techcrunch and kayak now render (37K and 1.26M chars). Verified: new
test_reparent_cycle_is_rejected, obstacle course 33/33, no perf regression.
|
||
|
SGavrl
|
a703b7ac30 |
WPT conformance + perf: O(1) querySelector('#id'), HTML attribute folding, compareDocumentPosition, URLSearchParams, DOMParser (#238)
* dom: complete detached documents; add ProgressEvent and parseHTMLUnsafe
createHTMLDocument returned a document whose body was null: parsing the
skeleton string as innerHTML of <html> collapsed through the fragment parser
and kept only <title>. Build head>title and body explicitly instead. The
DOMParser doc node also gains appendChild/removeChild/insertBefore and a
doctype getter, and createDocument carries its doctype argument.
This lets dom/common.js setupRangeTests() finish (it builds fixtures with
new Document(), createHTMLDocument(), createDocument()) instead of aborting at
harness setup, so the dom/ranges files run. Range-collapse 186/186,
Range-attributes 1/1, Range-cloneRange 55/62, StaticRange-constructor 14/17.
The large O(ranges x points) matrix files (Range-set, comparePoint,
compareBoundaryPoints) still time out: the boundary-point comparison walks the
tree through per-step DOM ops, which is too slow for those matrices. Speeding
that up (a Rust node-position op) is a separate change.
Also add globalThis.ProgressEvent (Event subclass with lengthComputable/
loaded/total) and the static Document.parseHTMLUnsafe(html).
* perf(dom): compute Range tree-order in Rust ops
The Range boundary-point algorithms compared nodes by walking ancestor chains
and sibling lists from JS, one DOM op per step. The dom/ranges matrices
(testRanges x testPoints, thousands of comparisons) made that time out.
Add node_index, compare_order, and node_root ops that do the walk in Rust over
the DOM tree (slab lookups, no per-step FFI), and use them in _rngNodeIndex,
_rngOrder, and _rngRoot. Range-comparePoint 3534/5580, Range-isPointInRange
5197/5733, Range-compareBoundaryPoints 6441/9313, all completing in under 10s
where they previously did not finish. Not on any engine hot path (querySelector,
mutation, navigation, fetch do not use these).
* fileapi: real byte-backed Blob/File, FileReader state machine, Request.blob
Blob was a one-line stub that string-joined its parts (wrong size for binary
or multibyte input) and exposed only text(). Replace it with a Uint8Array-backed
Blob: normalizes string/ArrayBuffer/typed-array/Blob parts, correct size,
slice(), text()/arrayBuffer()/bytes(), type normalization (lowercased, cleared
when non-ASCII), the endings option (native CRLF normalization), and
Symbol.toStringTag. File adds name/lastModified, the 2-argument requirement, and
its own toStringTag.
FileReader becomes a real state machine: EMPTY/LOADING/DONE, throws
InvalidStateError when a read starts while already LOADING, and fires
loadstart/progress/load/loadend (and abort) to both on* handlers and
addEventListener. readAsText/DataURL/ArrayBuffer/BinaryString produce correct
results. Request gains blob().
FileReader-multiple-reads 6/6, File-constructor-endings 10/11, Blob-slice
141/150.
* encoding: TextDecoder supports legacy charsets via encoding_rs
TextDecoder was a UTF-8-only JS shim that ignored its label, so
new TextDecoder('shift_jis'|'euc-jp'|'big5'|'gbk'|'windows-1252'|'iso-8859-1')
.decode() returned garbage. That matters for scraping: pages and scripts decode
legacy-encoded bytes (XHR/fetch responses, File/Blob text) all the time.
Back it with encoding_rs (already used by the net layer for response bodies):
obscura-net gains label_name() and decode_with_label(), exposed to JS as
op_encoding_for_label and op_text_decode. The constructor validates the label
(RangeError) and sets the canonical .encoding/.fatal/.ignoreBOM; decode()
supports fatal mode (TypeError on invalid input) and BOM handling.
Plain UTF-8 non-fatal decoding keeps a pure-JS fast path (no op, no JSON), so
Response.text()/Blob.text() on large bodies are unchanged; the op only runs for
legacy encodings or fatal mode.
textdecoder-labels 222/222, textdecoder-fatal 36/36, byte-order-marks 3/3,
utf16-surrogates 10/10, single-byte-decoder 168/504.
* selection: real Selection over Range (single instance per document)
getSelection() returned a fresh plain object each call, so it was not a Selection
instance, had no identity (window.getSelection() !== document.getSelection()),
and was missing removeRange/empty/collapseToStart/collapseToEnd/setPosition/
setBaseAndExtent/modify while collapse/extend/getRangeAt were broken stubs. That
was ~4862 failing 'selection[method] is not a function' subtests.
Replace it with a real Selection class built on the live Range: one cached
instance per document (window.getSelection() === document.getSelection(), null
for documents with no defaultView), an internal range + direction, derived
anchor/focus/isCollapsed/type/rangeCount, and the full method set
(getRangeAt with IndexSizeError, addRange, removeRange, removeAllRanges/empty,
collapse/setPosition, collapseToStart/End, extend, setBaseAndExtent,
selectAllChildren, containsNode, toString) with the spec DOMException names.
modify() stays a no-op (needs visual layout). No ops.rs changes; not on any hot
path.
setBaseAndExtent 120/120, removeAllRanges 104/104, collapseToStartEnd 51/51,
type 26/26, extend-exception 1/1.
* forms: HTMLInputElement valueAsNumber/valueAsDate/stepUp/stepDown
These were absent. Implement the number/date conversion API for the applicable
types (number, range, date, month, week, time, datetime-local): parsers and
formatters per type (Date.UTC for date/datetime-local, ISO-week Monday math for
week, months-since-epoch for month), valueAsNumber get/set, valueAsDate get/set,
and stepUp/stepDown(n) with the step scale/default per type, step-base snapping,
and min/max clamping. Non-applicable types and step="any" throw InvalidStateError;
non-finite valueAsNumber throws TypeError; range with empty value reads as the
midpoint (50 for 0..100). Plus min/max/step attribute reflection.
All on Element accessors gated by input type, only evaluated on explicit access;
not on any hot path.
input-stepup 51/53, input-valueasnumber 60/64, input-valueasdate 27/30.
* perf(encoding): skip label op for no-arg TextDecoder (Response/Blob text path)
new TextDecoder() with no argument (what Response.text()/Blob.text() and most
pages use) is UTF-8; resolve that without the op_encoding_for_label round trip.
Explicit labels still validate via the op.
* core: HTMLDialogElement, Node.normalize, Range.createContextualFragment
HTMLDialogElement was a bare alias of Element, so ~40 Core files crashed at the
first dialog.show()/showModal(). Add the dialog members on Element.prototype
(matching the popover/input convention): open reflection, returnValue (internal
slot, String coercion), oncancel/onclose defaulting to null, show()/showModal()
/close()/requestClose() with the open-state transitions, InvalidStateError
throwing (show-on-modal, showModal-on-open, showModal-disconnected), the async
non-bubbling close event, the synchronous cancelable cancel event (no re-entrant
cancel), beforetoggle/toggle, and closedBy. matches(':modal') reports a modally
shown dialog (like :popover-open). Modal top-layer/focus/render stays unhandled
(layout).
Node.normalize() now merges adjacent Text nodes and drops empty ones (was a
no-op). Range.createContextualFragment(html) parses a fragment.
createContextualFragment 34/35, dialog-open 3/3, dialog-return-value 1/1,
dialog-showModal 5/10, dialog-requestclose 22/75. Not on hot paths.
* custom-elements: CustomElementRegistry class, getName, ElementInternals, HTMLUnknownElement
customElements was a bare object literal, so 'CustomElementRegistry is not
defined' crashed ~22 Core files at top level (idlharness/feature-detect), and
HTMLUnknownElement/ElementInternals/attachInternals/getName were missing.
Promote the registry to a real CustomElementRegistry class (window.customElements
is now an instance, so 'define in CustomElementRegistry.prototype' works and the
class is exposed). Add spec define() validation/throw order (TypeError for a
non-constructor, SyntaxError for an invalid name, NotSupportedError for a
duplicate name/constructor or reentrant define), getName(), and a SyntaxError
reject in whenDefined for invalid names. The existing define -> upgrade ->
connectedCallback path (used by lit/Polymer on real pages) is preserved.
Add HTMLUnknownElement, ElementInternals (setFormValue/setValidity/checkValidity/
validity/validationMessage/form/states), and Element.attachInternals() with its
NotSupportedError cases.
Does NOT include the [[HTMLConstructor]] (new.target) construction rework or the
reaction wiring into appendChild/setAttribute (hot paths) - deferred to avoid
element-model and perf risk. getName 4/4.
* shadow-dom: ShadowRoot.adoptedStyleSheets (mirror Document)
ShadowRoot lacked adoptedStyleSheets; constructible-stylesheet adoption into a
shadow root threw. Add the getter/setter on ShadowRoot.prototype, matching
Document.adoptedStyleSheets.
* conformance: charset/URL encoding override, IDL reflection, Range and Storage fixes
Encoding: track the detected document character encoding and expose it as
document.characterSet/charset/inputEncoding instead of hardcoding UTF-8. Add the
WHATWG URL encoding override for <a>/<area> hrefs, so a legacy-charset document
percent-encodes the query in its own encoding (EUC-JP U+8108 becomes %CC%AE).
ASCII delimiters stay literal so structured queries are unaffected, and
unmappable code points become %26%23NNN%3B. UTF-8 documents take an early-return
so the href getter hot path is unchanged. fetch()/XHR text() now decode using
the response Content-Type charset instead of always UTF-8. Adds op_url_encode_query
and a document_encoding op; encoding state is plumbed through Page and the runtime.
Reflection: install the reflected IDL attribute layer once on Element.prototype
(no per-element defineProperty, so element creation and query paths are
untouched): global attributes (title, lang, dir, accessKey, slot, autofocus,
hidden, tabIndex) with correct type coercion and enum canonicalization, plus the
ARIAMixin (aria-* reflected as the ariaXxx nullable DOMString properties).
Range: compareBoundaryPoints ToUint16-converts the how argument (NaN/Infinity
no longer false-throw) and raises a real WrongDocumentError DOMException for
ranges that do not share a root.
Storage: reimplement localStorage/sessionStorage as a WHATWG Proxy so property
access (localStorage.foo = x, delete, in, Object.keys) stays in sync with
length, key() and iteration.
* conformance: HTML attribute folding, compareDocumentPosition, URLSearchParams, DOMParser, fetch UA
Next WPT Core batch, all verified live and benchmarked for no hot-path regression.
DOM:
- compareDocumentPosition rewritten to spec: real PRECEDING/FOLLOWING via the
compare_order op, CONTAINS/CONTAINED_BY, and DISCONNECTED|IMPLEMENTATION_SPECIFIC
for different roots. Adds the DOCUMENT_POSITION_* and missing nodeType constants
on Node and its prototype. WPT getPosition() depends on this, so it cascades into
the Range comparePoint/compareBoundaryPoints/isPointInRange expectations.
- HTML elements now ASCII-lowercase attribute names in getAttribute/setAttribute/
removeAttribute/hasAttribute, so el.setAttribute('accessKey') reflects via
getAttribute('accesskey'). The fold is gated behind a cheap uppercase scan and a
namespaceURI HTML guard (SVG/foreign attributes keep their case), and getAttribute
only folds on a miss, so the all-lowercase hot path is unchanged (benchmarked flat).
- Range comparePoint/isPointInRange ToUint32-coerce the offset (negative throws
IndexSizeError); createTreeWalker/createNodeIterator keep an explicit whatToShow of 0.
URL:
- URLSearchParams uses a proper form-urlencoded byte decoder (partial/invalid escapes
stay literal instead of decodeURIComponent throwing on the whole string), 2-arg
has(name,value)/delete(name,value), and a sequence-pair TypeError. Adds URL.parse
and URL.canParse statics.
Other:
- fetch()/XHR send a default Chrome User-Agent (the navigation path set one but
scripted requests went out with none, so UA-gated servers rejected them); honors an
explicit override.
- DOMParser documents expose URL/compatMode/characterSet/readyState/styleSheets/
defaultView and createTreeWalker/createNodeIterator; XMLDocument is defined globally.
* perf(dom): O(1) querySelector('#id') via the id index
querySelector('#id') went through the full selector engine, scanning descendants
in tree order until the first match (O(n), ~68us per call over a 2000-element
document in the micro-bench). A bare ASCII id selector now resolves through the
existing id index in O(1): ~68us -> ~1.25us per call, about 55x faster, with no
change to any other path (create/getAttribute/setAttribute/querySelectorAll/href
all flat). querySelector by id is one of the hottest scraping operations.
The fast path is conservative: only a bare `#name` selector with an ASCII
letter/underscore-led identifier takes it; escapes, combinators, commas,
whitespace, compounds, and non-ASCII fall through to the full engine unchanged.
For a scoped element.querySelector it returns the indexed element only when it is
a strict descendant of the scope, and falls back to the scan otherwise, so
duplicate-id scoped queries stay correct.
Also makes the id index keep the FIRST element in tree order per id (or_insert at
parse time) instead of the last, so getElementById and querySelector('#id') return
the first-in-tree-order element on duplicate ids, per spec (previously last).
|
||
|
SGavrl
|
6b3cc1a686 |
Fix/rescue 227 regressions (#232)
* fix: restore regressions introduced by #227 #227 deleted build.rs (reverting #224's version-from-tag feature), removed DOM cycle guards in append_child/insert_before that prevent infinite loops on self-append/self-insert, dropped the CharacterData textContent fast-path, removed PI/Doctype Rust ops, and cut the env_allows_private_network() check from validate_fetch_url. Restored: - crates/obscura-cli/build.rs (version from tag/env) - OBSCURA_BUILD_VERSION in version string and banner - allow_private_network env var propagation in main.rs - append_child/insert_before self-cycle guards in tree.rs - textContent CharacterData fast-path in tree.rs - create_processing_instruction, create_doctype, pi_target, doctype_name, doctype_public_id ops in ops.rs - ProcessingInstruction branch in set_text_content op - env_allows_private_network() check in validate_fetch_url * fix(js): Europe/Berlin timezone, plugins iterator recursion Two bugs in the stealth bootstrap code from #227: - 'Europe/Germany' is not a valid IANA timezone; Intl.DateTimeFormat silently fell back to UTC. Changed to 'Europe/Berlin'. - plugins[Symbol.iterator] called this[Symbol.iterator]() recursively, blowing the stack on Array.from(navigator.plugins). Fixed to bind Array.prototype[Symbol.iterator] directly to the array. |
||
|
Sander Spilleman
|
9ae36764da | feat(stealth): many stealth improvements | ||
|
SGavrl
|
5e23903063 |
fix: DocumentFragment flattening in appendChild / insertBefore / replaceChild (closes #217) (#221)
Heavy-JS sites build subtrees off-DOM in a DocumentFragment then
appendChild the fragment to a real parent. The DOM spec says: move
the fragment's children up and leave the fragment empty. Our shim
was passing the fragment's _nid straight to the Rust op, so the
fragment node itself got grafted instead of its children, and
framework code that read the inserted nodes' properties hit
"Cannot read properties of undefined (reading 'length')" the moment
it expected a real Element but found our fragment wrapper.
That is what was killing Google Shopping and Bing Shopping for
@Tarun-004 — Bing's anti-malware shim, Google's result-row renderer,
and a sticky-container helper (n.Setup / n.InitializeStickyContainer
in the stack) all use the off-DOM-fragment pattern.
Same defect existed in insertBefore and replaceChild — all three
now flatten DocumentFragment per spec.
While there, three smaller defenses that the same kind of page
exposes:
- _nid validation. Passing a non-Node truthy value (e.g. undefined,
a plain object) used to silently call op_dom("append_child",
parent, undefined), which the Rust side parses with unwrap_or(0)
and grafts as a child of the document root.
- Self-append guard mirrors the insert_before self-cycle fix:
appendChild(self) is a no-op rather than corrupting the
sibling-pointer linked list.
- Tree::append_child gains a parent_id == child_id early return as
defense in depth, parallel to the insert_before guard.
Sanity test in eval mode: DocumentFragment with two children appended
to a host now produces "<span>A</span><span>B</span>" and the
fragment is empty; insertBefore with a fragment containing C produces
"<span>C</span><span>A</span><span>B</span>"; self-append and
appendChild(null) no longer throw.
Verified against the actual reproducer: `obscura fetch --dump text
https://www.bing.com/shop?q=test` no longer produces the
TypeError / "Inline script error" / "Timer error" pattern from
the issue.
|
||
|
SGavrl
|
ee10e8ddb5 |
feat(engine): DOM + AbortSignal + storage fixes surfaced by WPT conformance work (#219)
Engine-only changes pulled out of the WPT conformance branch. The tests, runner, and harness vendoring that drove these have moved to a separate benchmark repo (obscura-bench). Real bug fixes: - Tree::insert_before created a self-cycle when the inserted node was its own reference (the spec-conformant `child.after(x)` path that the testharness's ChildNode helpers exercise). The fixup wrote node.prev_sibling = node.next_sibling = own id; every subsequent childNodes / querySelectorAll walk then looped forever and obscura burned RAM until OOM. Per spec the op is a no-op; we now early-return. - Tree::text_content extended to return own data for direct CharacterData access (Text / Comment / ProcessingInstruction). - Element.namespaceURI now reflects the namespace passed to createElementNS instead of collapsing every non-svg namespace to xhtml. Without this, code that creates elements in MathML, custom XML namespaces, or the empty namespace gets misclassified as HTML. - AbortController / AbortSignal: the previous shim was a plain object whose addEventListener was a no-op, so listeners for 'abort' silently never fired and any AbortSignal.timeout / AbortSignal.any chains broke. New impl tracks aborted/reason, has a real addEventListener / removeEventListener / dispatchEvent surface, honors `once`, and implements the static abort / timeout / any factories crawlers use for budgeted fetches. - StorageEvent class added. localStorage / sessionStorage mutations now dispatch 'storage' events on the parent window and every iframe window so cross-frame onstorage listeners fire. - ProcessingInstruction (nodeType 7) and DocumentType (nodeType 10) constructors plus document.createProcessingInstruction and document.implementation.createDocumentType. textContent returns null for Document and DocumentType per spec. - insertAdjacentText / insertAdjacentElement on Element. Without these the testharness output renderer threw in a forEach without try/catch, breaking the completion callback chain for every test with subtests. - Window + Node addEventListener filter non-callable listeners, accept the EventListenerObject shape (object with handleEvent), dedupe by (type, callback, capture), honor the `once` option, and snapshot the listener list before dispatch so a `once: true` removal mid-iteration doesn't corrupt the loop. - Iframe HTML loader now extracts head <script> blocks and body on-handler attributes (`<body onstorage=...>`), evals scripts in the parent realm after the iframe window is built, re-attaches handlers as window listeners. Same iframe element reusing its src no longer double-registers in the iframe registry. - ops.rs validate_fetch_url honors the OBSCURA_ALLOW_PRIVATE_NETWORK env var, and the CLI --allow-private-network flag mirrors itself into that env so the js-side fetch gate matches the http_client gate. Without this, iframe loads of 127.0.0.1 resources were blocked even with the flag set. - ops.rs: create_processing_instruction, create_doctype, pi_target, doctype_name, doctype_public_id ops added; set_text_content extended to handle PI nodes. |
||
|
SGavrl
|
fd2421940b |
Perf/cdp fetch cache and cookie alloc (#216)
* perf: cache HTTP clients + selector parsing + exponential wait backoff Three low-risk wins discovered while reading the hot paths. 1. op_fetch_url cached its reqwest::Client. Previously every JS-side fetch(), XHR, and dynamic script load built a fresh client, which re-loaded TLS roots and started with an empty connection pool. On an asset-heavy page with 30+ subresources this added ~150ms of wasted setup with zero connection reuse. The cache is a process- wide map keyed by proxy URL (empty string for no-proxy), so the first fetch on each proxy pays the build cost once and every subsequent fetch reuses the same pool. Also bumped pool_idle_timeout to 5 minutes and tcp_keepalive to 60s so connections stay warm for SPA-heavy workloads that hit the same origin dozens of times across a navigation. 2. DomTree::parse_selector now caches parsed selectors in a thread- local HashMap (cap 256). querySelector / querySelectorAll re- parsed the selector string every call before; batch DOM access patterns (an agent scraping a table row-by-row, a framework polling for elements) paid the cssparser cost N times. 3. browser_wait_for / browser_wait_for_text now exponentially back off (5 -> 10 -> 20 -> ... -> 200 ms). The old fixed 200ms tick meant a selector that appeared 30ms after the call still took the next full 200ms tick to be detected. Worst case is still 200ms per check, but the median wait drops sharply for fast appearances. 28/28 integration tests pass. Asset-heavy bench (rust-lang/rust, docs.stripe.com, etc) doesn't regress. * perf: module loader client cache, fetch concurrency cap, snapshot hide list, cookie alloc Round two of perf wins after a deep code audit. 1. ES module loader (module_loader.rs) was building a fresh reqwest::Client on every dynamic import. Modern SPAs (Next, Vite) load 20-50 chunks via dynamic import; each one used to spin up an empty connection pool with fresh TLS init. Now reuses the process-wide FETCH_CLIENT_CACHE that op_fetch_url already uses (the cached_request_client helper). Saves ~30-100ms on chunk- heavy first navigations. 2. Script and CSS fetches in execute_scripts used futures::join_all with unbounded concurrency. A 100-script page would open 100 TCP sockets at once, exhausting the connection pool and the OS ephemeral port range. Capped at 16 in-flight (well above the per- host pool ceiling real Chrome uses). 3. bootstrap.js __obscura_init() ran Object.keys(globalThis).filter() on every page setup to find internals to hide. The set of keys is static at snapshot-build time, so we now precompute it once (top-level statement just below the function) and __obscura_init just iterates the precomputed array. Was 5-40ms per page on heavy SPAs (1000+ globals); now ~0.2ms per page. 4. CookieJar::domain_matches allocated 2 lowercase Strings and a format! per call. The lookup runs per fetch per cookie domain; on a heavily-cookied site with 100 cookies across 30 domains that was thousands of allocations per page. Now zero-alloc with eq_ignore_ascii_case and char-boundary checks. |
||
|
SGavrl
|
71b9b4fbab |
fix(js,dom): scope element queries, real timers, form/prepend/equal (#180)
- Element.querySelector / querySelectorAll scope to the receiver subtree via a new query_selector_scoped Rust op, not always document root. Same for DocumentFragment. - setTimeout respects its delay argument via a new async op_sleep. setInterval is a real repeating timer instead of an alias to setTimeout. - document.forms / images / links return live querySelectorAll results instead of hardcoded []. - HTMLFormElement subclasses Element with .elements / .length, and form nodes are wrapped as HTMLFormElement so instanceof works. submit() stays on Element.prototype to preserve the existing submit-event dispatch and form-data navigation path. - Element.prepend has a real implementation. - Node.isEqualNode does a structural compare (nodeType, nodeName, nodeValue, attributes, recursive children) instead of nid identity. - Fix long-standing insert_before arg-order bug: bootstrap.js was passing (parent, new, ref) but _dom only forwards two args, so ref was silently dropped. Pass (new, ref) directly. Same fix in replaceChild. Adds 3 selector unit tests and 6 runtime regression tests covering the JS shim fixes. Closes #105. |
||
|
Ben Younes
|
183c444a56 |
fix(js,dom): add Storage global, CharacterData ChildNode methods, fix remove/isConnected (#176)
fix(dom): add missing DOM/JS shims for modern frameworks Closes #163. Adds missing DOM and JS shims required by modern frameworks (Svelte 5, Vue, Lit) and consent-management SDKs to execute without errors: - Define the `Storage` global constructor to resolve `ReferenceError`s and fix `Storage.prototype` access. - Apply standard ChildNode methods (`before`, `after`, `replaceWith`, `remove`) to `CharacterData.prototype`. This prevents `TypeError`s when frameworks call these on Text or Comment nodes during hydration. - Update `Element.prototype.isConnected` to perform a genuine parent-walk rather than returning a hardcoded `true`. - Fix `Element.prototype.remove` to properly clean up the internal ID index so removed elements are no longer queryable. - Remove outdated no-op stubs on the `Element` class that were shadowing the proper polyfill implementations. |
||
|
h4ckf0r0day
|
8749725b45 | Initial Release |