WPT conformance: legacy-charset URL encoding, IDL attribute reflection, FileAPI, Storage, Range (#237)

* dom: complete detached documents; add ProgressEvent and parseHTMLUnsafe

createHTMLDocument returned a document whose body was null: parsing the
skeleton string as innerHTML of <html> collapsed through the fragment parser
and kept only <title>. Build head>title and body explicitly instead. The
DOMParser doc node also gains appendChild/removeChild/insertBefore and a
doctype getter, and createDocument carries its doctype argument.

This lets dom/common.js setupRangeTests() finish (it builds fixtures with
new Document(), createHTMLDocument(), createDocument()) instead of aborting at
harness setup, so the dom/ranges files run. Range-collapse 186/186,
Range-attributes 1/1, Range-cloneRange 55/62, StaticRange-constructor 14/17.
The large O(ranges x points) matrix files (Range-set, comparePoint,
compareBoundaryPoints) still time out: the boundary-point comparison walks the
tree through per-step DOM ops, which is too slow for those matrices. Speeding
that up (a Rust node-position op) is a separate change.

Also add globalThis.ProgressEvent (Event subclass with lengthComputable/
loaded/total) and the static Document.parseHTMLUnsafe(html).

* perf(dom): compute Range tree-order in Rust ops

The Range boundary-point algorithms compared nodes by walking ancestor chains
and sibling lists from JS, one DOM op per step. The dom/ranges matrices
(testRanges x testPoints, thousands of comparisons) made that time out.

Add node_index, compare_order, and node_root ops that do the walk in Rust over
the DOM tree (slab lookups, no per-step FFI), and use them in _rngNodeIndex,
_rngOrder, and _rngRoot. Range-comparePoint 3534/5580, Range-isPointInRange
5197/5733, Range-compareBoundaryPoints 6441/9313, all completing in under 10s
where they previously did not finish. Not on any engine hot path (querySelector,
mutation, navigation, fetch do not use these).

* fileapi: real byte-backed Blob/File, FileReader state machine, Request.blob

Blob was a one-line stub that string-joined its parts (wrong size for binary
or multibyte input) and exposed only text(). Replace it with a Uint8Array-backed
Blob: normalizes string/ArrayBuffer/typed-array/Blob parts, correct size,
slice(), text()/arrayBuffer()/bytes(), type normalization (lowercased, cleared
when non-ASCII), the endings option (native CRLF normalization), and
Symbol.toStringTag. File adds name/lastModified, the 2-argument requirement, and
its own toStringTag.

FileReader becomes a real state machine: EMPTY/LOADING/DONE, throws
InvalidStateError when a read starts while already LOADING, and fires
loadstart/progress/load/loadend (and abort) to both on* handlers and
addEventListener. readAsText/DataURL/ArrayBuffer/BinaryString produce correct
results. Request gains blob().

FileReader-multiple-reads 6/6, File-constructor-endings 10/11, Blob-slice
141/150.

* encoding: TextDecoder supports legacy charsets via encoding_rs

TextDecoder was a UTF-8-only JS shim that ignored its label, so
new TextDecoder('shift_jis'|'euc-jp'|'big5'|'gbk'|'windows-1252'|'iso-8859-1')
.decode() returned garbage. That matters for scraping: pages and scripts decode
legacy-encoded bytes (XHR/fetch responses, File/Blob text) all the time.

Back it with encoding_rs (already used by the net layer for response bodies):
obscura-net gains label_name() and decode_with_label(), exposed to JS as
op_encoding_for_label and op_text_decode. The constructor validates the label
(RangeError) and sets the canonical .encoding/.fatal/.ignoreBOM; decode()
supports fatal mode (TypeError on invalid input) and BOM handling.

Plain UTF-8 non-fatal decoding keeps a pure-JS fast path (no op, no JSON), so
Response.text()/Blob.text() on large bodies are unchanged; the op only runs for
legacy encodings or fatal mode.

textdecoder-labels 222/222, textdecoder-fatal 36/36, byte-order-marks 3/3,
utf16-surrogates 10/10, single-byte-decoder 168/504.

* selection: real Selection over Range (single instance per document)

getSelection() returned a fresh plain object each call, so it was not a Selection
instance, had no identity (window.getSelection() !== document.getSelection()),
and was missing removeRange/empty/collapseToStart/collapseToEnd/setPosition/
setBaseAndExtent/modify while collapse/extend/getRangeAt were broken stubs. That
was ~4862 failing 'selection[method] is not a function' subtests.

Replace it with a real Selection class built on the live Range: one cached
instance per document (window.getSelection() === document.getSelection(), null
for documents with no defaultView), an internal range + direction, derived
anchor/focus/isCollapsed/type/rangeCount, and the full method set
(getRangeAt with IndexSizeError, addRange, removeRange, removeAllRanges/empty,
collapse/setPosition, collapseToStart/End, extend, setBaseAndExtent,
selectAllChildren, containsNode, toString) with the spec DOMException names.
modify() stays a no-op (needs visual layout). No ops.rs changes; not on any hot
path.

setBaseAndExtent 120/120, removeAllRanges 104/104, collapseToStartEnd 51/51,
type 26/26, extend-exception 1/1.

* forms: HTMLInputElement valueAsNumber/valueAsDate/stepUp/stepDown

These were absent. Implement the number/date conversion API for the applicable
types (number, range, date, month, week, time, datetime-local): parsers and
formatters per type (Date.UTC for date/datetime-local, ISO-week Monday math for
week, months-since-epoch for month), valueAsNumber get/set, valueAsDate get/set,
and stepUp/stepDown(n) with the step scale/default per type, step-base snapping,
and min/max clamping. Non-applicable types and step="any" throw InvalidStateError;
non-finite valueAsNumber throws TypeError; range with empty value reads as the
midpoint (50 for 0..100). Plus min/max/step attribute reflection.

All on Element accessors gated by input type, only evaluated on explicit access;
not on any hot path.

input-stepup 51/53, input-valueasnumber 60/64, input-valueasdate 27/30.

* perf(encoding): skip label op for no-arg TextDecoder (Response/Blob text path)

new TextDecoder() with no argument (what Response.text()/Blob.text() and most
pages use) is UTF-8; resolve that without the op_encoding_for_label round trip.
Explicit labels still validate via the op.

* core: HTMLDialogElement, Node.normalize, Range.createContextualFragment

HTMLDialogElement was a bare alias of Element, so ~40 Core files crashed at the
first dialog.show()/showModal(). Add the dialog members on Element.prototype
(matching the popover/input convention): open reflection, returnValue (internal
slot, String coercion), oncancel/onclose defaulting to null, show()/showModal()
/close()/requestClose() with the open-state transitions, InvalidStateError
throwing (show-on-modal, showModal-on-open, showModal-disconnected), the async
non-bubbling close event, the synchronous cancelable cancel event (no re-entrant
cancel), beforetoggle/toggle, and closedBy. matches(':modal') reports a modally
shown dialog (like :popover-open). Modal top-layer/focus/render stays unhandled
(layout).

Node.normalize() now merges adjacent Text nodes and drops empty ones (was a
no-op). Range.createContextualFragment(html) parses a fragment.

createContextualFragment 34/35, dialog-open 3/3, dialog-return-value 1/1,
dialog-showModal 5/10, dialog-requestclose 22/75. Not on hot paths.

* custom-elements: CustomElementRegistry class, getName, ElementInternals, HTMLUnknownElement

customElements was a bare object literal, so 'CustomElementRegistry is not
defined' crashed ~22 Core files at top level (idlharness/feature-detect), and
HTMLUnknownElement/ElementInternals/attachInternals/getName were missing.

Promote the registry to a real CustomElementRegistry class (window.customElements
is now an instance, so 'define in CustomElementRegistry.prototype' works and the
class is exposed). Add spec define() validation/throw order (TypeError for a
non-constructor, SyntaxError for an invalid name, NotSupportedError for a
duplicate name/constructor or reentrant define), getName(), and a SyntaxError
reject in whenDefined for invalid names. The existing define -> upgrade ->
connectedCallback path (used by lit/Polymer on real pages) is preserved.

Add HTMLUnknownElement, ElementInternals (setFormValue/setValidity/checkValidity/
validity/validationMessage/form/states), and Element.attachInternals() with its
NotSupportedError cases.

Does NOT include the [[HTMLConstructor]] (new.target) construction rework or the
reaction wiring into appendChild/setAttribute (hot paths) - deferred to avoid
element-model and perf risk. getName 4/4.

* shadow-dom: ShadowRoot.adoptedStyleSheets (mirror Document)

ShadowRoot lacked adoptedStyleSheets; constructible-stylesheet adoption into a
shadow root threw. Add the getter/setter on ShadowRoot.prototype, matching
Document.adoptedStyleSheets.

* conformance: charset/URL encoding override, IDL reflection, Range and Storage fixes

Encoding: track the detected document character encoding and expose it as
document.characterSet/charset/inputEncoding instead of hardcoding UTF-8. Add the
WHATWG URL encoding override for <a>/<area> hrefs, so a legacy-charset document
percent-encodes the query in its own encoding (EUC-JP U+8108 becomes %CC%AE).
ASCII delimiters stay literal so structured queries are unaffected, and
unmappable code points become %26%23NNN%3B. UTF-8 documents take an early-return
so the href getter hot path is unchanged. fetch()/XHR text() now decode using
the response Content-Type charset instead of always UTF-8. Adds op_url_encode_query
and a document_encoding op; encoding state is plumbed through Page and the runtime.

Reflection: install the reflected IDL attribute layer once on Element.prototype
(no per-element defineProperty, so element creation and query paths are
untouched): global attributes (title, lang, dir, accessKey, slot, autofocus,
hidden, tabIndex) with correct type coercion and enum canonicalization, plus the
ARIAMixin (aria-* reflected as the ariaXxx nullable DOMString properties).

Range: compareBoundaryPoints ToUint16-converts the how argument (NaN/Infinity
no longer false-throw) and raises a real WrongDocumentError DOMException for
ranges that do not share a root.

Storage: reimplement localStorage/sessionStorage as a WHATWG Proxy so property
access (localStorage.foo = x, delete, in, Object.keys) stays in sync with
length, key() and iteration.

crates/obscura-js/src/ops.rs

@@ -41,6 +41,10 @@ pub struct InterceptedRequest {
 pub struct ObscuraState {
     pub dom: Option<DomTree>,
     pub url: String,
+    /// WHATWG canonical name of the document's character encoding (e.g.
+    /// "UTF-8", "EUC-JP"). Backs `document.characterSet` and the URL query
+    /// encoding override for `<a>`/`<area>` hrefs in legacy-charset documents.
+    pub encoding: String,
     pub title: String,
     pub blocked_urls: Vec<String>,
     pub cookie_jar: Option<Arc<CookieJar>>,
@@ -60,6 +64,7 @@ impl ObscuraState {
         ObscuraState {
             dom: None,
             url: "about:blank".to_string(),
+            encoding: "UTF-8".to_string(),
             title: String::new(),
             blocked_urls: Vec::new(),
             cookie_jar: None,
@@ -89,6 +94,7 @@ fn op_dom(state: &OpState, #[string] cmd: String, #[string] arg1: String, #[stri
         "document_node_id" => dom.document().index().to_string(),
         "document_title" => serde_json::to_string(&gs.title).unwrap_or("\"\"".into()),
         "document_url" => serde_json::to_string(&gs.url).unwrap_or("\"\"".into()),
+        "document_encoding" => serde_json::to_string(&gs.encoding).unwrap_or("\"UTF-8\"".into()),
         "document_element" => {
             for cid in dom.children(dom.document()) {
                 if let Some(n) = dom.get_node(cid) {
@@ -1132,6 +1138,38 @@ fn op_url_resolve(#[string] href: &str, #[string] base: &str) -> String {
     .unwrap_or_default()
 }
 
+/// Canonical (lowercased) WHATWG name for a TextDecoder label, or "" if the
+/// label is unknown (the JS constructor turns "" into a RangeError).
+#[op2]
+#[string]
+fn op_encoding_for_label(#[string] label: &str) -> String {
+    obscura_net::label_name(label).unwrap_or_default()
+}
+
+/// Decode bytes with a legacy/explicit encoding via encoding_rs. Returns
+/// {"ok":true,"v":<string>} or {"ok":false} (unknown label, or a fatal decode
+/// error). The UTF-8 non-fatal common case is handled in JS without this op.
+#[op2]
+#[string]
+fn op_text_decode(#[string] label: &str, #[buffer] bytes: &[u8], fatal: bool, ignore_bom: bool) -> String {
+    match obscura_net::decode_with_label(label, bytes, fatal, ignore_bom) {
+        Some(s) => serde_json::json!({ "ok": true, "v": s }).to_string(),
+        None => "{\"ok\":false}".to_string(),
+    }
+}
+
+/// Re-encode a URL query component using a non-UTF-8 document encoding override
+/// (the WHATWG "encoding override"). `query` is the already-UTF-8-decoded query
+/// string; `label` the target charset; `special` whether the URL has a special
+/// scheme (adds `'` to the percent-encode set). Returns the encoded query, or
+/// the input unchanged if the label is unknown. Only called by the JS anchor
+/// path when the document is non-UTF-8, so the UTF-8 hot path never reaches it.
+#[op2]
+#[string]
+fn op_url_encode_query(#[string] query: &str, #[string] label: &str, special: bool) -> String {
+    obscura_net::url_encode_query(query, label, special).unwrap_or_else(|| query.to_string())
+}
+
 pub fn build_extension() -> Extension {
     Extension {
         name: "obscura_dom",
@@ -1148,6 +1186,9 @@ pub fn build_extension() -> Extension {
             op_url_parse(),
             op_url_set(),
             op_url_resolve(),
+            op_encoding_for_label(),
+            op_text_decode(),
+            op_url_encode_query(),
         ]),
         ..Default::default()
     }

crates/obscura-js/src/runtime.rs

@@ -85,6 +85,13 @@ impl ObscuraJsRuntime {
         self.state.borrow_mut().url = url.to_string();
     }
 
+    /// Set the document's character encoding (WHATWG canonical name). Backs
+    /// `document.characterSet` and the `<a>`/`<area>` URL query encoding
+    /// override for legacy-charset documents.
+    pub fn set_encoding(&self, encoding: &str) {
+        self.state.borrow_mut().encoding = encoding.to_string();
+    }
+
     pub fn set_title(&self, title: &str) {
         self.state.borrow_mut().title = title.to_string();
     }