fix(cdp): emit Network events for script-initiated requests (#415)
Requests made by page JS (fetch/XHR/dynamic script and stylesheet loads)
run in the V8 op layer and never produced a NetworkEvent, so CDP emitted
Network.requestWillBeSent / responseReceived only for the static
navigation subresources. Puppeteer/Playwright page.on('request') and
on('response') therefore missed every XHR/JSON response, which breaks
response capture (and is the root cause of the Aviasales half of #394).
op_fetch_url now records a JsNetworkEvent keyed by the same fetch-{N} id
under which it already stores the response body. The Page drains these
into its network_events before the CDP layer emits, so they flow through
emit_navigation_events as requestWillBeSent / responseReceived /
loadingFinished alongside the document subresources, and
Network.getResponseBody resolves for them via the shared id. Applied to
both navigation paths (server process_navigation and in-process
do_navigate). The queue is capped at 4096 events per page.
Closes #406.
This commit is contained in:
1 parent
b460b37e93
commit
d0b6fc4ffd
6 files changed
+270
No files matched your search
@@ -988,6 +988,13 @@ impl ObscuraJsRuntime {
|
||||
self.state.borrow().fetched_urls.clone()
|
||||
}
|
||||
|
||||
/// Drain the network events recorded for script-initiated requests
|
||||
/// (fetch/XHR/dynamic resource). The Page moves these into its own
|
||||
/// network_events so the CDP layer emits Network events for them (#406).
|
||||
pub fn take_js_network_events(&self) -> Vec<crate::ops::JsNetworkEvent> {
|
||||
std::mem::take(&mut self.state.borrow_mut().js_network_events)
|
||||
}
|
||||
|
||||
pub fn dom_ref(&self) -> Option<std::cell::Ref<'_, Option<DomTree>>> {
|
||||
let r = self.state.borrow();
|
||||
if r.dom.is_some() {
|
||||
|
||||
Reference in new issue
Block a user