fix(cdp): emit Network events for script-initiated requests (#415)

Requests made by page JS (fetch/XHR/dynamic script and stylesheet loads)
run in the V8 op layer and never produced a NetworkEvent, so CDP emitted
Network.requestWillBeSent / responseReceived only for the static
navigation subresources. Puppeteer/Playwright page.on('request') and
on('response') therefore missed every XHR/JSON response, which breaks
response capture (and is the root cause of the Aviasales half of #394).

op_fetch_url now records a JsNetworkEvent keyed by the same fetch-{N} id
under which it already stores the response body. The Page drains these
into its network_events before the CDP layer emits, so they flow through
emit_navigation_events as requestWillBeSent / responseReceived /
loadingFinished alongside the document subresources, and
Network.getResponseBody resolves for them via the shared id. Applied to
both navigation paths (server process_navigation and in-process
do_navigate). The queue is capped at 4096 events per page.

Closes #406.
This commit is contained in:
SGavrl authored and GitHub committed 2026-07-14 13:23:55 +00:00
1 parent b460b37e93
commit d0b6fc4ffd
6 files changed
+270

No files matched your search

+4
View File
@@ -778,6 +778,10 @@ async fn process_with_interception(
let mut page = page_back.expect("navigation task should return the page");
// Fold in network events for script-initiated requests (fetch/XHR/dynamic
// resource) so they emit as Network.requestWillBeSent / responseReceived
// alongside the static navigation subresources (#406).
page.sync_js_network_events();
let network_events: Vec<_> = page.network_events.drain(..).collect();
let page_url = page.url_string();
let page_id_for_events = page.id.clone();