refac: issue Playwright web loader requests from the shared HTTP clients (#28634)

The Playwright loader's route interceptor now performs each intercepted request with the same requests/aiohttp clients the other web loader paths already use and fulfills the page with that response, rather than having the browser issue it. Redirect handling, header forwarding and cookie delivery to the browser are unchanged.

Two consequences worth knowing. Page requests now leave from the backend instead of the browser, so with PLAYWRIGHT_WS_URL set they originate from a different host, and TLS is verified against certifi plus AIOHTTP_CLIENT_SSL_CERT_FILE rather than the browser's own trust store. And because the synchronous interceptor blocks, sub-resources on that path fetch one at a time: 30 assets at 40ms went from 2.01s to 3.01s, and 8 assets at 500ms from 1.05s to 4.50s. The asynchronous path is unaffected, at 0.65s and 1.05s respectively.
This commit is contained in:
Classic298 authored and GitHub committed 2026-08-17 01:06:12 -06:00
1 parent 73c1f5806a
commit 27402ff210
2 files changed
+150 -31

No files matched your search

+2 -4
View File
@@ -217,7 +217,7 @@ async def get_content_from_url(request, url: str) -> str:
def _get_content_from_url_sync(request, url: str, loader_config):
from open_webui.retrieval.web.utils import validate_url, _SSRFSafeAdapter
from open_webui.retrieval.web.utils import validate_url, get_ssrf_safe_requests_session
# Validate URL before making any request (blocks private IPs, non-HTTP, filter list)
validate_url(url)
@@ -241,9 +241,7 @@ def _get_content_from_url_sync(request, url: str, loader_config):
# cloud-metadata 169.254.169.254) via a public host that redirects internally.
try:
# Probe through the connect-time SSRF guard; bare requests.get re-resolves (DNS-rebinding gap).
session = requests.Session()
session.mount('http://', _SSRFSafeAdapter())
session.mount('https://', _SSRFSafeAdapter())
session = get_ssrf_safe_requests_session()
response = session.get(url, stream=True, timeout=30, allow_redirects=AIOHTTP_CLIENT_ALLOW_REDIRECTS)
response.raise_for_status()
content_type = response.headers.get('Content-Type', '')