fix: prevent redirect-based SSRF in web-fetch and image-load call sites (#24491)
validate_url() in retrieval/web/utils.py only validates the initial URL. The HTTP clients used downstream (sync requests, sync requests via the parent WebBaseLoader._scrape, aiohttp via load_url_image) followed 3xx redirects by default and did not re-validate the redirect target against the private-IP / metadata-IP block list. An authenticated user could submit a public URL that 302-redirected to an internal address (RFC1918, 127.0.0.1, 169.254.169.254, etc.) and the redirected response was returned to them, enabling SSRF reads of internal services and cloud metadata. Three call sites needed allow_redirects=False to match the policy already enforced on the async _fetch() path: - SafeWebBaseLoader: override requests_kwargs in __init__ so that the inherited synchronous _scrape() path passes allow_redirects=False to self.session.get() (the parent WebBaseLoader uses requests' default allow_redirects=True). - get_content_from_url (retrieval/utils.py): pass allow_redirects=False on the streamed requests.get(...) call. - load_url_image (routers/images.py, image-edits endpoint): pass allow_redirects=False on the aiohttp session.get(...) call. Reports consolidated under GHSA-rh5x-h6pp-cjj6: - GHSA-rh5x-h6pp-cjj6 (tenbbughunters / Tenable) - sync _scrape - GHSA-5vxg-6gmv-m2qr (YLChen-007) - load_url_image - GHSA-hf76-c83f-63w2 (tempcollab) - aiohttp _fetch (already fixed) - GHSA-h55f-h5fh-mvm4 (sneaXOR) - get_content_from_url
This commit is contained in:
1 parent
793e628ac3
commit
8854541508
3 files changed
+22
-3
No files matched your search
@@ -485,6 +485,17 @@ class SafeWebBaseLoader(WebBaseLoader):
|
||||
"""
|
||||
super().__init__(*args, **kwargs)
|
||||
self.trust_env = trust_env
|
||||
# Prevent redirect-based SSRF on the synchronous _scrape() path.
|
||||
# validate_url() is called once on the originally-submitted URL, but the
|
||||
# parent WebBaseLoader's _scrape() invokes self.session.get(url, **self.requests_kwargs)
|
||||
# which by default follows redirects. Without the override below, an attacker
|
||||
# can submit a public URL that 302-redirects to an internal address (RFC1918,
|
||||
# 127.0.0.1, 169.254.169.254, etc.) and the redirected target is fetched without
|
||||
# re-validation. Matches the policy enforced on the async _fetch() path below.
|
||||
self.requests_kwargs = {
|
||||
**(self.requests_kwargs or {}),
|
||||
'allow_redirects': False,
|
||||
}
|
||||
|
||||
async def _fetch(self, url: str, retries: int = 3, cooldown: int = 2, backoff: float = 1.5) -> str:
|
||||
async with aiohttp.ClientSession(trust_env=self.trust_env) as session:
|
||||
|
||||
Reference in new issue
Block a user