refac: consolidate the web fetch address checks onto the request path (#27823)

* fix: apply the SSRF checks to redirect targets on every web fetch path

Two guards protect server-side fetches: a private-IP check and the operator's `WEB_FETCH_FILTER_LIST`. Neither reached a redirect hop on the aiohttp paths, and the filter list never reached one on the requests paths either.

aiohttp answers IP-literal hosts itself without consulting a resolver, so `_SSRFSafeResolver` was never invoked for a hop such as `http://169.254.169.254/` and the private-IP check simply did not run. With redirect following enabled, a submitted public URL that redirects to an IP literal reached loopback, RFC1918 and cloud-metadata addresses, and the response body was returned to the caller. The filter list was consulted only in `validate_url`, on the originally submitted URL, so a redirect to a filter-listed host was fetched without it ever being applied.

`_SSRFSafeResolver` is replaced by `_SSRFSafeConnector`, which hooks `_resolve_host` so the IP check also covers the IP-literal shortcut and both DNS cache paths. The filter list moves to a per-request hook on each transport, `connect()` for aiohttp and `send()` for the requests adapter, because those see the request destination: at the connection layer a proxied request presents the proxy's host, and a pooled connection skips resolution entirely. This covers every hop, including redirects, on all five aiohttp call sites and both requests sessions. The Playwright loader already validated each hop and is unchanged.

Both gaps required `AIOHTTP_CLIENT_ALLOW_REDIRECTS=true`, which is not the default.

Two behaviour changes for operators. The filter list now applies to redirect targets rather than only to submitted URLs. Under a forward proxy it is evaluated against the request destination instead of the proxy, which also fixes allowlist entries rejecting every fetch in proxied deployments.

* refac: match the web fetch filter list against resolved addresses

The filter list is now evaluated against the hostname together with the addresses it resolves to, at URL validation and on each connection, on both transports. An IPv6 address is also matched by the IPv4 address it carries.

* refac: screen outbound fetch addresses against reserved ranges ipaddress misses

`ipaddress.is_global` was the only test behind the web-fetch address check, and it answers a narrower question than "may we fetch this". Several special-purpose ranges are globally routable by registry while nothing on them is a legitimate destination, so they passed. Classification now screens those ranges on top of `is_global`, and applies the same screen to the IPv4 address embedded in an IPv6 transition encoding rather than only to the literal. All three checkpoints share the predicate, so they all inherit it.

The range list is the exact complement of what CPython's `ipaddress` already models, checked entry by entry against both IANA special-purpose registries. Prefixes IANA marks globally reachable are deliberately left out, so no real destination changes behaviour. Verified against 31 addresses covering every entry, their transition-encoded forms, and public controls in both families: 31/31 expected after, 18/31 before.

* refac: match web fetch filter entries that name an address or a range

A filter entry that parses as an address or a CIDR range is matched by containment rather than by DNS label suffix, so a range covers the addresses inside it and an address matches however it is spelled. A range entry previously matched nothing at all, silently.

The built-in list gains the special-purpose networks that ipaddress.is_global reports as reachable while nothing on them is a legitimate destination, so taking an address out of reach is a WEB_FETCH_FILTER_LIST change rather than a release. Those entries hold whether or not local web fetch is enabled; the private-address rule still follows the toggle.

backend/open_webui/utils/misc.py

@@ -2,12 +2,14 @@ from __future__ import annotations
 
 import collections.abc
 import hashlib
+import ipaddress
 import logging
 import re
 import threading
 import time
 import uuid
 from datetime import timedelta
+from functools import lru_cache
 from pathlib import Path
 from typing import Callable, Optional, Sequence, Union
 
@@ -87,17 +89,39 @@ def is_string_allowed(string: Union[str, Sequence[str]], filter_list: list[str |
     return True
 
 
+@lru_cache(maxsize=512)
+def as_network(pattern: str) -> ipaddress.IPv4Network | ipaddress.IPv6Network | None:
+    """A filter entry read as an address range, or None when the entry names a host instead.
+
+    Surrounding whitespace and a trailing dot are stripped here rather than by each caller,
+    since ip_network rejects both and the callers do not normalise the same way.
+    """
+    try:
+        return ipaddress.ip_network((pattern or '').strip().lower().rstrip('.'), strict=False)
+    except ValueError:
+        return None
+
+
 def _host_matches_pattern(host: str, pattern: str) -> bool:
     """Match a hostname against a filter entry on DNS label boundaries.
 
     `pattern` matches `host` when equal or a parent domain of it, so `corp.com`
-    matches `api.corp.com` but not `evilcorp.com`, and an IP literal matches only
-    itself. Avoids the raw-suffix confusion of a plain endswith.
+    matches `api.corp.com` but not `evilcorp.com`. Avoids the raw-suffix confusion
+    of a plain endswith.
+
+    An entry that names an address or a CIDR range is matched by containment instead, so
+    `10.0.0.0/8` covers `10.1.2.3` and an address matches any spelling of itself in its own family.
     """
     host = (host or '').strip().lower().rstrip('.')
     pattern = (pattern or '').strip().lower().rstrip('.')
     if not host or not pattern:
         return False
+    network = as_network(pattern)
+    if network is not None:
+        try:
+            return ipaddress.ip_address(host) in network
+        except ValueError:
+            return False  # a hostname is never inside an address range
     return host == pattern or host.endswith('.' + pattern)
 
 
@@ -108,21 +132,26 @@ def is_host_allowed(host: Union[str, Sequence[str]], filter_list: list[str | Non
     Pass a parsed hostname, never a full URL: matching against a URL lets a path
     component defeat the filter (e.g. ``https://blocked.example/x`` ends with ``/x``,
     not the blocked host). Entries prefixed with ``!`` are blocked; the rest form an allowlist.
+    An entry naming an address or a CIDR range is matched by containment instead.
     """
     if not filter_list:
         return True
 
-    allow_list, block_list = get_allow_block_lists(filter_list)
+    allow_list, _ = get_allow_block_lists(filter_list)
     hosts = [host] if isinstance(host, str) else list(host or [])
 
     if allow_list:
         if not any(_host_matches_pattern(h, allowed) for h in hosts for allowed in allow_list):
             return False
 
-    if any(_host_matches_pattern(h, blocked) for h in hosts for blocked in block_list):
-        return False
+    return not is_host_blocked(hosts, filter_list)
 
-    return True
+
+def is_host_blocked(host: Union[str, Sequence[str]], filter_list: list[str | None] = None) -> bool:
+    """Whether a host or resolved address matches a block entry, ignoring any allow entries."""
+    _, block_list = get_allow_block_lists(filter_list)
+    hosts = [host] if isinstance(host, str) else list(host or [])
+    return any(_host_matches_pattern(h, blocked) for h in hosts for blocked in block_list)
 
 
 def get_message_list(messages_map, message_id):