perf: build debug log messages lazily so disabled debug logs cost nothing (#27834)
GLOBAL_LOG_LEVEL defaults to INFO, so every log.debug(...) in the backend is discarded, but the message is built first: 187 call sites interpolate their payload into an f-string before the logging call runs, so the work happens on every request and the result is thrown away. The worst one sits in process_chat_payload and stringifies the whole request body, full conversation history included, once per chat completion.
That one line with DEBUG disabled, CPython 3.12:
| conversation | payload | before | after |
| ------------ | ------- | -------- | ------- |
| 4 messages | 1.2 kB | 3.4 us | 0.07 us |
| 20 messages | 17 kB | 24.8 us | 0.07 us |
| 60 messages | 123 kB | 216.6 us | 0.07 us |
The lazy form log.debug('form_data: %s', form_data) hands the payload to record.getMessage(), which the InterceptHandler only reaches once a record has passed the level check. With DEBUG enabled the emitted lines are byte-identical, f'{x=}' sites included: those map to %r. MistralLoader._debug_log callers get the same treatment, since that wrapper already forwards *args.
This commit is contained in:
1 parent
b67804f2b6
commit
52cfb02c72
40 files changed
+198
-190
No files matched your search
@@ -443,7 +443,7 @@ async def get_all_models(request: Request, user: UserModel | None = None):
|
||||
dt = datetime.fromisoformat(expires_map[m['model']])
|
||||
m['expires_at'] = int(dt.timestamp())
|
||||
except Exception as exc:
|
||||
log.debug(f'Failed to get loaded models: {exc}')
|
||||
log.debug('Failed to get loaded models: %s', exc)
|
||||
|
||||
request.app.state.OLLAMA_MODELS = {m['model']: m for m in models_dict['models']}
|
||||
return models_dict
|
||||
@@ -691,7 +691,7 @@ async def push_model(
|
||||
url_idx = models[form_data.model]['urls'][0]
|
||||
|
||||
url = (await Config.get('ollama.base_urls', []))[url_idx]
|
||||
log.debug(f'url: {url}')
|
||||
log.debug('url: %s', url)
|
||||
|
||||
return await send_request(
|
||||
f'{url}/api/push',
|
||||
@@ -723,7 +723,7 @@ async def create_model(
|
||||
if not await Config.get('ollama.enable'):
|
||||
raise HTTPException(status_code=503, detail=ERROR_MESSAGES.OLLAMA_API_DISABLED)
|
||||
|
||||
log.debug(f'form_data: {form_data}')
|
||||
log.debug('form_data: %s', form_data)
|
||||
url = (await Config.get('ollama.base_urls', []))[url_idx]
|
||||
|
||||
return await send_request(
|
||||
|
||||
Reference in new issue
Block a user