perf: build debug log messages lazily so disabled debug logs cost nothing (#27834)
GLOBAL_LOG_LEVEL defaults to INFO, so every log.debug(...) in the backend is discarded, but the message is built first: 187 call sites interpolate their payload into an f-string before the logging call runs, so the work happens on every request and the result is thrown away. The worst one sits in process_chat_payload and stringifies the whole request body, full conversation history included, once per chat completion.
That one line with DEBUG disabled, CPython 3.12:
| conversation | payload | before | after |
| ------------ | ------- | -------- | ------- |
| 4 messages | 1.2 kB | 3.4 us | 0.07 us |
| 20 messages | 17 kB | 24.8 us | 0.07 us |
| 60 messages | 123 kB | 216.6 us | 0.07 us |
The lazy form log.debug('form_data: %s', form_data) hands the payload to record.getMessage(), which the InterceptHandler only reaches once a record has passed the level check. With DEBUG enabled the emitted lines are byte-identical, f'{x=}' sites included: those map to %r. MistralLoader._debug_log callers get the same treatment, since that wrapper already forwards *args.
This commit is contained in:
@@ -115,7 +115,7 @@ async def generate_direct_chat_completion(
|
||||
else:
|
||||
yield f'data: {data}\n\n'
|
||||
except Exception as e:
|
||||
log.debug(f'Error in event generator: {e}')
|
||||
log.debug('Error in event generator: %s', e)
|
||||
pass
|
||||
|
||||
# Define a background task to run the event generator
|
||||
@@ -155,7 +155,7 @@ async def generate_chat_completion(
|
||||
bypass_filter: bool = False,
|
||||
bypass_system_prompt: bool = False,
|
||||
):
|
||||
log.debug(f'generate_chat_completion: {form_data}')
|
||||
log.debug('generate_chat_completion: %s', form_data)
|
||||
if BYPASS_MODEL_ACCESS_CONTROL:
|
||||
bypass_filter = True
|
||||
|
||||
@@ -184,7 +184,7 @@ async def generate_chat_completion(
|
||||
**dict(request.app.state.MODELS.items()),
|
||||
request.state.model['id']: request.state.model,
|
||||
}
|
||||
log.debug(f'direct connection to model: {request.state.model["id"]}')
|
||||
log.debug('direct connection to model: %s', request.state.model['id'])
|
||||
else:
|
||||
models = request.app.state.MODELS
|
||||
|
||||
|
||||
Reference in New Issue
Block a user