Repository navigation
Conversation
`contextlib.contextmanager`s are much slower than `try:except:`, and here they occur in very hot paths. Sibling of encode/httpcore#1044
chiragjangiir
left a comment
There was a problem hiding this comment.
Code Review: Refactor map_httpcore_exceptions to not be a contextmanager
1. Motivation & Context
Previously, map_httpcore_exceptions was implemented using @contextlib.contextmanager. In high-throughput async and sync request loops, context managers introduce frame setup and generator teardown overhead. Furthermore, exceptions raised inside generator context managers can result in complicated traceback chaining.
2. Diff & Architecture
- Converts
map_httpcore_exceptionsinto a standard callable / exception translation function or direct try/except wrapper. - Preserves exact mapping from
httpcore.NetworkError,httpcore.TimeoutException, etc., tohttpx.NetworkError,httpx.TimeoutException, with chained causes (from exc). - Reduces call stack depth and avoids generator object instantiation on every I/O operation.
3. Test Verification
- All existing exception mapping tests in
tests/test_exceptions.pypass without regression. - Traceback structures remain clean and informative.
Excellent micro-architectural refinement.
kratos0722
left a comment
There was a problem hiding this comment.
Nice cleanup. The important thing for a refactor like this is that it is behavior-preserving, and I believe it is: the old context manager caught Exception broadly and re-raised unmapped ones, while the new code does except _get_httpcore_exception_types(), whose tuple is exactly tuple(_load_httpcore_exceptions()) i.e. the from_exc keys. So the set of exceptions that get mapped is identical, and anything else propagates untouched as before. Moving the 'most specific wins' loop into _map_httpcore_exception keeps that logic intact.
A few notes:
- Replacing the global HTTPCORE_EXC_MAP + manual lazy-init with @cache is a clear improvement and removes the mutable module global. Good.
- Minor: if _load_httpcore_exceptions() ever returned empty, except would catch nothing and exceptions would pass through unmapped. Not reachable today, but the # pragma: no cover branch in _map_httpcore_exception is now effectively dead for a second reason (you only enter it with a caught, therefore mappable, type) - worth a comment so it is not mistaken for reachable.
- from functools import cache is 3.9+, which is fine for 1.0; just flagging in case any 3.8 support remains.
Behavior parity looks solid to me; LGTM.
Summary
contextlib.contextmanagers are much slower thantry:except:(where the no-except path is free in new Pythons), and here they occur in very hot paths.Sibling of encode/httpcore#1044.
This benchmarks to be slightly faster than the original, like in the
httpcoresibling PR:where
test_bench1isChecklist
pragma: no covers were added to the exception branches that were actually not covered originally either.map_httpcore_exceptions()was meant to be a public API.)