You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
asyncio: awaiting StreamWriter.wait_closed() in an except block rewrites the in-flight exception's traceback in place (same exception object shared between read futures and the close waiter) #156278
While debugging a network client I found that the logged tracebacks for connection failures pointed at the cleanup code (writer.close() / await writer.wait_closed()) instead of the read call that actually failed. It turns out that a very common cleanup idiom silently rewrites the caught exception's __traceback__ in place.
When a stream transport dies (e.g. peer sends TCP RST), StreamReaderProtocol.connection_lost sets one and the same exception object on both the reader (and thus any pending read future) and the stream's _closed waiter:
Since bpo-45924 / gh-90082 (the fix for traceback accumulation on repeated Future.result() calls), a future snapshots the exception's traceback at set_exception time and restores it with with_traceback() on every re-raise:
with_traceback() mutates the exception object. That is harmless when the exception belongs to a single future, but here the same object is owned by two futures, so awaiting the second future (the close waiter) rewrites the traceback of the exception the user is currently handling — even if that second raise is caught/suppressed.
Consequence: the widely used cleanup idiom (the close() + wait_closed() sequence recommended by the asyncio.StreamWriter docs), e.g.
try:
data=awaitreader.readexactly(n) # raises ConnectionResetError E with the true tracebackexceptException:
writer.close()
withcontextlib.suppress(OSError):
awaitwriter.wait_closed() # re-raises the SAME object E; suppressed, but# Future.result() has already rewritten E.__traceback__raise# propagates E with a misleading traceback
produces a final traceback that:
points at the wrong raise site — the top-most application frame is the await writer.wait_closed() line, not the readexactly() call that actually failed;
is missing intermediate frames — the frames added while E propagated out of readexactly() / _wait_for_data() are gone, replaced by wait_closed()'s frames;
has no During handling of the above exception... chain — it is the same object, so no __context__ is attached and nothing hints that a second raise happened.
The result is a log/traceback.format_exc() output that claims the failure happened during cleanup, erasing the actual failing operation. This is quite misleading when debugging production failures.
Reproducer
importasyncioimportsocketimportstructimportthreadingimporttracebackfromcontextlibimportsuppressdeffmt(tb):
return" <- ".join(f"{f.name}:{f.lineno}"forfintraceback.extract_tb(tb))
port_holder= []
defserver():
# Accept one connection, read the request, then close with SO_LINGER(0)# so the client gets a TCP RST -> ConnectionResetError.srv=socket.socket()
srv.bind(("127.0.0.1", 0))
port_holder.append(srv.getsockname()[1])
srv.listen(1)
conn, _=srv.accept()
conn.recv(4)
conn.setsockopt(socket.SOL_SOCKET, socket.SO_LINGER, struct.pack("ii", 1, 0))
conn.close()
srv.close()
asyncdefmain():
threading.Thread(target=server, daemon=True).start()
whilenotport_holder:
awaitasyncio.sleep(0.01)
reader, writer=awaitasyncio.open_connection("127.0.0.1", port_holder[0])
try:
writer.write(b"REQ1")
awaitwriter.drain()
awaitreader.readexactly(4) # <- the real raise siteexceptExceptionase:
print("BEFORE wait_closed:", fmt(e.__traceback__))
writer.close()
try:
awaitwriter.wait_closed()
exceptExceptionase2:
print("wait_closed raised the identical object:", e2ise)
print("AFTER wait_closed:", fmt(e.__traceback__))
raise# propagates e with the rewritten tracebackwithsuppress(ConnectionResetError):
asyncio.run(main())
Output on 3.14.3 (3.12.4 and 3.13.5 differ only in stdlib line numbers):
BEFORE wait_closed: main:38 <- readexactly:769 <- _wait_for_data:539 <- _read_ready__data_received:1009
wait_closed raised the identical object: True
AFTER wait_closed: main:43 <- wait_closed:358 <- _read_ready__data_received:1009
Before wait_closed(), the traceback correctly shows readexactly → _wait_for_data. After it, those frames are gone: the traceback now claims the exception surfaced at the await writer.wait_closed() line (main:43), and readexactly/_wait_for_data have been replaced by wait_closed. The exception object raised by wait_closed() is id-identical to the one being handled.
Expected behavior
Handling (even suppressing) the exception raised by await writer.wait_closed() should not mutate the traceback of the in-flight exception being handled in the except block. Either the traceback restore should not mutate a shared exception object in place, or the two waiters should not share one exception object.
Notes
The mechanism is the interaction of two individually reasonable behaviors: (a) connection_lost sharing one exception object across the read future and the close waiter (Lib/asyncio/streams.py), and (b) the anti-accumulation traceback snapshot/restore from Incorrect traceback when future's exception is raised multiple times #90082 (Lib/asyncio/futures.py, both the Python and C implementations), which restores via with_traceback() and therefore writes to the shared object.
asyncio: initial exception traceback frames are lost for Future.result() #154791 is related (it is about the C future clearing its stored traceback after the first result() call) but distinct: this report is about in-place mutation of an exception object that is shared between two futures, observable from an unrelated except block.
The same pattern presumably affects any place where one exception object is set on multiple futures, not just streams.
CPython versions tested on
3.12.4, 3.13.5, 3.14.3 (identical behavior on all three)
Bug description
While debugging a network client I found that the logged tracebacks for connection failures pointed at the cleanup code (
writer.close()/await writer.wait_closed()) instead of the read call that actually failed. It turns out that a very common cleanup idiom silently rewrites the caught exception's__traceback__in place.When a stream transport dies (e.g. peer sends TCP RST),
StreamReaderProtocol.connection_lostsets one and the same exception object on both the reader (and thus any pending read future) and the stream's_closedwaiter:https://github.com/python/cpython/blob/v3.14.3/Lib/asyncio/streams.py#L260-L271
Since bpo-45924 / gh-90082 (the fix for traceback accumulation on repeated
Future.result()calls), a future snapshots the exception's traceback atset_exceptiontime and restores it withwith_traceback()on every re-raise:https://github.com/python/cpython/blob/v3.14.3/Lib/asyncio/futures.py#L208
with_traceback()mutates the exception object. That is harmless when the exception belongs to a single future, but here the same object is owned by two futures, so awaiting the second future (the close waiter) rewrites the traceback of the exception the user is currently handling — even if that second raise is caught/suppressed.Consequence: the widely used cleanup idiom (the
close()+wait_closed()sequence recommended by theasyncio.StreamWriterdocs), e.g.produces a final traceback that:
await writer.wait_closed()line, not thereadexactly()call that actually failed;Epropagated out ofreadexactly()/_wait_for_data()are gone, replaced bywait_closed()'s frames;During handling of the above exception...chain — it is the same object, so no__context__is attached and nothing hints that a second raise happened.The result is a log/
traceback.format_exc()output that claims the failure happened during cleanup, erasing the actual failing operation. This is quite misleading when debugging production failures.Reproducer
Output on 3.14.3 (3.12.4 and 3.13.5 differ only in stdlib line numbers):
Before
wait_closed(), the traceback correctly showsreadexactly→_wait_for_data. After it, those frames are gone: the traceback now claims the exception surfaced at theawait writer.wait_closed()line (main:43), andreadexactly/_wait_for_datahave been replaced bywait_closed. The exception object raised bywait_closed()is id-identical to the one being handled.Expected behavior
Handling (even suppressing) the exception raised by
await writer.wait_closed()should not mutate the traceback of the in-flight exception being handled in theexceptblock. Either the traceback restore should not mutate a shared exception object in place, or the two waiters should not share one exception object.Notes
connection_lostsharing one exception object across the read future and the close waiter (Lib/asyncio/streams.py), and (b) the anti-accumulation traceback snapshot/restore from Incorrect traceback when future's exception is raised multiple times #90082 (Lib/asyncio/futures.py, both the Python and C implementations), which restores viawith_traceback()and therefore writes to the shared object.result()call) but distinct: this report is about in-place mutation of an exception object that is shared between two futures, observable from an unrelatedexceptblock.CPython versions tested on
3.12.4, 3.13.5, 3.14.3 (identical behavior on all three)
Operating systems tested on
macOS
Linked PRs