await in a finally block + cancellation of task group.

[jonathanslenders]

When scheduling a coroutine in a task group, and cancelling this task, the await in the finally block of that task doesn't execute. Somehow, we receive a second CancelledError here. This makes it impossible to rely on async code in finally blocks.

I'm not entirely sure whether the error is on my end or not. In the code below, the "after finally" print never happens.

import asyncio
from anyio import create_task_group

async def func():
    try:
        await asyncio.sleep(5)
    finally:
        print("Before finally")
        await asyncio.sleep(1)
        print("After finally")

async def main():
    async with create_task_group() as tg:
        tg.start_soon(func)
        await asyncio.sleep(1)
        tg.cancel_scope.cancel()

asyncio.run(main())

The work-around for me would be to start a normal asyncio task for the cleanup without waiting for it, or shielding it from cancellation, but that doesn't sound like structured concurrency.

There are a couple of similar issues, but mostly related to nested task groups.

[graingert]

This is part of the trio level cancellation semantics eg: https://trio.readthedocs.io/en/stable/reference-core.html#cancellation-semantics

Cancellations in Trio are “level triggered”, meaning that once a block has been cancelled, all cancellable operations in that block will keep raising Cancelled. This helps avoid some pitfalls around resource clean-up. For example, imagine that we have a function that connects to a remote server and sends some messages, and then cleans up on the way out:

with anyio.move_on_after(TIMEOUT):
    conn = make_connection()
    try:
        await conn.send_hello_msg()
    finally:
        with anyio.move_on_after(CLEANUP_TIMEOUT) as cleanup_scope:
            cleanup_scope.shield = True
            await conn.send_goodbye_msg()

[smurfix]

Why? Given that you need to handle cancellations vs. finally: blocks correctly anyway, since otherwise they swallow "real" exceptions, I see no advantage in doing that.

[agronholm]

This behavior is fairly well documented: https://anyio.readthedocs.io/en/stable/cancellation.html#finalization

[jonathanslenders]

Thanks everyone! with move_on_after(..., shield=True) works great!

I did read the anyio docs about cancellation, but not the trio docs (which appear to be great as well!).
Adding something like the above example with a finally-block to the anyio docs would be helpful, if that's something which makes sense. I was mostly confused, because I had no idea why the CancelledError was reraised continously.

[agronholm]

But there is such an example, at the link I provided?

async def do_something():
    try:
        await run_async_stuff()
    except get_cancelled_exc_class():
        with CancelScope(shield=True):
            await some_cleanup_function()

        raise

Or do you mean using move_on_after(..., shield=True) instead of just a CancelScope?

[jonathanslenders]

@agronholm: you're right. I must have read over that, looking for afinally block, but this is the same indeed.