Test suite freezes during teardown

[m-reichle]

Hi,

ever since the recent update my test suite freezes on teardown. I'm not sure what goes wrong, but here is a simple test case to reproduce:

I set up a test environment as follows:

python3 -m venv env
source env/bin/activate
pip install pytest-playwright-async
playwright install firefox

pytest.ini

[pytest]
asyncio_mode = auto

test_playwright.py

from playwright.async_api import Page as APage
from playwright.sync_api import Page as SPage


async def test_async_api(page: APage):

    # access webpage
    await page.goto('https://playwright.dev/')


def test_sync_api(page: SPage):

    # access webpage
    page.goto('https://playwright.dev/')

Run the tests like this
python -m pytest -sv --setup-show --browser firefox test_playwright.py::<test_name>

test_sync_api runs without issue, test_async_api gets as far as shown below, then freezes and needs to be stopped using Ctrl-c

platform linux -- Python 3.11.2, pytest-8.2.1, pluggy-1.5.0 -- /tmp/test_test/env/bin/python
cachedir: .pytest_cache
rootdir: /tmp/test_test
configfile: pytest.ini
plugins: asyncio-0.23.7, playwright-0.5.0, pytest_playwright_async-0.17.0, base-url-2.1.0
asyncio: mode=Mode.AUTO
collected 1 item                                                                                                                                                                           

test_playwright.py::test_async_api[firefox] 
SETUP    S event_loop_policy
SETUP    S pytestconfig
SETUP    S delete_output_dir (fixtures used: pytestconfig)
SETUP    S base_url
SETUP    S _verify_url (fixtures used: base_url)
SETUP    S browser_type_launch_args (fixtures used: pytestconfig)
SETUP    S playwright
SETUP    S browser_name['firefox']
SETUP    S browser_type (fixtures used: browser_name, playwright)
SETUP    S launch_browser (fixtures used: browser_type, browser_type_launch_args)
SETUP    S browser (fixtures used: launch_browser)
SETUP    S device (fixtures used: pytestconfig)
SETUP    S _pw_artifacts_folder
SETUP    S browser_context_args (fixtures used: _pw_artifacts_folder, base_url, device, playwright, pytestconfig)
        SETUP    F _artifacts_recorder (fixtures used: _pw_artifacts_folder, playwright, pytestconfig)
        SETUP    F new_context (fixtures used: _artifacts_recorder, browser, browser_context_args)
        SETUP    F context (fixtures used: new_context)
        SETUP    F page (fixtures used: context)
        SETUP    F event_loop
        test_playwright.py::test_async_api[firefox] (fixtures used: _artifacts_recorder, _pw_artifacts_folder, _verify_url, base_url, browser, browser_context_args, browser_name, browser_type, browser_type_launch_args, context, delete_output_dir, device, event_loop, event_loop_policy, launch_browser, new_context, page, playwright, pytestconfig, request)FAILED
        TEARDOWN F event_loop
        TEARDOWN F page
        TEARDOWN F context

[m9810223]

Hi @m-reichle

After some debugging, I think you can use:

• pytest-asyncio==0.20.3
• pytest==8.0.0
• nest-asyncio

and add

# conftest.py
import asyncio
import nest_asyncio
import pytest_asyncio
@pytest_asyncio.fixture(scope='session')
def event_loop(): 
    policy = asyncio.get_event_loop_policy()
    loop = policy.new_event_loop()
    nest_asyncio.apply(loop) # !!
    yield loop
    loop.close()

---

Please let me know if this helps!

[jluebbe]

Note that https://github.com/erdewit/nest_asyncio has been marked as archived recently, so it's not a long-term solution.

[m-reichle]

Thanks for your quick response!

This does indeed fix the issue, but, as jluebbe mentioned, the nest_asyncio project has sadly been archived when its maintainer passed away in March.

From what I've read the python project isn't planning on including nested loops any time soon (python/cpython#66435 (comment)). It seems https://github.com/oremanj/greenback does similar things, I've seen that mentioned in related discussions, not sure if it is applicable here.

[m9810223]

I have done some research and found no other solutions.
Until we find a suitable solution, this is how it will work for now.
If anyone finds a package or example that solves this problem, please let me know, I'd appreciate it.

[m9810223]

Most of the packages I've found don't provide event_loop, which is what we need for pytest-asyncio.
The way to achieve this is to hack/modify pytest-asyncio with the packages.

[Wurstnase]

Hi, thanks for this nice transfer to async.

I played last days with playwright and async. I need async because I'm using fastapi with uvicorn in parallel and mock some endpoints.

I'm wondering that @m-reichle is using page: APage instead of page_async: APage. Was this intended?

I need to vendor this project, because I don't use pytest-asyncio. Instead I'm using anyio which has pytest support and comes with fastapi. Do you tried this? I need to replace @pytest_asyncio.fixture to the normal decorator @pytest.fixture.

[m-reichle]

The page in my example code is just the name of the parameter passed to function. It's not relevant to what it does.

[Wurstnase]

Hmmm. I don't understand. In pytest these arguments are normally fixtures and the fixture name for the async page in this pytest plugin ist page_async.

[m-reichle]

Apologies, I got that wrong.

I created this issue a couple months ago and have since moved on to other topics, so I'm a bit hazy on what the details were.

I've fixed the example code to this:

test_playwright.py

async def test_async_api(page_async):

    # access webpage
    await page_async.goto('https://playwright.dev/')


def test_sync_api(page):

    # access webpage
    page.goto('https://playwright.dev/')

However the actual issue seems to remain, the sync test runs without problems, while the async one gets stuck creating the context.

From what I (vaguely) remember about the underlying problem it was that playwright-async-pytest cannot access the event_loop and tries to create one of its own from within the existing loop. @m9810223 can probably tell you more.

[Wurstnase]

I've deinstalled pytest_asyncio, installed anyio and patched this plugin with this:

anyio.diff

diff --git a/pytest_playwright_async.py b/pytest_playwright_async.py
index 858899a..15a6dbb 100644
--- a/pytest_playwright_async.py
+++ b/pytest_playwright_async.py
@@ -29,9 +29,8 @@ from playwright.async_api import StorageState
 from playwright.async_api import ViewportSize
 from playwright.async_api import async_playwright
 import pytest
-import pytest_asyncio
 from pytest_playwright.pytest_playwright import _build_artifact_test_folder
-from pytest_playwright.pytest_playwright import create_guid
+from pytest_playwright.pytest_playwright import _create_guid as create_guid
 from pytest_playwright.pytest_playwright import slugify
 
 
@@ -60,7 +59,7 @@ def browser_context_args_async(
     return context_args
 
 
-@pytest_asyncio.fixture
+@pytest.fixture
 async def _artifacts_recorder_async(
     request: pytest.FixtureRequest,
     playwright_async: Playwright,
@@ -77,7 +76,7 @@ async def _artifacts_recorder_async(
     await async_artifacts_recorder.did_finish_test(failed)
 
 
-@pytest_asyncio.fixture(scope='session')
+@pytest.fixture(scope='session')
 async def playwright_async() -> AsyncGenerator[Playwright, None]:
     apw = await async_playwright().start()
     yield apw
@@ -102,7 +101,7 @@ def launch_browser_async(
     return launch
 
 
-@pytest_asyncio.fixture(scope='session')
+@pytest.fixture(scope='session')
 async def browser_async(
     launch_browser_async: Callable[..., Awaitable[Browser]],
 ) -> AsyncGenerator[Browser, None]:
@@ -151,7 +150,7 @@ class AsyncCreateContextCallback(Protocol):
     ) -> BrowserContext: ...
 
 
-@pytest_asyncio.fixture
+@pytest.fixture
 async def new_context_async(
     browser_async: Browser,
     browser_context_args_async: dict,
@@ -183,12 +182,12 @@ async def new_context_async(
         await context_async.close()
 
 
-@pytest_asyncio.fixture
+@pytest.fixture
 async def context_async(new_context_async: AsyncCreateContextCallback) -> BrowserContext:
     return await new_context_async()
 
 
-@pytest_asyncio.fixture
+@pytest.fixture
 async def page_async(context_async: BrowserContext) -> Page:
     return await context_async.new_page()
 
@@ -303,4 +302,4 @@ class AsyncArtifactsRecorder:
                     )
                     self._screenshots.append(str(screenshot_path))
                 except Error:
-                    pass
\ No newline at end of file
+                    pass

Finally my test_playwright.py

import anyio
from playwright.async_api import Page as APage
from playwright.sync_api import Page as SPage
import pytest


@pytest.fixture(scope="session", autouse=True)
def anyio_backend():
    return "asyncio"


async def test_async_api(page_async: APage):

    # access webpage
    await page_async.goto('https://playwright.dev/')


def test_sync_api(page: SPage):

    # access webpage
    page.goto('https://playwright.dev/')

[m9810223]

Hi everyone, just now I released a new version. https://pypi.org/project/pytest_playwright_async/1.0.0/

The changes are:

• Use anyio instead of pytest-asyncio, as in the example provided by @Wurstnase
• Use nest-asyncio to handle nested event loops, even though the original author has passed away (RIP @erdewit). Thanks to @jluebbe for the author information
• Add sync testing in addition to the existing async. As @m-reichle encountered in this issue

Finally, even though it's a lot of work, I'm looking forward to @microsoft - @mxschmitt improving Playwright's async pytest libirary as soon as possible. microsoft/playwright-pytest#74

I hope this repo is just a short-term solution!

[Wurstnase]

Thanks @m9810223 for the fast implementation.

I think that nest-asyncio shouldn't be a mandatory dependency. It will helps in your examples and may be installed in some circumstances, but not generally with this plugin.