Add Chrome DevTools Protocol (CDP) for Performance metrics

v1.14.1 - 2022.01.27

Overview

Pylenium now uses Selenium v4.1.0! The goal was to start leveraging Selenium 4 features and this release focused on the Performance metrics from the Chrome DevTools Protocol (CDP).

Highlights

Capturing web performance metrics is a powerful feature of web automation. Selenium 4 introduced the ability to leverage CDP which has a Performance API, but it has its limitations. Let's dive into examples because we provide a few ways to capture perf metrics 😄

Chrome DevTools Protocol (CDP)

• selenium.dev docs
• chromedevtools.github.io/devtools-protocol

We created a CDP class so you could call the API directly from Pylenium. Currently, there are only two methods:

# Run any command against the CDP API and get a result. (See the docs linked above for usage examples)
execute_command(cmd: str, cmd_args: Dict) -> Dict

# Return performance metrics like 'ScriptDuration', 'ThreadTime', 'ProcessTime', and 'DomContentLoaded'
get_performance_metrics() -> Dict

Using it is easy!

def test_capture_performance_metrics(py):
    py.visit("https://qap.dev")
    metrics = py.cdp.get_performance_metrics()
    assert metrics["metrics"]
    assert metrics["metrics"][0]["name"] == "Timestamp"
    assert metrics["metrics"][0]["value"] > 0

And in case you're wondering, py.cdp.get_performance_metrics() returns a dictionary that looks kinda like this:

{
  'metrics': [
    {'name': 'Timestamp', 'value': 425608.80694},
    {'name': 'AudioHandlers', 'value': 0},
    {'name': 'ThreadTime', 'value': 0.002074},
    ...
  ]
}

Limitations

⚠️ Selenium says that CDP is meant for Chrome and more experimental for now!

This is good to be aware of. CDP may not work for other browsers or if you're running remotely against something like Selenium Grid.

Pylenium Performance API

The limitations may prevent you from using CDP, but we've got you covered! Pylenium has a Performance API that you can use to capture metrics as well.

💡 Some of these metrics may not be from a "Standard", but they all use W3C objects like PaintTiming. So, you'll see that some of the calculations are made by us, but they have already been applied and used to provide insights and solve problems at a few companies (including Adobe 😎). I hope they can help you too!

Using it is easy as well!

perf = py.performance.get()

This returns a lot of data compared to CDP, so I recommend looking at the performance.py module to see all that's available!

Examples

💡 Calling this method too soon may yield NoneTypes because the browser hasn't generated them yet.

• Store the entire WebPerformance object and print it as a dictionary

  perf = py.performance.get()
  print(perf.dict())

• Get a single data point from WebPerformance

  tti = py.performance.get().time_to_interactive()
  assert tti < 5000 # 5 seconds

What else changed?

• Upgrade to Selenium 4! by @ElSnoMan in #221
• Fix executable_path deprecation warning for ChromeDriver by @ElSnoMan in #223
• Add EdgeOptions and EdgeService to webdriver_factory by @ElSnoMan in #224
• Add new_window() and new_tab() methods to SwitchTo class by @ElSnoMan in #225

  • This does exactly like what it sounds like 😄

    def test_new_window_and_tab(py: Pylenium):
        # Opens a new, empty browser window and switches to it
        py.switch_to.new_window()
        assert len(py.window_handles) == 2
    
        # Opens a new, empty tab in the current browser window and switches to it
        py.switch_to.new_tab()
        assert len(py.window_handles) == 3

Pass in arbitrary kwargs to WebDriver

v1.13.0 - 2021.08.24

Overview

You can now pass in other, arbitrary keyword arguments (aka "kwargs") to WebDriver. For example, to change the service log path, you would do this:

{
    "driver": {
        "webdriver_kwargs": {"service_log_path": "webdriver.log"}
    }
}

Added

• Pass in arbitrary kwargs to WebDriver through pylenium.json
• Element highlighting with the Element.highlight() method

# highlight the button element
py.get("#button").hightlight()

Driver Capabilities for Firefox

1.12.8 - 2021-07-20

Overview

The Firefox browser can now take a capabilities object!

Thank you @rafutek for adding this feature 💪🏾

Fix race condition with test_results directory

1.12.7 - 2021-06-05

Overview

Fix #130

Made creating the test_results directory more "thread safe" which fixed the intermittent (aka race condition) with the test_run fixture.

Shoutout to @chingc for helping with these fixes 💪🏾 🙏🏾 😄

Fix py.find() method when timeout equals 0

1.12.6 - 2021-05-23

Overview

Fix #196

This is a quick fix to find() method in driver.py, but no other functionality has changed.

py.find(timeout=0) was using

self.webdriver.find_element()

which only returns a single element. This is now fixed to use the appropriate find_elements() method instead.

So py.find() returns an Elements object (aka a list of elements) as expected! All is well in the world again 😄

Fix test_results intermittent errors issue

1.12.5 - 2021-05-21

Overview

Fix #130

The test_run fixture would check for the test_results directory and create it and its subdirectories if needed. However, the previous implementation was not thread-safe and was subject to multiple race conditions.

This should be fixed now that we're using Path from the pathlib module

.select() is now .select_by_*()

1.12.3 - 2021-14-04

Overview

.select() and .select_many() worked pretty well as expected. However, they wouldn't fail as expected! Because we were trying to combine all the "select strategies" in a single function, it made it harder to test and debug and hid some exceptions... like when an <option> didn't exist... whoops!

Now, each strategy has been pulled out into its own method and positive and negative tests have been added to make sure it works 😉

• .select_by_index(index: int)
• .select_by_text(text: str)
• .select_by_value(value)

Changes

• New .select_by_*() methods exist on the Element object. Just remember that the dropdowns have to be a <select> element!
• .select() and .select_many() still exist, but show a DEPRECATED warning. We will be removing them in a future release
• Our docs have a new home! We are still using GitBook, but we now have the official docs.pylenium.io domain! Check it out 😄

Edge Browser for Mac

1.12.2 - 2021-28-01

Overview

Microsoft's Edge Browser can be used on Macs now, but we hadn't tested it locally on a Mac before. One of our amazing users did and they found a bug! This should now be fixed 😄

Fixes

Removed options from webdriver_factory.build_edge() since it isn't needed and was causing the MicrosoftEdgeDriver to raise an error.

Contributors

For anyone looking to contribute, we have changed using pipenv as our package manager to poetry.

• You can find out more about poetry by visiting their website: https://python-poetry.org
• You can see how to setup your machine for Python Development with poetry with my Video on YouTube

Allow users to use local driver path instead of WebDriverManager

1.12.0 - 2020-12-14

Overview

TL;DR
You can update the new config.driver.local_path field in pylenium.json to the path of your locally installed driver executable. If you don't need this, then nothing has changed for you! Keep using Pylenium as usual.

By default, Pylenium would leverage WebDriverManager to automatically install and manage driver binaries for you. However, some use cases required allowing the user to install and manage their own binaries for more granular control. So, this release handles that by introducing a new field into the pylenium.json

Added

config.driver.local_path

The local_path field is now part of the driver object in pylenium.json:

"driver": {
    "local_path": ""
}

By default, Pylenium will continue automatically installing and managing driver binaries for you unless you override this value.

Examples

Globally set for all tests:

"driver": {
    "local_path": "/path/to/geckodriver.exe"
}

Set for a single test:

def test(py):
    py.config.driver.browser = 'chrome'
    py.config.driver.local_path = '/path/to/chromedriver.exe'
    py.visit("https://example.com")
    ...

Accessibility (A11y) Testing with aXe

1.11.0 - 2020-10-30

Overview

Pylenium can now do Accessibility (a11y) Testing and Audits using aXe! Easily generate JSON reports to share and export or write assertions against it directly in your tests.

Added

aXe integration

There are two ways to start using aXe in your tests:

• PyleniumAxe class from pylenium.a11y module
• axe fixture (recommended)

def test_axe_fixture(py, axe):
    py.visit('https://qap.dev')
    # save the axe report as a file
    report = axe.run(name='a11y_audit.json')
    # and/or use the report directly in the test(s)
    assert len(report.violations) == 0

In the above example, we are using Pylenium to navigate to the website and then axe to run the audit, generate the report, and check that we have zero violations!

iframes

The main change here is the ability to to drag and drop within iframes. Pylenium uses jQuery to perform this action, but we need to inject jQuery if the page doesn't already have it. However, in V1 of our jQuery implementation, it would only inject into the main document and not within each iframe. This is now fixed!

• Pylenium's jQuery V2 now comes in its own module and injects into all iframes of the page
• py.switch_to now comes with a py.switch_to.frame_by_element() which is useful when the iframe does not have an id or name attribute

iframe = py.get('iframe')
py.switch_to.frame_by_element(iframe)